Unmanned aerial vehicle battery dynamic management reinforcement learning method for adaptive path planning

Abstract

The invention discloses an unmanned aerial vehicle battery dynamic management reinforcement learning method for adaptive path planning, and relates to the technical field of unmanned aerial vehicles, and the method comprises the steps: activating the multi-modal sensing of an unmanned aerial vehicle at a ground control station, collecting the multi-modal sensing data of the unmanned aerial vehicle, and carrying out the preprocessing; carrying out fusion analysis on the preprocessed multi-modal data by utilizing a deep learning model, generating an environment perception graph, carrying out adaptive path planning by adopting a reinforcement learning algorithm based on the environment perception graph, and dynamically generating an optimal flight path and a battery management strategy deep learning model through the state and environment conditions of the unmanned aerial vehicle; and after the flight task is completed, collecting flight data, environment sensing data and decision records, feeding back to the ground control station, and carrying out iterative training. According to the method, multi-modal perception and deep learning are fused, the path and battery management are optimized through reinforcement learning, and the task efficiency and safety are iteratively improved through group intelligent collaboration.

Description

Technical Field

[0001] The invention relates to the technical field of unmanned aerial vehicles, and in particular to a dynamic management reinforcement learning method for unmanned aerial vehicle batteries with adaptive path planning. Background Art

[0002] The rapid evolution of UAV technology, especially in the field of autonomous navigation and intelligent control, has promoted its widespread application in many industries. However, the path planning and battery management of traditional UAVs are limited by preset routes and fixed modes, and lack the ability to adapt to environmental changes. Although the introduction of multimodal perception technology has enhanced the data collection capabilities of UAVs, the existing technology still has deficiencies in data fusion, real-time path planning, and dynamic battery management, including sensor data fusion bias, inability to dynamically respond to environmental changes, conservative battery management strategies, and inefficient resource allocation when multiple UAVs work together.

[0003] To overcome the above challenges, the present invention proposes a reinforcement learning method for dynamic battery management of unmanned aerial vehicles with adaptive path planning. Through the deep learning model, multimodal perception data is intelligently integrated to generate an environmental perception map, and then the reinforcement learning algorithm is used to plan the optimal path in real time and dynamically adjust the battery management strategy, which significantly improves the autonomous decision-making ability of unmanned aerial vehicles in complex environments. In addition, the swarm intelligence algorithm is combined to optimize the task allocation of multiple unmanned aerial vehicles, realize the efficient utilization of resources and the intelligence of collaborative operations, and comprehensively enhance the performance and reliability of the unmanned aerial vehicle system. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a reinforcement learning method for dynamic management of UAV batteries with adaptive path planning to solve the key problems of real-time perception, intelligent decision-making, efficient path planning and optimized battery usage in complex environments.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In the first aspect, an embodiment of the present invention provides a reinforcement learning method for dynamic management of batteries of unmanned aerial vehicles with adaptive path planning, which includes: activating the multimodal perception of the unmanned aerial vehicle at a ground control station, collecting and preprocessing the multimodal perception data of the unmanned aerial vehicle; using a deep learning model to fuse and analyze the preprocessed multimodal data to generate an environmental perception map; based on the environmental perception map, using a reinforcement learning algorithm to perform adaptive path planning, and dynamically generating an optimal flight path and a deep learning model for battery management strategy through the state of the unmanned aerial vehicle and environmental conditions; uploading the path planning and battery management strategy to a swarm intelligence center node, using a swarm intelligence algorithm to perform global optimization, and generating optimized task allocation and flight strategy;

[0008] Execute the optimized flight strategy, continuously monitor environmental changes and update the environmental perception map in real time, and dynamically adjust the flight path and battery management strategy; after the flight mission is completed, collect flight data, environmental perception data and decision records and feed them back to the ground control station for iterative training.

[0009] As a preferred solution of the UAV battery dynamic management reinforcement learning method for adaptive path planning of the present invention, wherein: the multimodal perception of the UAV is activated at the ground control station, and the multimodal perception data of the UAV is collected and preprocessed, and the specific steps are:

[0010] Before the drone takes off, the drone's multimodal perception is activated at the ground control station. The multimodal perception includes a high-definition camera sensor C, an infrared thermal imager sensor I, a radar sensor R, a sonar sensor S, and a lidar sensor L.

[0011] Sensors begin to collect environmental information data;

[0012] The data is transmitted in real time to the onboard processing unit for pre-processing.

[0013] As a preferred solution of the UAV battery dynamic management reinforcement learning method with adaptive path planning described in the present invention, the deep learning model is used to perform fusion analysis on the pre-processed multimodal data to generate an environmental perception map, and the specific steps are as follows:

[0014] Fusing the pre-processed environmental information data of each sensor;

[0015] The fused data is processed through deep learning of the multimodal fusion network MMFN model to generate an environmental perception map.

[0016] As a preferred solution of the UAV battery dynamic management reinforcement learning method for adaptive path planning of the present invention, wherein: based on the environmental perception map, the reinforcement learning algorithm is used for adaptive path planning, and the optimal flight path and battery management strategy are dynamically generated according to the UAV state and environmental conditions. The specific steps are as follows:

[0017] Based on the environmental perception map, the central controller of the drone uses a reinforcement learning algorithm to plan the path, which is defined as the path planning function P(x), expressed as:

[0018]

[0019] Among them, P(x) is a function of the current state and the environment perception to select the optimal path decision, Q(S, E, a) is a measure called Q function, S s represents the state of the environment, E represents environmental perception, and a represents the action set;

[0020] Taking into account the battery status and predicted energy consumption, the flight mode and speed are intelligently adjusted through path planning to optimize battery usage. The expression is:

[0021]

[0022] Among them, B opt is the optimized battery status, B represents the initial battery status of the drone, T is the flight time, C e (a t ) represents the execution of action a at time t t The energy cost, a t is the action performed by the drone at time step t.

[0023] As a preferred solution of the UAV battery dynamic management reinforcement learning method for adaptive path planning of the present invention, the path planning and battery management strategy are uploaded to the swarm intelligence center node, and the swarm intelligence algorithm is used for global optimization to generate optimized task allocation and flight strategy. The specific steps are as follows:

[0024] In a multi-drone scenario, each drone uploads path planning and battery management strategies to the swarm intelligence central node;

[0025] The central node runs the swarm intelligence algorithm for global optimization and defines the global optimization function G(x), based on the strategy set P of each drone. i and battery status B i , generate optimized task allocation and flight strategy, the expression is:

[0026]

[0027] Among them, G(x) is the output of the global optimization function, i is the index of each drone in the traversal drone group, and P i represents the path planning strategy of the i-th UAV, B i represents the battery status of the i-th drone, D i represents the mission requirement of the i-th UAV, N is the total number of UAVs, and B avg is the average battery status of all drones in the swarm.

[0028] As a preferred solution of the UAV battery dynamic management reinforcement learning method for adaptive path planning of the present invention, the optimized flight strategy is executed, environmental changes are continuously monitored and the environmental perception map is updated in real time, and the flight path and battery management strategy are dynamically adjusted. The specific steps are as follows:

[0029] The drone performs tasks according to the optimized strategy, and multimodal perception continuously monitors environmental changes and updates the environmental perception map in real time;

[0030] The central controller of the drone dynamically adjusts the flight path and battery management strategy according to the new environmental perception map, and defines the dynamic adjustment function D(x), which is expressed as:

[0031]

[0032] Where D(x) represents the dynamic adjustment function, P′ represents the adjusted flight path, and B m represents the battery management strategy based on the new path and environment perception graph, E(X t ) is the predicted environment state based on the original environment perception map, E′(X t ) is the actual environment state based on the updated environment perception map, X t is the position of the drone at time t, T is the number of predicted time steps, and λ P and λ B is the weight of path deviation and battery status adjustment, B′ is the current battery status of the drone, and V is the current speed of the drone.

[0033] As a preferred solution of the UAV battery dynamic management reinforcement learning method for adaptive path planning of the present invention, wherein: after the flight mission is completed, the flight data, environmental perception data and decision records are collected and fed back to the ground control station for iterative training, and the specific steps are as follows:

[0034] After the flight mission is completed, the drone will feed back the flight data, environmental perception data and decision-making process records to the ground control station;

[0035] The ground control station receives the data and pre-processes it;

[0036] The loss function L is used to comprehensively consider the importance of classification tasks and regression tasks, and the expression is:

[0037]

[0038] Among them, L(D c θ old ) is the loss function, CE(y p ,y t ) is the cross entropy loss function, y p is the label predicted by the model, y t is the actual label, is the mean square error loss function, is the battery state predicted by the model, B t is the actual battery state, λ is the balance factor;

[0039] Based on the calculated loss function L, calculate the current model parameter θ old The gradient of , using the learning rate η to update the parameters, is expressed as:

[0040]

[0041] θ new =θ old +Δθ;

[0042] Among them, Δ θ is the parameter update amount, η is the learning rate, is the loss function L with respect to the model parameters θ old The gradient of θ old is the current model parameter, θ new are the updated model parameters;

[0043] Through iterative training of the model, and using the back propagation algorithm to update the model parameters, the model update function M(x) is defined, and the expression is:

[0044]

[0045] Among them, θ new is the updated model parameter, θ old is the model parameter before updating, η is the learning rate, D c is the collected dataset;

[0046] Model updates are repeated in a training loop;

[0047] Finally, the model performance is evaluated on the test dataset.

[0048] As a preferred solution of the UAV battery dynamic management reinforcement learning method for adaptive path planning of the present invention, the ground control station performs preprocessing after receiving the data, and the specific steps are as follows:

[0049] Collect all necessary data sets and conduct preliminary review of the data;

[0050] Delete and fill missing data using mean, median, mode, and use machine learning algorithms to identify and handle missing data;

[0051] The Z-score method is used to identify and handle outliers;

[0052] Check consistency of data types, units, and formats;

[0053] Generate statistical summaries and use visualization tools to explore the underlying distribution and intrinsic structure of data;

[0054] Identify and remove duplicate data records and transform data to the same scale;

[0055] Use one-hot encoding and label encoding to convert categorical variables into machine-readable numerical variables, create new features, and extract seasonal features from time series data;

[0056] Connect the cleaned and converted data tables to integrate different sensor information of UAV remote sensing data;

[0057] After data integration is completed, dimensionality reduction is performed through principal component analysis PCA;

[0058] Apply filtering, wrapping, and embedding methods to select the most predictive subset from a large number of features;

[0059] Finally, the dataset is divided into training set, validation set and test set.

[0060] In a second aspect, an embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the reinforcement learning method for dynamic management of drone batteries with adaptive path planning as described in the first aspect of the present invention is implemented.

[0061] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the reinforcement learning method for dynamic management of batteries of unmanned aerial vehicles with adaptive path planning as described in the first aspect of the present invention.

[0062] The beneficial effects of the present invention are as follows: through multimodal perception and preprocessing, the environmental adaptability of the UAV is enhanced, deep learning fusion analysis is used to generate an environmental perception map, the path planning accuracy is improved, reinforcement learning is used to dynamically adjust the flight and battery strategies, the endurance and mission efficiency are optimized, the swarm intelligence is globally optimized, and efficient collaboration of multiple machines is achieved. Dynamic adjustment of strategies ensures flexible response to changes, iterative training continuously improves the system intelligence, and enhances the accuracy of decision-making. This invention significantly improves the safety, efficiency and intelligence level of UAV mission execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0064] Figure 1 This is a flow chart of the reinforcement learning method for dynamic management of UAV batteries with adaptive path planning in Example 1.

[0065] Figure 2This is a flow chart for collecting multimodal perception data of drones in Example 1. DETAILED DESCRIPTION

[0066] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0067] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0068] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0069] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a reinforcement learning method for dynamic management of UAV batteries with adaptive path planning, comprising the following steps:

[0070] S1. Activate the multimodal perception of the UAV at the ground control station, collect the multimodal perception data of the UAV and perform preprocessing.

[0071] Furthermore, before the drone takes off, the multimodal perception of the drone is activated at the ground control station. The multimodal perception includes a high-definition camera sensor C, an infrared thermal imager sensor I, a radar sensor R, a sonar sensor S, and a lidar sensor L;

[0072] Sensors begin to collect environmental information data;

[0073] Environmental information includes: terrain, obstacles, weather conditions, wind direction and speed, etc.;

[0074] The data is transmitted in real time to the onboard processing unit for pre-processing;

[0075] First, the data is standardized, and the expression is:

[0076]

[0077] Among them, Z i is the standardized data, X i is the raw sensor data, μ i is the mean value of the original data, σi is the standard deviation of the original data;

[0078] Then the standardized data is time synchronized, the expression is:

[0079]

[0080] Among them, T sync is the timestamp after synchronization, that is, the common time point when the difference between the timestamps of the two sensors is the smallest. tsensor1 is the timestamp of the first sensor, tsensor2 is the timestamp of the second sensor, and argmin t is to find the value of t that minimizes the internal function value, |||·||| 2 It is the square of the vector's two-norm, that is, the sum of the squares of the differences of each element;

[0081] After the data is synchronized, the data needs to be denoised. The denoising method is determined based on different data. The most commonly used methods are: Kalman filtering, median filtering and wavelet transform.

[0082] Kalman filter is a recursive filter, which is particularly suitable for state estimation problems in dynamic systems. The expression is:

[0083] Prediction Steps:

[0084]

[0085] P k|k-1 =AP k-1|k-1 A T +Q;

[0086] in, is the prediction of the state at time k based on the information at time k-1. A is the state transfer matrix, which describes how the system state transfers from the previous moment to the next moment. B is the control input matrix. If the system is controlled externally, then u k is the control vector. If there is no external control, B can be ignored and u k The control input vector acting on the system, is the state estimate based on all available information at time k-1, P k∣k-1 is the predicted state error covariance matrix, P k-1∣k-1 is the state error covariance matrix based on all available information at time k-1, Q is the covariance matrix of process noise, describing the uncertainty of the system state changes between time steps;

[0087] Update steps:

[0088] K k =P k|k-1 HT (HP k|k-1 H T +R) -1 ;

[0089]

[0090] P k|k =(IK k H)P k|k-1 ;

[0091] Among them, K k is the Kalman gain, which determines the relative importance of measurements when updating the state estimate, H is the observation matrix, which describes the relationship between the system state and the observations, and z k is the measured value at time k, is the state estimate based on all available information at time k, R is the covariance matrix of the measurement noise, describing the uncertainty of the measurement error, and I is the identity matrix, which is related to K k H multiplication is used to update the state error covariance matrix.

[0092] Median filtering is a nonlinear filtering method, mainly used to eliminate impulse noise, and the expression is:

[0093] y t =median{x t-(n-1) / 2 ,…,x t+(n-1) / 2};

[0094] Among them, y t is the data value after median filtering at time point t, median represents the median, x t-(n-1) / 2 to x t+(n-1) / 2 are all the data points within the window;

[0095] Wavelet transform is used for multi-scale analysis and can effectively identify and remove high-frequency noise. The expression is:

[0096]

[0097] in, is the wavelet coefficient after thresholding after decomposition at the jth layer, c j (k) is the wavelet coefficient of the jth layer decomposition, T is the threshold used to decide which wavelet coefficients are retained and which are set to zero. Usually T is determined based on the noise level and signal characteristics. j is the level of wavelet decomposition. Wavelet transform is usually performed at multiple scales. k is the index position of the wavelet coefficient at the jth layer.

[0098] The missing value processing is performed using linear interpolation on the denoised data. The expression is:

[0099]

[0100] Among them, X interpolated (t) is the interpolated data at time t, X(t1) is the known data point at time t1, X(t2) is the known data point at time t2, t1 is the time point closest to t that is earlier than t, t2 is the time point closest to t that is later than t, and t is the time point that needs to be interpolated;

[0101] Finally, convolutional neural network CNN is used for feature extraction to complete data preprocessing. The expression is:

[0102] F(X) = f(W*X+b);

[0103] Among them, F(X) is the feature map, that is, the output after the input data X passes through the neural network layer, W is the weight matrix, which represents the connection strength of the neural network layer, X is the input data, which can be the original data or the output of the previous layer, b is the bias term, which is similar to the intercept in mathematics and is used to translate the activation function, f is the activation function, and * is the convolution operator, which represents the convolution operation;

[0104] It should be noted that this series of preprocessing steps improves the quality and consistency of raw sensor data and provides accurate data support for the drone's autonomous flight, environmental perception, and decision-making. Through these steps, the drone can understand the surrounding environment more accurately and improve the safety and efficiency of its mission execution.

[0105] S2. Use the deep learning model to fuse and analyze the preprocessed multimodal data to generate an environmental perception map.

[0106] Furthermore, the pre-processed environmental information data of each sensor is fused, and the expression is:

[0107]

[0108] Among them, Data f is the fused data output, α represents the weight coefficient of the HD camera sensor C data, β represents the weight coefficient of the infrared thermal imager sensor I data, γ represents the weight coefficient of the radar sensor R data, δ represents the weight coefficient of the sonar sensor S data, ∈ represents the weight coefficient of the lidar sensor L data, C represents the output data of the HD camera, B represents the output data of the infrared thermal imager, R represents the output data of the radar, S represents the output data of the sonar, and L represents the output data of the lidar. This sum is used to normalize the weighted sum in the numerator to ensure that the fused data Data f There will be no skewness due to different weights;

[0109] The fused data is processed through deep learning by the multimodal fusion network MMFN model to generate an environmental perception map, which is defined as the environmental perception function E(x) and is expressed as:

[0110] E(x)=MMFN(Data f );

[0111] Among them, E(x) is the environment perception function, MMFN represents the multimodal fusion network model, and Data f is the fused data;

[0112] Furthermore, the multimodal fusion network MMFN model is a deep learning model that processes and integrates multiple types of data from different sensors. This model is particularly important in the fields of drones, autonomous vehicles, robots, etc., because it can obtain comprehensive information from multiple angles and sensory inputs to improve the understanding and response capabilities of the environment. The design goal of the multimodal fusion network MMFN model is to enable machines to use a variety of sensory information like humans to form a comprehensive understanding of the surrounding environment. This can not only enhance the robustness and accuracy of the system, but also provide redundant information in the event of some sensor failure or poor data quality, ensuring the stable operation of the system.

[0113] It should be noted that after completing the preprocessing of the multimodal perception data, a weighted average fusion technology was adopted, which comprehensively considered the outputs of sensors such as high-definition cameras, infrared thermal imagers, radars, sonars and lidars, and generated a unified fusion data output by assigning specific weight coefficients to different sensor data. This fusion step provides high-quality, multi-dimensional information input for the subsequent deep learning model, ensuring the accuracy and comprehensiveness of the environmental perception map. Next, the fused data is input into the multimodal fusion network MMFN model. Through deep learning processing, a detailed environmental perception map is finally generated, which provides a key basis for the intelligent decision-making and navigation of the UAV. This series of operations not only enhances the reliability and applicability of the data, but also lays a solid foundation for subsequent analysis and application.

[0114] S3. Based on the environmental perception map, a reinforcement learning algorithm is used for adaptive path planning, and the optimal flight path and battery management strategy are dynamically generated according to the drone status and environmental conditions.

[0115] Furthermore, based on the environmental perception map, the central controller of the drone uses a reinforcement learning algorithm to perform path planning, which is defined as the path planning function P(x), expressed as:

[0116]

[0117] Among them, P(x) is a function of the current state and environmental perception to select the optimal path decision, It is a mathematical operator, which means to find the action that maximizes the value of the function in the subsequent brackets among all possible actions a. Q(S, E, a) is a measure called Q function, which is used to evaluate the expected benefit of taking action a under a given state S and environmental perception E. S s represents the state of the environment, E represents environmental perception, and a represents the action set;

[0118] Furthermore, reinforcement learning is a machine learning method that enables an agent to learn behavioral strategies in the process of interacting with the environment. The goal of reinforcement learning is to find a strategy that enables the agent to maximize the cumulative reward in a given environment. In this process, the agent learns by observing the state of the environment, selecting actions, and receiving the model library, reward and punishment signals. Over time, the agent improves its decision-making process to obtain higher cumulative rewards.

[0119] Taking into account the battery status and predicted energy consumption, the flight mode and speed are intelligently adjusted through path planning to optimize battery usage. The expression is:

[0120]

[0121] Among them, B opt is the battery status of a single drone after optimization, is a mathematical operator, which means to select the action that minimizes the expression value in the brackets among all possible action sequences a. B represents the initial battery state of the drone, that is, the remaining battery power at the beginning of the mission. is a summation expression that represents the cumulative predicted energy consumption from the start of the mission to the end of time T, where T is the flight time and C e (a t ) represents the execution of action a at time t t The energy cost, a t is the action performed by the drone at time step t;

[0122] It should be noted that in drone technology, efficient path planning and energy management are crucial to completing the mission. The central controller of the drone uses a reinforcement learning algorithm to analyze the environmental perception map and make the best path decision from it. This process is implemented through the path planning function P(x), which uses the argmax operator to select the one that can maximize the Q function from all possible actions a. The Q function here evaluates the expected return of taking a specific action a under a given environmental state S and environmental perception E; at the same time, in order to optimize battery usage, the drone also calculates an optimized battery state B opt, which involves minimizing the difference between the remaining battery capacity B and the predicted energy consumption. By intelligently adjusting the flight mode and speed, the drone can ensure that the battery energy is used most efficiently during the mission and avoid unnecessary energy waste. This optimization is achieved by finding a solution that minimizes battery loss among all possible action sequences a, thereby ensuring that the drone can complete its mission within the scheduled time without running out of power.

[0123] S4. Upload the path planning and battery management strategy to the swarm intelligence center node, use the swarm intelligence algorithm to perform global optimization, and generate optimized task allocation and flight strategy.

[0124] Furthermore, in a multi-UAV scenario, each UAV uploads the path planning and battery management strategy to the swarm intelligence central node;

[0125] The central node runs the swarm intelligence algorithm for global optimization and defines the global optimization function G(x), based on the strategy set P of each drone. i and battery status B i , generate optimized task allocation and flight strategy, the expression is:

[0126]

[0127] Among them, G(x) is the output of the global optimization function, where (x) represents the optimization output, and G(x) as a whole represents the optimized task allocation and flight strategy, ensuring efficient collaboration of the entire drone group and balanced use of resources. Is a mathematical operator, indicating that among all possible path planning P i and battery status B i In the combination, find the set P that minimizes the value of the function in the brackets i and B i , i is the index of each drone in the traversal drone group, P i represents the path planning strategy of the i-th UAV, B i represents the battery status of the i-th swarm drone, D i represents the mission requirement of the i-th UAV, N is the total number of UAVs, that is, the number of UAVs participating in the mission, and B avg is the average battery status of all drones in the swarm, -P i ) represents the mission requirements of all drones D i and their actual path planning P i The total gap between Indicates the battery status of all drones B i With the average battery state B avg The sum of squared deviations between .

[0128] It should be noted that the global optimization function G(x) is responsible for determining an optimal task allocation and flight strategy based on the strategy set Pi and battery status Bi of all drones before the mission begins. The result of this step, namely the optimized task allocation and flight strategy, will become the basis for the drone to start executing the mission;

[0129] The swarm intelligence central node usually refers to a core computing entity or a logically centralized decision point. It is responsible for collecting and processing information from all drones, including but not limited to location, speed, remaining battery, sensor data, mission status, etc., and using swarm intelligence algorithms to coordinate the actions between drones. This central node can be a physical device, such as a ground station, satellite or cloud server, or a logical component in the software architecture;

[0130] The process of the central node running the swarm intelligence algorithm for global optimization is divided into several steps, including data collection, evaluation, optimization and decision-making. The detailed process is as follows:

[0131] The central node first collects real-time data from each drone, including:

[0132] Path planning strategy Pi: the path that each drone plans to execute;

[0133] Battery status Bi: the current battery level of each drone;

[0134] Mission demand Di: the amount of mission each drone needs to complete or the energy consumption forecast;

[0135] After collecting all necessary data, the central point calculates the global optimization function G(x);

[0136] Next, the central node uses a swarm intelligence algorithm to iteratively optimize G(x), which involves the following algorithm and its variants;

[0137] Genetic Algorithm: Evolves the UAV’s strategy set and battery status through crossover, mutation, and selection operations to find the configuration that minimizes G(x);

[0138] Particle swarm optimization (PSO): Particles, representing the strategies of drones, move in the search space and find the optimal solution by updating speed and position;

[0139] Ant Colony Optimization (ACO): It simulates the pheromone trails left by ants when searching for food to optimize the path planning and battery management strategies of drones.

[0140] In each iteration, the swarm intelligence algorithm evaluates the impact of the current strategy set on G(x) and adjusts the strategy;

[0141] When the algorithm reaches a satisfactory solution, the central node generates optimized task allocation and flight strategies; these strategies are then distributed to each drone to update their flight plans and battery management strategies;

[0142] After the drones execute the optimized strategy, the central node continues to monitor the status of the group, collect new data and prepare for the next round of optimization cycle. If the environment or mission conditions change, the central node may need to re-run the swarm intelligence algorithm to adapt to the new situation;

[0143] It should be noted that through these algorithms, the swarm intelligence central node can dynamically adjust the flight strategy and task allocation of drones to ensure that the entire drone group performs tasks efficiently and safely, while maximizing the working time and task completion rate of the entire group.

[0144] S5. Execute the optimized flight strategy, continuously monitor environmental changes and update the environmental perception map in real time, and dynamically adjust the flight path and battery management strategy.

[0145] Furthermore, the drone performs tasks according to the optimized strategy, and multimodal perception continuously monitors environmental changes and updates the environmental perception map in real time;

[0146] The central controller of the drone dynamically adjusts the flight path and battery management strategy according to the new environmental perception map, and defines a dynamic adjustment function D(x) to ensure that the drone can cope with emergencies. The expression is:

[0147]

[0148] Where D(x) represents a dynamic adjustment function, where (x) represents the dynamic adjustment of real-time data, and D(x) as a whole represents the function used to determine the optimal flight path and battery management strategy. Represents the operator, determines the optimal flight path P′ and battery management strategy B m , so that the weighted sum of the path adjustment error and the battery state deviation is minimized, P′ represents the adjusted flight path, B m represents the battery management strategy based on the new path and environment perception graph, represents the summation operator, which means the sum of the number of predicted time steps from t=1 to T over all time steps t (E′(Xt)-E(Xt)) 2 To sum, E(X t ) is the predicted environment state based on the original environment perception map, E′(X t ) is the actual environment state based on the updated environment perception map, X t is the position of the UAV at time t, T is the number of predicted time steps, which is used for the forward-looking path planning, and λ P and λ Bis the weight of path deviation and battery state adjustment, which is used to balance the importance of the two. B′ is the current battery state of the drone, V is the current speed of the drone, and |·| represents the absolute value operation, which is used to calculate the battery state deviation. The denominators λP and λB are used to normalize the path deviation and battery state deviation to ensure that they are compared on the same scale.

[0149] It should be noted that the dynamic adjustment function D(x) is used to adjust the flight path and battery management strategy when the UAV encounters unexpected situations or environmental changes during the mission execution. Here, P′ and B′ can be regarded as modified versions of the original strategy and state output by G(x), that is, the input of D(x) is adjusted based on the output of G(x);

[0150] Although it is not explicitly stated in the mathematical formula that D(x) is directly dependent on G(x), in fact, the work of D(x) is performed on the basis of G(x), which means that D(x) receives the initial conditions provided by G(x) and makes real-time adjustments based on them. The two cooperate with each other at different stages of the UAV mission execution to form a complete decision-making process;

[0151] To better understand this connection, we can imagine that G(x) determines a set of task assignments and flight strategies, and D(x) adjusts these strategies in real time according to environmental changes during the execution of these tasks. For example, if the battery status of a drone is lower than expected, the B predicted by G(x) will be i , then D(x) will be based on the current battery state B′ and the energy requirement E′(X t ) to adjust the flight path and mission execution sequence of the UAV to ensure the efficiency and safety of the entire system;

[0152] Multimodal perception continuously monitors environmental changes, such as the appearance of obstacles and changes in wind speed, and updates the environmental perception map in real time;

[0153] Based on the updated environment perception map, the path deviation at each predicted time point t is evaluated;

[0154] Based on the new flight path P^' and the environmental perception map E', the battery management strategy B is calculated. m , and evaluate the deviation from the current battery state B′;

[0155] Use the dynamic adjustment function D(x) to determine the optimal P^' and B m , which requires solving an optimization problem, usually achieved through numerical methods or heuristic algorithms;

[0156] The drone performs the mission according to the calculated optimal flight path and battery management strategy, while continuing to monitor environmental changes and preparing for the next round of dynamic adjustments;

[0157] The goal of the dynamic adjustment function D(x) is to find a set of flight paths and battery management strategies that minimize the weighted sum of path adjustment error and battery state deviation. This means that the drone needs to minimize path deviation caused by environmental changes while maintaining the original mission execution, and make the best decision on battery state management to avoid premature power exhaustion.

[0158] Through such a dynamic adjustment mechanism, drones can respond to complex mission environments more flexibly and autonomously, improving the efficiency and success rate of mission execution.

[0159] S6. After the flight mission is completed, the flight data, environmental perception data and decision records are collected and fed back to the ground control station for iterative training.

[0160] Furthermore, after the flight mission is completed, the drone will feed back the flight data, environmental perception data and decision-making process records to the ground control station;

[0161] The ground control station receives the data and pre-processes it to make it suitable for input into the machine learning model;

[0162] The loss function L is used to comprehensively consider the importance of classification tasks and regression tasks, and the expression is:

[0163]

[0164] Among them, L(D c θ old ) is the loss function, which measures the model's performance in the dataset D c The difference between the predicted and actual values, CE(y p ,y t ) is the cross entropy loss function, which is used for classification tasks. It measures the label y predicted by the model. p With the actual label y t The difference between p is the label predicted by the model, that is, the model's classification prediction of environmental obstacles, y t is the actual label, i.e. the actual obstacle category, is the mean square error loss function, which is used for regression tasks and quantifies the battery state predicted by the model. The actual battery status B t The gap between is the battery state predicted by the model, B tis the actual battery state, λ is a balancing factor between 0 and 1, which is used to adjust the relative weights of the classification task and the regression task in the total loss function;

[0165] Based on the calculated loss function L, calculate the current model parameter θ old The gradient of , using the learning rate η to update the parameters, minimizes the loss function, expressed as:

[0166]

[0167] θ new =θ old +Δθ;

[0168] Among them, Δ θ is the parameter update amount, that is, the model parameters are updated from θ old to θ new The amount of change, η is the learning rate, which controls the step size of parameter update, is the loss function L with respect to the model parameters θ old The gradient of θ old is the current model parameter, the value before the parameter update, θ new is the updated model parameter, that is, the value after the parameter is updated;

[0169] Through iterative training of the model, and using the back propagation algorithm to update the model parameters, the model performance is improved, and the model update function M(x) is defined, and the expression is:

[0170]

[0171] Among them, θ new is the updated model parameter, θ old is the model parameter before the update, that is, the model parameter value before this update, η is the learning rate, a hyperparameter used to control the step size of each gradient descent update, is the loss function L(D c θ old ) in the parameter θ old The gradient of θ is the partial derivative vector of the loss function with respect to the model parameters, which measures the model's performance in the dataset D. c The difference between the prediction and the true label, D c is the collected dataset;

[0172] The model update is repeated in the training loop until the predetermined number of training epochs is reached and the loss function value no longer decreases significantly;

[0173] Finally, the model performance is evaluated on the test dataset to ensure that the model has good generalization ability;

[0174] Furthermore, the ground control station performs pre-processing after receiving the data.

[0175] Collect all necessary data sets and conduct a preliminary review of the data to understand the basic structure and size;

[0176] Delete and fill missing data using mean, median, mode, and use machine learning algorithms to identify and handle missing data;

[0177] The Z-score method is used to identify and handle outliers;

[0178] Check the uniformity of data types, units and formats, and determine the consistency of data fields;

[0179] Generate statistical summaries and use visualization tools to explore the underlying distribution and intrinsic structure of data;

[0180] Identify and remove duplicate data records, convert data to the same scale, and normalize;

[0181] Use one-hot encoding and label encoding to convert categorical variables into machine-readable numerical variables, create new features, and extract seasonal features from time series data;

[0182] Connect the cleaned and converted data tables to integrate different sensor information of UAV remote sensing data;

[0183] After data integration is completed, principal component analysis (PCA) is used to reduce the dimensionality of the data, reduce the number of features, and avoid the dimensionality disaster.

[0184] Apply filtering, wrapping, and embedding methods to select the most predictive subset from a large number of features;

[0185] Finally, the data set is divided into training set, validation set and test set for model training and testing;

[0186] It should be noted that flight data mainly includes various status information of the drone during flight, such as:

[0187] Location information: GPS coordinates, recording the exact location of the drone in the air;

[0188] Speed ​​and acceleration: reflects the speed and direction of the drone's movement, as well as the change in acceleration, which is very important for understanding and analyzing the drone's maneuverability;

[0189] Attitude data: including pitch angle, roll angle and heading angle, which help to understand the attitude stability of the drone;

[0190] Altitude information: usually measured by sensors such as barometers or lidar, used to ensure that the drone flies at a specified altitude;

[0191] Flight mode and instructions: record the flight mode used by the drone during flight, such as manual, automatic, return, etc., and the control instructions received;

[0192] Environmental perception data is the surrounding environment information collected by drones through various sensors. This data is crucial for drone mission planning, flight safety and environmental monitoring. Common environmental perception data includes:

[0193] Meteorological data: such as temperature, humidity, air pressure, wind speed and direction, which can help UAV systems predict flight conditions and make safer flight decisions;

[0194] Obstacle detection: Obstacle information detected by LiDAR, infrared camera or visual sensor for obstacle avoidance and flight path planning;

[0195] Terrain and landform information: used to build high-precision maps to help drones navigate in complex terrain;

[0196] Biological and chemical data: In environmental monitoring missions, drones may carry sensors to collect data on air quality and biodiversity;

[0197] During the mission execution, the drone will make decisions based on its perceived environment and mission requirements. The decision record may include:

[0198] Path planning and re-planning: Record how the drone adjusts its flight path based on environmental changes;

[0199] Energy management: battery status monitoring and optimized flight mode to save energy;

[0200] Task priority adjustment: In a multi-task environment, how does the drone adjust the execution order according to the urgency of the task?

[0201] Emergency response: Emergency decision-making and response of drones in the event of emergencies, such as equipment failure and bad weather;

[0202] After collecting these data, we start to conduct data feedback;

[0203] The collected data is used to train and optimize the drone’s autonomous flight algorithms, including path planning, obstacle avoidance, energy management, and environmental adaptability;

[0204] By analyzing flight data, we can evaluate the performance of the drone, identify potential failure points, and improve the stability and reliability of the system;

[0205] Based on historical flight data, it helps to optimize future mission planning, especially when performing missions in complex or dangerous environments, the flight path can be simulated and planned in advance;

[0206] It should be added that in some countries and regions, flight data recording is necessary for compliance with aviation regulations and accident investigations;

[0207] It should be noted that the preprocessing steps are:

[0208] First, all UAV flight data, environmental perception data, and decision-making process records are collected and transmitted to the ground control station;

[0209] Suppose we have multiple datasets D1, D2, ..., D n , collect all data sets, merge data sets, the expression is:

[0210]

[0211] Among them, D represents the collected and merged data set, D i represents the i-th individual data set, where i ranges from 1 to n, i.e. D1, D2, ..., D n , ∪ is a set union symbol;

[0212] Then clean the data;

[0213] Using mean filling, the expression is:

[0214]

[0215] Among them, D′ is the data set after missing values ​​are filled, and D represents the original data set. represents the mean of the jth column, that is, the average of all non-missing values. M represents the missing value indicator matrix, which has the same dimension as D. When the value at the corresponding position in D is missing, the value in M ​​is 1, otherwise it is 0;

[0216] Using median filling, the expression is:

[0217] D′ ij =D ij ·(1-M ij )+d j,～ ·M ij ;

[0218] Among them, Di′j represents the element in the i-th row and j-th column of the new data set after median filling, D ij represents the element in the i-th row and j-th column of the original data set, d j , ~ represents the median of the jth column, M ij Indicates the element in the i-th row and j-th column of the missing value indicator matrix. If D ij If missing, it is 1, otherwise it is 0;

[0219] Using majority filling, the expression is:

[0220]

[0221] Where D* represents the data set after mode filling, and md is the mode vector;

[0222] The Z-score method is used to identify outliers. The expression is:

[0223]

[0224] Where Z is the standardized score of outliers, d j is a single observation in the data set, μ is the mean of the data set, and σ is the standard deviation of the data set;

[0225] Outlier processing, the expression is:

[0226]

[0227] Among them, f(z) is a function used to deal with outliers, z represents the Z-score standard score of the data point, z td is the outlier threshold, and the Z-score standard score exceeding this threshold is considered anomaly, and else represents other than this;

[0228] The data is normalized and the expression is:

[0229]

[0230] Among them, d i ' is the normalized data point, d i is the original data point, min(D) is the minimum value in the data set D, and max(D) is the maximum value in the data set D;

[0231] Data type format consistency. For the date format, use Pandas' to_datetime function to convert it to the datetime type. The expression is:

[0232] D date =pd.to d atetime(D date_str );

[0233] Among them, D date is the converted date data, D date_str The date data needs to be converted into string data of numeric type;

[0234] For numeric data, use to_numeric to convert it to a number. The expression is:

[0235] D num =pd.to n umeric(D str ,errors=′coerce′);

[0236] Among them, D num is the converted data, which will be stored as a numeric type. str It is the string data that needs to be converted to a numeric type;

[0237] Identify and delete duplicate data records. The expression is:

[0238] D ue =Ue(D);

[0239] Among them, D ue Represents the data set after deleting duplicate records, U e (D) represents a function used to extract unique records in D;

[0240] Perform feature engineering;

[0241] Applying one-hot encoding, the expression is:

[0242] D one-hot =pd.get d ummies(D cat ,prefix=D cat .name);

[0243] Among them, D one-hot It is the data after one-hot encoding, which converts the categorical variable into a binary vector. Each category corresponds to a new column with only 0 or 1 in the column. cat This represents the categorical variable data in the dataset.

[0244] Define a function label_encode to map categorical variables to integers. The expression is:

[0245]

[0246] Where L is the label encoding function, {c1, c2, ..., c m} are all possible values ​​of the categorical variable, and {0, 1, ..., m-1} are the corresponding integer values;

[0247] Next, apply label encoding to the categorical variable C of the dataset D, expressed as:

[0248]

[0249] Feature creation, the expression is:

[0250] D month =Month(D date );

[0251] Among them, D month is the newly created month feature, which will contain the date Month(·) is a function that receives date data as input and returns the month value corresponding to each date. date is the original date feature, which contains the timestamp or date information of each sample in the dataset;

[0252] Dataset collection;

[0253] Merge multiple datasets into one. Suppose we have multiple datasets D1, D2, ..., D n , we need to integrate them together based on the common key k. If datasets D1 and D2 have a common key k, the expression is:

[0254]

[0255] Among them, D integrated represents the merged set of data, represents connection, k represents the common key;

[0256] The feature selection expression is:

[0257] F selected =RFE(E,n)(X,y);

[0258] Among them, F selected It is the feature subset selected after the feature selection process. RFE(E,n): is a function call, where E is a machine learning model or estimator, n is the number of features you want to retain, X is the feature matrix, and y is the target variable vector corresponding to the true value of each sample in X;

[0259] The F in this step is actually converted according to its name. F is the abbreviation of Features, so the feature selection step must be expressed by F;

[0260] Perform PCA dimensionality reduction, the expression is:

[0261] X reduced =XW k ;

[0262] Among them, X reduced is the data matrix after dimensionality reduction, W kIt is a matrix composed of the first k principal components. This step assumes that the feature matrix of the data set D is X. The goal of PCA is to find a projection matrix W that projects the data into a lower-dimensional space while retaining as much information as possible. W consists of the first k principal components of the covariance matrix of X.

[0263] It should be noted that missing data are deleted and filled with mean, median, and mode. Random forest algorithm and K-nearest neighbor algorithm are used to identify and process missing data. If the data set is large and there are complex relationships between features, then random forest is a good choice. If the data set is small and needs to be processed quickly, K-nearest neighbor algorithm is more suitable.

[0264] Choose Tableau as the visualization tool. The ease of use and powerful functionality of Tableau make it a very popular choice in enterprise environments. It can handle large data sets and provide various types of visualizations, from simple bar charts to complex geospatial analysis.

[0265] One-hot encoding is a method of converting categorical variables into binary vectors. For each possible category of each categorical variable, one-hot encoding creates a new binary variable. This new variable is called a dummy variable or indicator variable. If an observation belongs to that category, the corresponding dummy variable has a value of 1, otherwise, its value is 0.

[0266] Label encoding is a simple encoding method that maps each category to a unique integer. For example, if there are three categories red, green, blue, label encoding may convert them into 0, 1, 2. This encoding method is suitable for situations where there is some order between categories, but even if there is no order between categories, label encoding is often used to simplify data processing;

[0267] The purpose of the entire step is to build and optimize an artificial intelligence system that can make drone autonomous flight safer and more efficient. By collecting flight data, environmental perception data and decision records, we can train the model to better understand the environment and make more accurate decisions. Preprocessing data and feature engineering help the model not only perform well on training data, but also maintain stable and reliable performance on new data that has not been seen. This is achieved by removing noise, handling missing values, standardizing data and selecting important features. Through dimensionality reduction and feature selection, unnecessary computational burden is reduced, making the model run faster while reducing the risk of overfitting. Continuous training and evaluation of the model ensure that the drone can safely perform tasks in a changing environment and reduce potential accident risks. As the performance of the model improves, the drone can make decisions more independently, reduce dependence on external control, and improve the flexibility and efficiency of task execution. By feeding back data to the ground control station, the model can be continuously iterated and trained to continuously optimize its performance and adapt to new challenges and environmental changes. The implementation of this series of steps is to build a high-precision, efficient and reliable drone autonomous flight system that can operate autonomously in complex and changing environments, reduce the need for human intervention, and thus achieve a wider range of application scenarios and a higher mission success rate.

[0268] This embodiment also provides a computer device, which is suitable for the case of a reinforcement learning method for dynamic management of batteries of unmanned aerial vehicles with adaptive path planning, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the reinforcement learning method for dynamic management of batteries of unmanned aerial vehicles with adaptive path planning as proposed in the above embodiment.

[0269] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0270] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for dynamic management and reinforcement learning of unmanned aerial vehicle batteries for adaptive path planning proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk.

[0271] In summary, the present invention clarifies the defects of the prior art that the present invention overcomes by emphasizing complex dynamic environments, traditional path planning and battery management static strategies, as well as poor adaptability and low resource utilization, namely, how to effectively plan paths and manage battery resources under constantly changing conditions to achieve better task execution efficiency and energy utilization efficiency, and solve the problems of poor adaptability and low resource utilization of UAVs in complex dynamic environments due to traditional path planning and battery management static strategies.

[0272] Example 2, referring to Table 1, is the second example of the present invention. To further verify the advancement of the present invention, experimental simulation data of a reinforcement learning method for dynamic management of UAV batteries with adaptive path planning are provided.

[0273] The method of the present invention and the prior art are used to perform UAV missions. The experiment is set in a complex urban environment, including obstacles such as buildings, trees, pedestrians, and vehicles, as well as changeable weather conditions, such as wind speed changes. The experiment involves a total of 10 UAVs, each equipped with a high-definition camera sensor C, an infrared thermal imager sensor I, a radar sensor R, a sonar sensor S, and a lidar sensor L. The UAV missions include environmental monitoring, cargo transportation, and emergency rescue.

[0274] First, the preprocessing stage: collect multimodal perception data of the drone and perform preprocessing, including standardization, time synchronization, denoising, missing value processing and feature extraction. Kalman filtering, median filtering and wavelet transform are used for denoising, missing values ​​are processed by linear interpolation, and convolutional neural network (CNN) is used for feature extraction.

[0275] Fusion analysis: Fusion of pre-processed data, generating an environmental perception map through a multimodal fusion network (MMFN);

[0276] Secondly, path planning and battery management: Based on the environmental perception map, a reinforcement learning algorithm is used for path planning. At the same time, the optimal flight path and battery management strategy are dynamically generated according to the drone status and environmental conditions.

[0277] Next, global optimization: upload the strategy to the swarm intelligence center node, optimize it through the global optimization function G(x), and generate optimized task allocation and flight strategy; execution and adjustment: execute the optimized flight strategy, continuously monitor environmental changes, update the environmental perception map in real time, and dynamically adjust the flight path and battery management strategy.

[0278] Finally, iterative training: After the flight mission is completed, flight data, environmental perception data and decision records are collected and fed back to the ground control station for iterative training to optimize model performance.

[0279] The details are shown in Table 1 below:

[0280] Table 1 Experimental record table

[0281]

[0282] Through the comparison of the above test data, it can be clearly seen that the method of the present invention shows significant advantages in multiple key indicators. Flight time: The average flight time of the drone using the method of the present invention is increased by 30% compared with the prior art, indicating that the present invention improves the efficiency in path planning and battery management strategy, allowing the drone to perform longer tasks; Battery consumption: The battery consumption of the drone under the method of the present invention is reduced by 15%, indicating that the present invention is more optimized in battery management strategy and reduces unnecessary energy waste;

[0283] Path deviation: The path deviation of the method of the present invention is only 15 meters, which is much lower than the 50 meters of the prior art. This shows that the present invention is more accurate in environmental perception and path planning, and improves the accuracy and safety of flight. Mission completion rate: The mission completion rate of the UAV using the method of the present invention is as high as 98%, which significantly improves the success rate of the mission compared to 80% of the prior art, indicating that the present invention is more efficient and reliable in overall mission execution.

[0284] In summary, the present invention realizes intelligent path planning and dynamic battery management of UAVs in complex environments through the combination of deep learning models and reinforcement learning algorithms, which not only improves the efficiency and safety of task execution, but also significantly improves the battery utilization efficiency, proving the innovation and practicality of the present invention in the field of UAV technology.

[0285] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A reinforcement learning method for dynamic battery management of unmanned aerial vehicles with adaptive path planning, characterized by: include, Activate the multimodal perception of the UAV at the ground control station, collect the multimodal perception data of the UAV and perform preprocessing; Use deep learning models to fuse and analyze preprocessed multimodal data to generate an environmental perception map; Based on the environmental perception map, the reinforcement learning algorithm is used for adaptive path planning, and the optimal flight path and battery management strategy are dynamically generated according to the drone status and environmental conditions; Upload the path planning and battery management strategy to the swarm intelligence center node, use the swarm intelligence algorithm to perform global optimization, and generate optimized task allocation and flight strategy; Execute optimized flight strategies, continuously monitor environmental changes and update environmental perception maps in real time, and dynamically adjust flight paths and battery management strategies; After the flight mission is completed, flight data, environmental perception data and decision records are collected and fed back to the ground control station for iterative training.

2. The method for dynamic management of unmanned aerial vehicle battery with adaptive path planning according to claim 1, characterized in that: The multimodal perception of the UAV is activated at the ground control station, and the multimodal perception data of the UAV is collected and preprocessed. The specific steps are: Before the drone takes off, the drone's multimodal perception is activated at the ground control station. The multimodal perception includes a high-definition camera sensor C, an infrared thermal imager sensor I, a radar sensor R, a sonar sensor S, and a lidar sensor L. Sensors begin to collect environmental information data; The data is transmitted in real time to the onboard processing unit for pre-processing.

3. The method for dynamic management of unmanned aerial vehicle battery with adaptive path planning according to claim 2, characterized in that: The deep learning model is used to perform fusion analysis on the pre-processed multimodal data to generate an environmental perception map. The specific steps are: Fusing the pre-processed environmental information data of each sensor; The fused data is processed through deep learning of the multimodal fusion network MMFN model to generate an environmental perception map.

4. The method for dynamic battery management of unmanned aerial vehicle with adaptive path planning according to claim 3 is characterized in that: Based on the environmental perception map, the reinforcement learning algorithm is used for adaptive path planning, and the optimal flight path and battery management strategy are dynamically generated according to the drone status and environmental conditions. The specific steps are as follows: Based on the environmental perception map, the central controller of the drone uses a reinforcement learning algorithm to plan the path, which is defined as the path planning function P(x), expressed as: Among them, P(x) is a function of the current state and the environment perception to select the optimal path decision, Q(S, E, a) is a measure called Q function, S s represents the state of the environment, E represents environmental perception, and a represents the action set; Taking into account the battery status and predicted energy consumption, the flight mode and speed are intelligently adjusted through path planning to optimize battery usage. The expression is: Among them, B opt is the optimized battery status, B represents the initial battery status of the drone, T is the flight time, C e (a t ) represents the execution of action a at time t t The energy cost, a t is the action performed by the drone at time step t.

5. The method for dynamic management of unmanned aerial vehicle battery with adaptive path planning according to claim 4 is characterized in that: The path planning and battery management strategy are uploaded to the swarm intelligence center node, and the swarm intelligence algorithm is used for global optimization to generate optimized task allocation and flight strategy. The specific steps are as follows: In a multi-drone scenario, each drone uploads path planning and battery management strategies to the swarm intelligence central node; The central node runs the swarm intelligence algorithm for global optimization and defines the global optimization function G(x), based on the strategy set P of each drone. i and battery status B i , generate optimized task allocation and flight strategy, the expression is: Among them, G(x) is the output of the global optimization function, i is the index of each drone in the traversal drone group, and P i represents the path planning strategy of the i-th UAV, B i represents the battery status of the i-th drone, D i represents the mission requirement of the i-th UAV, N is the total number of UAVs, and B avg is the average battery status of all drones in the swarm.

6. The method for dynamic battery management of unmanned aerial vehicle with adaptive path planning according to claim 5, characterized in that: The optimized flight strategy is executed, environmental changes are continuously monitored, the environmental perception map is updated in real time, and the flight path and battery management strategy are dynamically adjusted. The specific steps are as follows: The drone performs tasks according to the optimized strategy, and multimodal perception continuously monitors environmental changes and updates the environmental perception map in real time; The central controller of the drone dynamically adjusts the flight path and battery management strategy according to the new environmental perception map, and defines the dynamic adjustment function D(x), which is expressed as: Where D(x) represents the dynamic adjustment function, P′ represents the adjusted flight path, and B m represents the battery management strategy based on the new path and environment perception graph, E(X t ) is the predicted environment state based on the original environment perception map, E′(X t ) is the actual environment state based on the updated environment perception map, X t is the position of the drone at time t, T is the number of predicted time steps, and λ P and λ B is the weight of path deviation and battery status adjustment, B′ is the current battery status of the drone, and V is the current speed of the drone.

7. The method for dynamic battery management of a drone with adaptive path planning according to claim 6, characterized in that: After the flight mission is completed, the flight data, environmental perception data and decision records are collected and fed back to the ground control station for iterative training. The specific steps are as follows: After the flight mission is completed, the drone will feed back the flight data, environmental perception data and decision-making process records to the ground control station; The ground control station receives the data and pre-processes it; The loss function L is used to comprehensively consider the importance of classification tasks and regression tasks, and the expression is: Among them, L(D c θ old ) is the loss function, CE(y p ,y t ) is the cross entropy loss function, y p is the label predicted by the model, y t is the actual label, is the mean square error loss function, is the battery state predicted by the model, B t is the actual battery state, λ is the balance factor; Based on the calculated loss function L, calculate the current model parameter θ old The gradient of , using the learning rate η to update the parameters, is expressed as: i new =θ old +Δθ; Among them, Δ θ is the parameter update amount, η is the learning rate, is the loss function L with respect to the model parameters θ old The gradient of θ old is the current model parameter, θ new are the updated model parameters; Through iterative training of the model, and using the back propagation algorithm to update the model parameters, the model update function M(x) is defined, and the expression is: Among them, θ new is the updated model parameter, θ old is the model parameter before update, η is the learning rate, D c is the collected dataset; Model updates are repeated in a training loop; Finally, the model performance is evaluated on the test dataset.

8. The method for dynamic battery management of unmanned aerial vehicle with adaptive path planning according to claim 7, characterized in that: The ground control station performs preprocessing after receiving the data, and the specific steps are as follows: Collect all necessary data sets and conduct preliminary review of the data; Delete and fill missing data using mean, median, mode, and use machine learning algorithms to identify and handle missing data; The Z-score method is used to identify and handle outliers; Check consistency of data types, units, and formats; Generate statistical summaries and use visualization tools to explore the underlying distribution and intrinsic structure of data; Identify and remove duplicate data records and transform data to the same scale; Use one-hot encoding and label encoding to convert categorical variables into machine-readable numerical variables, create new features, and extract seasonal features from time series data; Connect the cleaned and converted data tables to integrate different sensor information of UAV remote sensing data; After data integration is completed, dimensionality reduction is performed through principal component analysis PCA; Apply filtering, wrapping, and embedding methods to select the most predictive subset from a large number of features; Finally, the dataset is divided into training set, validation set and test set.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the reinforcement learning method for dynamic management of batteries of unmanned aerial vehicles with adaptive path planning described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the reinforcement learning method for dynamic management of batteries of unmanned aerial vehicles with adaptive path planning described in any one of claims 1 to 8 are implemented.

Images