Unmanned aerial vehicle autonomous decision reinforcement learning method, system, device and storage medium based on multi-objective optimization

Abstract

The application relates to the technical field of unmanned aerial vehicle autonomous decision-making, and provides an unmanned aerial vehicle autonomous decision-making reinforcement learning method, system, device and storage medium based on multi-target optimization, which solves the problems of limited dynamic task allocation capability and poor decision-making capability of the unmanned aerial vehicle. The method comprises the following steps: based on infrared data and radar data, a suspicious target movement trajectory prediction sequence in a target region is generated, and a movement probability heat map is constructed, wherein the target region comprises a region that has been patrolled; Doppler shift analysis is performed on the radar data, and a target velocity vector field is generated; in combination with position data of the unmanned aerial vehicle, a regional redivision reinforcement learning model is used to output an unmanned aerial vehicle autonomous decision-making reinforcement learning result of a dynamic segmentation scheme containing an unpatrolled region based on a reward function for multi-target optimization, and the reward function is used to maximize region coverage, maximize target tracking success probability and minimize path energy consumption. The application realizes autonomous intelligent decision-making of the unmanned aerial vehicle on the unpatrolled region.

Description

Technical Field

[0001] This application relates to the field of autonomous decision-making technology for unmanned aerial vehicles (UAVs), and in particular to a reinforcement learning method and system for autonomous decision-making of UAVs based on multi-objective optimization. Background Technology

[0002] In scenarios such as disaster search and rescue and border patrol, multiple unmanned aerial vehicle (UAV) systems need to collaborate to complete dynamic target tracking and area coverage tasks. This task faces three core challenges: First, infrared and radar sensors need to be fused and processed to detect targets with weak thermal signals or those that are obscured in complex environments. Second, target movement is highly random, requiring real-time construction of motion models to improve tracking success rates. Third, area coverage, target acquisition probability, and UAV energy consumption need to be optimized simultaneously, with trade-offs existing between different targets.

[0003] The existing solution employs action masking and a multi-agent near-field policy optimization algorithm, combining wide-area detection by high-altitude UAVs with fine-grained scanning by low-altitude UAVs. Reinforcement learning is used to optimize path planning, maximizing target capture while minimizing regional uncertainty. Action masking rules are introduced to restrict collision paths, and a vision-encoded state representation method is used to handle input dimension variations, improving the algorithm's convergence speed. A reward function is used to integrate region coverage, target capture count, and path length to drive the agents to balance exploration and utilization.

[0004] Current solutions still rely on real-time sensor data and fail to integrate historical patrol information to construct a target movement probability model, resulting in weak predictive capabilities for recurring, elusive targets. Ignoring Doppler shift analysis prevents the generation of target velocity vector fields, making it difficult to handle sudden maneuvers. The reward function focuses on target acquisition and area coverage, failing to incorporate path energy consumption into the optimization objective, and lacks dynamic task allocation capabilities. Area segmentation relies on a pre-set geographic grid and does not real-time re-divide unpatrolled areas based on target probability heatmaps, leading to inefficient resource allocation. Summary of the Invention

[0005] This application provides a reinforcement learning method and system for autonomous decision-making of unmanned aerial vehicles (UAVs) based on multi-objective optimization, which solves the problems of poor autonomous decision-making ability of UAVs caused by insufficient fusion of historical information leading to deviations in target trajectory prediction, lack of dynamic target velocity field modeling, incomplete multi-objective optimization dimensions and limited dynamic task allocation capabilities, as well as rigid regional resource allocation mechanisms.

[0006] Firstly, this application provides a reinforcement learning method for autonomous decision-making of unmanned aerial vehicles (UAVs) based on multi-objective optimization, including:

[0007] Infrared and radar data are collected from the patrolled area using an airborne sensor array deployed on the drone.

[0008] Based on the infrared data and the radar data, a movement trajectory prediction sequence of the suspicious target within the target area is generated, and a movement probability heat map is constructed based on the movement trajectory prediction sequence. The target area includes the patrolled area.

[0009] Doppler frequency shift analysis is performed on the radar data to generate the target velocity vector field;

[0010] Using the motion probability heatmap and the target velocity vector field as input features, and combined with the UAV's position data, a pre-trained region repartitioning reinforcement learning model is used to output the UAV's autonomous decision-making reinforcement learning results, which include a dynamic segmentation scheme for unpatched areas, based on a reward function for multi-objective optimization. The reward function is used to reflect maximizing region coverage, maximizing the probability of successful target tracking, and minimizing path energy consumption.

[0011] Optionally, the step of using the movement probability heatmap and the target velocity vector field as input features, combined with the UAV's position data, and outputting the UAV autonomous decision-making reinforcement learning result containing a dynamic segmentation scheme for uninspected areas through a pre-trained region re-partitioning reinforcement learning model based on a reward function for multi-objective optimization, includes:

[0012] Extract the probability distribution features of the movement probability heatmap and the probability distribution features of the target velocity vector field, and then perform spatiotemporal correlation and fusion of the probability distribution features and the UAV position data to generate a joint state feature vector;

[0013] The joint state feature vector is input into the region repartition reinforcement learning model, and the policy network outputs the segmentation parameter set of the uninspected region.

[0014] Based on the set of segmentation parameters, a multidimensional evaluation is performed using the reward function for multi-objective optimization to generate a multidimensional evaluation result.

[0015] Based on the multidimensional evaluation results, the policy network of the region re-partitioning reinforcement learning model is driven to update the weight parameters and output a dynamic segmentation scheme for the uninspected region.

[0016] The dynamic segmentation scheme is bound to the real-time environmental state to generate UAV autonomous decision reinforcement learning results that include the dynamic segmentation scheme for unpatched areas.

[0017] Optionally, the step of binding the dynamic segmentation scheme with the real-time environmental state to generate the UAV autonomous decision reinforcement learning result containing the dynamic segmentation scheme of the uninspected area includes:

[0018] Based on the boundary information of the dynamic segmentation scheme, the geometric vertex coordinates of the unpatched area and the geospatial coordinates in the real-time environmental state are extracted, a mapping relationship between the geometric vertex coordinates and the geospatial coordinates is established, and a spatial coordinate mapping table is generated.

[0019] Based on the time window constraints in the real-time environmental state and the motion dynamics parameters of the UAV, the priority information of the dynamic segmentation scheme is converted into timestamp markers of the path execution sequence to generate execution timing logic;

[0020] The spatial coordinate mapping table and execution timing logic are integrated to construct a structured decision data flow;

[0021] The structured decision data stream is encoded to generate reinforcement learning results for UAV autonomous decision-making.

[0022] Optionally, the step of performing multidimensional evaluation based on the segmentation parameter set and using the reward function for multi-objective optimization to generate multidimensional evaluation results includes:

[0023] Based on the regional boundary control points in the segmentation parameter set, the spatial coverage blind zone distribution of the unpatched area is calculated using the first term in the reward function, and a coverage scalar corresponding to the first target is generated, where the first target is the regional coverage rate.

[0024] Based on the partitioning priority weights in the segmentation parameter set, combined with the high-probability target distribution of the moving probability heatmap and the consistency of the motion direction of the target velocity vector field, the matching degree scalar corresponding to the second target is generated using the second term in the reward function. The second target is the probability of successful tracking of a suspicious target.

[0025] Based on the spatial density of boundary control points in the segmentation parameter set and the current position of the UAV, the energy consumption scalar corresponding to the third target is generated using the third term in the reward function, where the third target is path energy consumption.

[0026] The coverage scalar, matching degree scalar, and energy consumption scalar are linearly superimposed to generate the multidimensional evaluation result.

[0027] Optionally, constructing a movement probability heatmap based on the movement trajectory prediction sequence includes:

[0028] The predicted movement trajectory sequence is divided into geographic grid units, and the target area is decomposed into multiple discrete trajectory points containing spatial coordinates and observation time to form a discrete trajectory point set.

[0029] The spatial distance weight is calculated based on the preset spatial weight kernel function and the spatial coordinates of the discrete trajectory points, and the time decay weight is calculated based on the preset time decay coefficient and the observation time of the discrete trajectory points.

[0030] Based on the spatial distance weight and time decay weight, the spatial coordinates and the observation time are fused to generate the initial thermal value of the geographic grid cell;

[0031] The initial heat values ​​of all the geographic grid cells are converted into color gradient distributions to form a motion probability heatmap.

[0032] Optionally, performing Doppler frequency shift analysis on the radar data to generate a target velocity vector field includes:

[0033] The radar data is deconstructed into a set of discrete echo cells, each of which contains azimuth, elevation and original frequency shift.

[0034] The original frequency shift is periodically folded to generate the true Doppler frequency shift value;

[0035] The true Doppler frequency shift value is converted into a radial velocity component, and combined with the azimuth angle and the elevation angle, the three-dimensional velocity projection is calculated.

[0036] Motion consistency aggregation is performed on the three-dimensional velocity projection to obtain the target velocity vector field.

[0037] Optionally, generating a predicted trajectory sequence of a suspected target within the target area based on the infrared data and the radar data includes:

[0038] Based on the temperature gradient distribution corresponding to the infrared data, the center coordinates of abnormal hot spots whose temperature gradient distribution is higher than the preset temperature difference threshold are extracted in the inspected area. All abnormal hot spot center coordinates are sorted according to the acquisition timestamp to generate a target hot spot location sequence.

[0039] Within a preset time window, the spatial coordinates of the target hotspot location sequence and the trajectory segments of radar data are fused to generate a fused trajectory point set;

[0040] The fused trajectory point set is connected in the order of the timestamps to form a trajectory chain;

[0041] Based on the consistency of velocity vector direction, the trajectory is extended according to the trajectory chain, and a predicted sequence of the movement trajectory of the suspicious target within the target area is output.

[0042] Secondly, this application provides a reinforcement learning system for autonomous decision-making of unmanned aerial vehicles based on multi-objective optimization, including:

[0043] The data acquisition module is used to collect infrared and radar data in the patrolled area through an airborne sensor array deployed on the UAV.

[0044] The module is used to generate a movement trajectory prediction sequence of a suspicious target within a target area based on the infrared data and the radar data, and to construct a movement probability heatmap based on the movement trajectory prediction sequence. The target area includes the patrolled area.

[0045] The generation module is used to perform Doppler frequency shift analysis on the radar data to generate the target velocity vector field;

[0046] The output module is used to take the movement probability heatmap and the target velocity vector field as input features, and combine them with the UAV's position data. Through a pre-trained region re-partitioning reinforcement learning model based on a reward function for multi-objective optimization, it outputs the UAV autonomous decision reinforcement learning results containing the dynamic segmentation scheme of unpatched areas. The reward function is used to reflect the maximization of region coverage, the maximization of target tracking success probability, and the minimization of path energy consumption.

[0047] Thirdly, this application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are to be invoked and executed by the processing component to implement a multi-objective optimization-based UAV autonomous decision reinforcement learning method as described in any of the first aspects.

[0048] Fourthly, this application provides a computer storage medium storing a computer program, which, when executed by a computer, implements a multi-objective optimization-based reinforcement learning method for autonomous decision-making of unmanned aerial vehicles as described in any of the first aspects.

[0049] This application provides a reinforcement learning method for autonomous decision-making of unmanned aerial vehicles (UAVs) based on multi-objective optimization. The method includes: collecting infrared and radar data from a patrolled area using an onboard sensor array deployed on the UAV; generating a predicted trajectory sequence of a suspicious target within a target area based on the infrared and radar data; constructing a motion probability heatmap based on the motion trajectory prediction sequence, where the target area includes the patrolled area; performing Doppler frequency shift analysis on the radar data to generate a target velocity vector field; using the motion probability heatmap and the target velocity vector field as input features, and combining them with the UAV's position data, outputting an autonomous decision-making reinforcement learning result for the UAV, including a dynamic segmentation scheme for unpatched areas, through a pre-trained region re-partitioning reinforcement learning model based on a reward function for multi-objective optimization. The reward function reflects maximizing area coverage, maximizing the probability of successful target tracking, and minimizing path energy consumption.

[0050] The beneficial effects of this application are:

[0051] This application achieves multi-source heterogeneous perception fusion, breaking through the detection blind spots of single sensors in complex environments and improving the reliability of initial target positioning; it predicts the long-term behavior patterns of targets through time-series modeling, transforming discrete detection points into continuous probability distributions to solve the problem of sudden target trajectory interruption; it quantifies the target's motion direction and instantaneous velocity, identifies maneuvering behaviors such as sharp turns and rapid acceleration, and improves the robustness of dynamic target tracking; and it integrates heat maps and velocity fields to achieve joint environmental modeling, outputting dynamic segmentation schemes through multi-target reward functions, overcoming resource rigidity caused by fixed partitions, and ultimately improving the autonomous decision-making capability of UAVs.

[0052] Furthermore, the probability distribution features of the motion probability heatmap and target velocity vector field are deeply extracted and spatiotemporally fused with UAV position data to generate a joint state feature vector. This vector is then input into the policy network of the region re-partitioning reinforcement learning model, outputting a set of segmentation parameters. A multi-objective reward function is used to evaluate the segmentation scheme in multiple dimensions and drive network weight updates. Finally, the dynamic segmentation scheme is bound to the real-time environmental state: a spatial coordinate mapping table is generated by establishing a mapping relationship between geometric vertices and geographic coordinates. Priorities are transformed into temporal logic by combining time window constraints and UAV dynamic parameters. A structured decision data stream is then constructed and encoded into executable autonomous decision results. Deep fusion of probability distribution features transforms the target motion trend into quantifiable decision factors, avoiding the response lag of traditional rule-based segmentation to sudden trajectory changes. Simultaneously, the structured data stream enables real-time binding of the segmentation scheme to the UAV's physical environment, overcoming the accumulation of execution errors and response lag caused by manual analysis.

[0053] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A flowchart illustrating a reinforcement learning method for autonomous decision-making in unmanned aerial vehicles (UAVs) based on multi-objective optimization, provided for embodiments of this application;

[0056] Figure 2 A schematic diagram of the structure of a UAV autonomous decision-making reinforcement learning system based on multi-objective optimization, provided for an embodiment of this application;

[0057] Figure 3 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0058] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0059] In some of the processes described in the specification, claims, and accompanying drawings of this application, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 11, 12, etc., are merely used to distinguish different operations and do not themselves represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0060] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] To address the problems of poor UAV autonomous decision-making capabilities caused by insufficient historical information fusion leading to target trajectory prediction errors, lack of dynamic target velocity field modeling, incomplete multi-objective optimization dimensions, limited dynamic task allocation capabilities, and rigid regional resource allocation mechanisms in existing technologies, this application provides a reinforcement learning method for UAV autonomous decision-making based on multi-objective optimization. This method employs the following concept: First, airborne infrared and radar sensors are used to collect real-time data of the patrol area; through fusion processing, a predicted sequence of suspicious target movement trajectories is generated, and a spatial distribution heatmap reflecting the probability of target appearance is constructed accordingly; simultaneously, Doppler frequency shift analysis is performed on the radar data to extract the target's velocity vector field; then, the heatmap, velocity vector field, and real-time UAV position data are jointly input into a pre-trained reinforcement learning model. This model designs a reward function based on multi-objective optimization criteria of maximizing regional coverage, maximizing the probability of successful target tracking, and minimizing flight path energy consumption. Through dynamic environmental interaction and autonomous decision-making, the final output is a dynamic path planning result including an intelligent segmentation scheme for unpatrolled areas, achieving an autonomous closed loop from perception and prediction to decision-making.

[0062] Figure 1A flowchart illustrating a reinforcement learning method for autonomous decision-making in unmanned aerial vehicles (UAVs) based on multi-objective optimization, as provided in this application embodiment, is shown below. Figure 1 As shown, the method includes:

[0063] S11. Collect infrared and radar data of the patrolled area through the airborne sensor array deployed on the drone.

[0064] The airborne sensor array refers to a combination of multimodal detection equipment carried by the UAV, including infrared and radar sensors, used to simultaneously collect information on ambient temperature distribution and target movement. The surveyed area refers to the geographic space area that the UAV has completed scanning and exploration of; its boundaries can be determined by historical flight paths. Infrared data refers to two-dimensional matrix data reflecting the temperature gradient of an object's surface, acquired through infrared sensors. Radar data refers to a three-dimensional point cloud dataset containing target range, azimuth, and Doppler shift, generated from radar echoes.

[0065] In this embodiment of the application, the patrolled area is first scanned by an airborne sensor array deployed on the UAV, and infrared data reflecting the temperature distribution of objects and radar data detecting the position and motion state of objects are collected simultaneously to provide raw sensory input for subsequent analysis.

[0066] S12. Based on infrared and radar data, generate a predicted sequence of the movement trajectory of the suspicious target within the target area, and construct a heat map of movement probability based on the predicted movement trajectory sequence. The target area includes the patrolled area.

[0067] Suspicious targets refer to moving objects in infrared or radar data that meet preset anomalous characteristics, such as sudden temperature changes or high-speed moving entities. Target area refers to a continuous geographic space encompassing both patrolled areas and unpatrolled areas awaiting further exploration. Motion trajectory prediction sequence refers to a time series of target future locations predicted based on historical trajectory points through linear interpolation or machine learning. Motion probability heatmap is a two-dimensional distribution map that maps geographic grids to color gradients based on target occurrence probability values; the probability values ​​can be calculated by weighting the spatiotemporal density of trajectory points.

[0068] In this embodiment, the movement trajectory points of suspicious targets are first identified based on infrared and radar data. A spatiotemporal correlation algorithm is then used to generate a predicted sequence of the target's movement trajectory within the target area. Next, based on the spatiotemporal distribution density of the trajectory points in this sequence, probability weights are calculated using the spatial weight kernel function and time decay coefficient of the geographic grid cells. Finally, a movement probability heatmap reflecting the probability distribution of the target's appearance is generated. It should be noted that this embodiment does not specifically limit the representation of the movement trajectory points, the representation of the spatial weight kernel function, or the magnitude of the time decay coefficient.

[0069] S13. Perform Doppler frequency shift analysis on the radar data to generate the target velocity vector field.

[0070] Doppler frequency shift analysis refers to the process of periodically folding and correcting the radar echo frequency offset and calculating the radial velocity. The target velocity vector field refers to the distribution field of group motion directions generated by aggregating the three-dimensional velocity vectors of discrete targets.

[0071] In this embodiment, firstly, Doppler frequency shift analysis is performed on each discrete echo cell in the radar data, and the original frequency shift is corrected by periodic folding to obtain the true Doppler frequency shift value; secondly, the frequency shift value is converted into radial velocity components, and the three-dimensional velocity projection is calculated by combining the azimuth and elevation angles; finally, a target velocity vector field that reveals the motion direction of the target group is generated by motion consistency clustering.

[0072] S14. Using the motion probability heatmap and target velocity vector field as input features, and combining them with the UAV's position data, the UAV autonomous decision reinforcement learning results, which include the dynamic segmentation scheme of unpatroled areas, are output through a pre-trained region re-segmentation reinforcement learning model based on a reward function for multi-objective optimization. The reward function is used to reflect the maximization of region coverage, the maximization of target tracking success probability, and the minimization of path energy consumption.

[0073] In this context, location data refers to the 3D geographic coordinates and attitude angles output by the UAV's real-time positioning system. The region repartitioning reinforcement learning model is a deep reinforcement learning framework that takes environmental features as input and outputs dynamic partitioning parameters through a policy network. The reward function for multi-objective optimization is an evaluation function that simultaneously quantifies the increase in region coverage, target tracking matching degree, and path energy consumption cost. Unexplored areas refer to the set of undetected sub-regions within the target area that are not covered by sensor scanning. The dynamic segmentation scheme refers to the set of geometric boundary partitioning parameters for unexplored areas generated based on real-time environmental conditions. The UAV autonomous decision-making reinforcement learning result refers to encoding the dynamic segmentation scheme into an executable instruction set containing a sequence of waypoints and timestamps. Maximizing region coverage means minimizing the detection blind area of ​​unexplored areas. Maximizing the probability of successful target tracking means prioritizing the detection of high-probability target areas. Minimizing path energy consumption means minimizing the total length of the UAV's path and the energy cost of turning.

[0074] The value R of the reward function can be calculated using the following formula: ,in, This is a regional coverage bonus item. For target tracking rewards, This is a path energy consumption penalty term. The default non-negative weighting values ​​are used for the corresponding items. This application embodiment does not specifically limit the magnitude of each weighting value; for example, , , Furthermore, this embodiment does not impose specific limitations on the specific expressions of each item.

[0075] In this embodiment, the probability distribution features of the motion probability heatmap and the probability distribution features of the target velocity vector field are first spatiotemporally fused with the UAV position data to form a joint state feature vector. Next, this vector is input into the policy network of the region repartition reinforcement learning model, outputting a set of segmentation parameters for unpatched areas. Then, a multi-objective optimization reward function is used to evaluate scalar values ​​of coverage, tracking success probability, and energy consumption, driving network weight updates to generate a dynamic segmentation scheme. Finally, this scheme is bound to real-time geographic coordinates, and the UAV autonomous decision-making reinforcement learning result is generated through a spatial coordinate mapping table and temporal logic encoding.

[0076] Here is a specific example: First, the UAV scans the patrolled forest area using onboard infrared and radar sensors, collecting raw data on abnormal heat spots and moving targets. Second, based on the heat spot coordinates and radar trajectory points, a predicted sequence of movement trajectories for illegal immigrants is generated, thus constructing a high-probability heat map of the eastern side of the forest area. Next, Doppler shift analysis of the radar data reveals that the target group is moving towards an unpatrolled valley, generating a northwest-oriented velocity vector field. Subsequently, the heat map, vector field, and the UAV's real-time position are input into a reinforcement learning model, outputting a segmentation scheme for the unpatrolled valley. Finally, the scheme boundaries are mapped to geographic coordinates, and combined with the UAV's turning angle constraints, a flight path command is generated to first block the valley entrance and then cover the slope, completing the collaborative task of dynamic tracking and regional coverage.

[0077] Specifically, during drone patrol missions in border forest areas, the infrared and radar sensors on the drones were activated simultaneously. The infrared sensor collected temperature distribution data of the patrolled area at a frequency of 30 frames per second, generating a two-dimensional temperature matrix with a resolution of 512×512. Three areas were identified where the temperature was significantly higher than the environmental average by approximately 25°C, reaching a maximum of 36°C. Simultaneously, the radar sensor detected at a 120° horizontal scanning angle and a 5° vertical scanning angle, generating data containing 5000 three-dimensional point clouds. This data recorded the distance of objects corresponding to these high-temperature areas from the drone as approximately 800-1200 meters, with azimuth angles between 30° and 45°. This infrared and radar data was stored in real-time in the drone's onboard storage module as raw input for subsequent analysis. Based on the data collected by the S11 sensor, the system first identified three suspicious targets, judged as suspicious due to abnormal temperatures and accompanying movement. By analyzing the movement trajectory of the infrared hotspots and the positional changes of the radar echoes, the historical trajectory points of each target were obtained. For example, the trajectory points of target A in the past 5 minutes are (X1, Y1, T1), (X2, Y2, T2)...(X5, Y5, T5). Linear interpolation is used to extend these trajectory points, predicting the target's movement trajectory sequence within the target area, including the patrolled area and the unpatrolled valley to the west, in the next 10 minutes. For example, target A might pass through (X6, Y6, T6)...(X15, Y15, T15). Subsequently, the target area is divided into a 10m × 10m geographic grid, and the spatiotemporal density of trajectory points within each grid is calculated. Spatially, a Gaussian kernel function is used to calculate the weight of the distance from the grid center, with closer points having higher weights. Temporally, a time decay coefficient of 0.9 is set, with more recent trajectory points having higher weights. Finally, a heatmap of movement probability is generated, where the probability value of the grid at the valley entrance reaches 0.8 and is displayed in dark red, while the probability value of areas far from the trajectory is below 0.2 and is displayed in light blue. The radar data acquired by S11 is processed and deconstructed into 1000 discrete echo cells. Each cell contains an azimuth angle of 30°-45°, an elevation angle of -2°-1°, and the original frequency shift. Since the maximum unambiguous frequency shift corresponds to a velocity of 10 m / s, some original frequency shifts were found to exceed this range. Periodic folding correction was used; for example, the frequency shift corresponding to +15 m / s was corrected to -5 m / s, and the periodic value of 20 m / s was subtracted to obtain the true Doppler frequency shift value. According to the formula "radial velocity = (frequency shift value × wavelength) / (2 × transmission frequency)," the radial velocity component of each cell is calculated. Combined with the azimuth and elevation angles, a three-dimensional velocity projection is decomposed. For example, the eastward velocity of target A is -1.2 m / s, and the northward velocity is -0.8 m / s, indicating that it is moving northwest. By using motion-consistent clustering, velocity vectors with a directional deviation of less than 10° are grouped into one class, ultimately generating a target velocity vector field. This field shows that 85% of the target vectors point in the northwest direction, with an average velocity of approximately 1.5 m / s.The UAV's current location is 116.5°E, 40.2°N, and 500 meters above sea level. The motion probability heatmap generated in S12 and the target velocity vector field generated in S13 are input into a pre-trained region re-segmentation reinforcement learning model along with this location data. The model's reward function is set with a weight of 0.3 for region coverage, 0.4 for target tracking success probability, and 0.3 for path energy consumption. After policy network analysis, a dynamic segmentation scheme is output, dividing the unpatrolled area into three sub-regions: valley entrance priority 1, western slope priority 2, and northern forest priority 3, and determining the boundary control point coordinates for each region. Considering the UAV's maximum turning angle of 30° and cruising speed of 10 m / s, an autonomous decision-making result is generated, with the flight path sequence as: entrance coordinates 10:00, slope coordinates 10:05, and forest coordinates 10:15. The encoded commands are sent to the flight control system to ensure priority blocking of high-probability areas while minimizing flight energy consumption.

[0078] By executing S11~S14, this embodiment of the application solves the problems of lagging environmental dynamic response and fragmented multi-target optimization in traditional patrols through deep fusion of multi-source perception data and dynamic decision-making through reinforcement learning. It achieves adaptive segmentation of unpatrolled areas and precise scheduling of UAV resources, and simultaneously improves target tracking reliability, area coverage completeness and energy utilization efficiency in complex scenarios.

[0079] In one possible embodiment, S14, the mobile probability heatmap and target velocity vector field are used as input features, and combined with the UAV's position data, a pre-trained region re-partitioning reinforcement learning model is used based on a reward function for multi-objective optimization to output the UAV autonomous decision-making reinforcement learning result containing a dynamic segmentation scheme for unpatched areas, including:

[0080] Step 141: Extract the probability distribution features of the motion probability heatmap and the probability distribution features of the target velocity vector field, and perform spatiotemporal correlation and fusion of the probability distribution features and the UAV position data to generate a joint state feature vector.

[0081] Among them, probability distribution characteristics refer to the probability density distribution characteristics of grid cells in the motion probability heatmap, including probability value variance, gradient rate of change, and the concentration of vector directions and velocity magnitude distribution range in the target velocity vector field. Spatiotemporal correlation fusion refers to the process of aligning the target probability distribution characteristics with UAV position data according to timestamps and geographic coordinates, and combining the UAV observation perspective weights with the target motion trend weights through a weighted fusion function. The joint state feature vector refers to a multidimensional tensor that fuses environmental probability characteristics and UAV pose data, and its dimensions include heatmap probability values, vector field direction angles, UAV latitude and longitude, and pitch angles.

[0082] In this embodiment, the spatial probability distribution features of the motion probability heatmap are first extracted, including the peak value of the probability density and the gradient change trend; at the same time, the motion direction probability distribution features of the target velocity vector field are extracted, including the consistency of the vector direction and the velocity magnitude distribution; secondly, the two types of probability distribution features are spatiotemporally aligned with the real-time position data of the UAV, and the UAV observation perspective and the target motion trend are fused through a spatiotemporal attention mechanism; finally, a joint state feature vector representing the coupling relationship between environmental dynamics and UAV state is generated.

[0083] Step 142: Input the joint state feature vector into the region repartition reinforcement learning model, and output the segmentation parameter set of the unpatched region through the policy network.

[0084] In this model, the policy network is a deep neural network-based decision-making component that takes a joint state vector as input and outputs a weight matrix of segmentation parameters. The segmentation parameter set refers to the set of unexplored region segmentation parameters generated by the policy network, including the 3D coordinates of boundary control points, sub-region priority coefficients, and boundary smoothness coefficients. The policy network is the core decision-making component within the region re-partitioning reinforcement learning model. It directly receives a joint state feature vector generated by fusing motion probability heatmaps, target velocity vector fields, and UAV position data as input. Through the computational rules within the neural network, it analyzes the current environmental state and mission objectives, and outputs a dynamic set of segmentation parameters for unexplored regions in real time. This set includes key parameters such as boundary control point coordinates and segmentation priority weights. The region re-partitioning reinforcement learning model drives the generation of segmentation schemes through the action output of the policy network. Simultaneously, it uses a reward function to perform multi-objective optimization evaluation of the segmentation parameters output by the policy network. By continuously iterating and updating the parameter weights of the policy network, the final segmentation scheme output by the model can synergistically maximize region coverage, tracking success probability, and minimize path energy consumption.

[0085] In this embodiment, the joint state feature vector is first input into the policy network of the region repartition reinforcement learning model, which is composed of multiple fully connected neurons; then, the segmentation parameter set of the unpatched region is calculated and output through forward propagation, including the coordinates of the region boundary control points, the segmentation priority weights, and the boundary density parameters.

[0086] Step 143: Based on the set of segmentation parameters, perform multidimensional evaluation using the reward function for multi-objective optimization to generate multidimensional evaluation results.

[0087] Among them, the multidimensional evaluation result refers to the weighted sum of three types of scalar values ​​calculated through a multi-objective reward function, including regional coverage capability evaluation, target tracking effect evaluation, and path energy consumption efficiency evaluation.

[0088] In this embodiment, firstly, based on the regional boundary control points in the segmentation parameter set, the coverage scalar of the unpatched area is calculated using the first term of the reward function; secondly, based on the division priority weight combined with the high probability distribution of the heat map and the consistency of the vector field direction, the target tracking matching degree scalar is calculated; then, based on the spatial density of the boundary control points and the current position of the UAV, the path energy consumption scalar is calculated; finally, the three types of scalars are linearly superimposed to generate a multidimensional evaluation result.

[0089] Step 144: Based on the multidimensional evaluation results, update the weight parameters of the policy network of the region re-division reinforcement learning model and output the dynamic segmentation scheme for the uninspected region.

[0090] The weight parameters refer to the weight values ​​of the neuron connections in the policy network, which are iteratively updated using the gradient descent algorithm to optimize decision-making. The real-time environmental state refers to the dynamic set of constraints the UAV faces while performing its mission, including geospatial coordinates, remaining flight time, maximum turning angle, and speed threshold.

[0091] In this embodiment, the gradient direction of the policy network is first calculated based on the multidimensional evaluation results; then the network weight parameters are updated through the backpropagation algorithm; subsequently, a dynamic segmentation scheme for the unpatched region is regenerated based on the updated policy network, which includes optimized boundary control points and priority parameters.

[0092] Step 145: Bind the dynamic segmentation scheme to the real-time environmental state to generate the UAV autonomous decision reinforcement learning results that include the dynamic segmentation scheme for uninspected areas.

[0093] In this embodiment, firstly, geometric vertex coordinates are extracted based on the boundary information of the dynamic segmentation scheme; secondly, a mapping relationship is established with the geospatial coordinates in the real-time environment to generate a spatial coordinate mapping table; then, the scheme priority is converted into execution timing logic marked with timestamps by combining time window constraints and the maximum turning angle and speed parameters of the UAV; finally, the coordinate mapping table and timing logic are fused to encode and generate the autonomous decision reinforcement learning result that the UAV can execute.

[0094] Here's a specific example: The drone first analyzes the distribution patterns of population clusters in the forest heat map, including which areas have the highest hotspot density and which areas experience the fastest temperature changes. Simultaneously, it analyzes the movement patterns of population groups in the radar velocity field, such as most people moving northwest at similar speeds. These distribution patterns are then matched and fused with the drone's current coordinates and flight altitude to generate a comprehensive data package describing the environment and the drone's status. This data package is then input into the intelligent partitioning model. The model, through its built-in decision network, outputs partitioning parameters for unpatrolled areas, including the coordinates of valley boundary points requiring priority coverage and the priority numbers of each area. Based on these parameters, the system evaluates the effectiveness of the plan: calculating the coverage area of ​​the blind spots to obtain a coverage score, calculating a tracking and matching score based on hotspot distribution and population movement direction, and estimating flight energy consumption based on boundary point density and the drone's current position. After the evaluation, the system automatically adjusts the parameters of the decision network based on the three scores, regenerating a better partitioning plan, for example, setting valley entrances as the highest priority blockade area and slopes as secondary coverage areas. Finally, the boundary point coordinates of the new scheme were converted into actual geographic coordinates, and specific instructions were generated based on the drone's maximum turning angle: the drone was required to arrive at the valley entrance coordinates before 10:00 AM to implement the blockade, and to complete the coverage patrol of the slope coordinates before 10:30 AM, forming a flight mission book that could be directly executed. The coverage effect score was calculated by subtracting the actual blind spot area from the area to be covered; the tracking and matching score was obtained by multiplying the population density in the hotspot area by the consistency ratio of the movement direction; and the energy consumption score was determined by multiplying the straight-line distance from the drone to the boundary point by the path curvature coefficient.

[0095] In another specific example, in step 141, features of the motion probability heatmap are extracted, with the probability peak at the valley entrance and a gradient change rate of 0.3 / m. Features of the velocity vector field are also extracted, with directional consistency of 85% and a velocity magnitude of 1-2 m / s. These features are aligned with the UAV's location (116.5°E, 40.2°N, pitch angle 5°) using the timestamp T=10:00 and geographic coordinates. Through a spatiotemporal attention mechanism, features within the UAV's observation range are assigned a higher weight of 0.7, generating a 128-dimensional joint state feature vector covering heatmap values, orientation angles, latitude and longitude, and other information. Step 142 inputs the joint state feature vector into the policy network, which consists of three fully connected layers. Through forward propagation, it outputs a set of segmentation parameters, including five coordinate points for the valley entrance boundary control point (priority coefficient 1.0, boundary smoothness coefficient 0.8) and four coordinate points for the slope control point (priority coefficient 0.7). Step 143 calculates the multi-dimensional evaluation results based on the parameters: coverage scalar 0.8, blind zone area ratio 20%, matching degree scalar 0.9, high probability area matches the target movement direction, energy consumption scalar 0.3, short flight distance and few turns, and the weighted score is 0.8×0.3+0.9×0.4+0.3×0.3=0.75. Step 144 adjusts the policy network weights through backpropagation based on the evaluation results, such as adjusting the weight of the heatmap feature layer from 0.2 to 0.3. The updated output is a better segmentation scheme, densifying the valley entrance control points and improving boundary accuracy. Step 145 converts the boundary points of the new scheme into actual geographic coordinates, such as the entrance center point at 116.51°E, 40.19°N. Combining the time window of 10:00-10:20 and the UAV's performance, an execution sequence is generated: arriving at the entrance at 10:00, starting to cover the slope at 10:05. This is encoded as a binary instruction containing coordinates and a timestamp, serving as the final decision result.

[0096] By executing steps 141 to 145, this embodiment of the application solves the problems of environmental response lag and execution disconnect in dynamic segmentation through deep feature fusion and reinforcement learning closed-loop optimization, realizes adaptive binding of unpatroled area division and UAV physical constraints, and improves the real-time performance of target tracking and the energy efficiency balance of coverage path in complex scenarios.

[0097] In one possible embodiment, step 145, binding the dynamic segmentation scheme with the real-time environmental state, generates a UAV autonomous decision reinforcement learning result containing the dynamic segmentation scheme for unpatched areas, including:

[0098] Step a1: Based on the boundary information of the dynamic segmentation scheme, extract the geometric vertex coordinates of the unpatched area and the geospatial coordinates in the real-time environmental state, establish the mapping relationship between the geometric vertex coordinates and the geospatial coordinates, and generate a spatial coordinate mapping table.

[0099] Boundary information refers to the geometric contour description data of the unpatched area defined in the dynamic segmentation scheme, including the vertex connectivity and topological structure of the polygon boundary. Geometric vertex coordinates refer to the three-dimensional position data of the boundary points of the unpatched area described in the local coordinate system, usually stored in Cartesian coordinates. Geospatial coordinates refer to the position data in the global coordinate system based on the Earth ellipsoid model, including longitude, latitude, and altitude values. Mapping relationships refer to the transformation correspondence rules between geometric vertex coordinates and geospatial coordinates, achieving coordinate system alignment through an affine transformation matrix. The spatial coordinate mapping table is a two-dimensional table recording the transformation results of all boundary points from local coordinates to geographic coordinates, containing the original vertex number and the transformed coordinate values.

[0100] In this embodiment, firstly, the geometric vertex coordinates of the unpatched area are extracted based on the boundary information of the dynamic segmentation scheme. These coordinates exist in the form of a polygon vertex sequence. Secondly, the geospatial coordinates in the real-time environmental state are obtained. These coordinates originate from the high-precision three-dimensional coordinate system of the geographic information system. Subsequently, a one-to-one correspondence mapping relationship between the geometric vertex coordinates and the geospatial coordinates is established, and the conversion from the local coordinate system to the global coordinate system is achieved through a coordinate transformation matrix. Finally, a spatial coordinate mapping table containing all vertex mapping relationships is generated.

[0101] Step a2: Based on the time window constraints in the real-time environmental state and the motion dynamics parameters of the UAV, the priority information of the dynamic segmentation scheme is converted into timestamps of the path execution sequence to generate execution timing logic.

[0102] Among these, the time window constraint refers to the set of time-limited conditions for task execution, including the effective target tracking time limit, the maximum endurance of the UAV, and the time thresholds for key actions. Motion dynamics parameters are quantitative indicators of the UAV's physical motion capabilities, covering maximum pitch angle, roll angle limits, cruise speed range, and maximum acceleration for sharp turns. Priority information refers to the importance ranking data of different sub-regions in the dynamic segmentation scheme; higher values ​​reflect the urgency of task execution. The path execution sequence is an ordered set of waypoints sorted by task priority, defining the order of UAV spatial movement. Timestamps are absolute time identifiers assigned to each waypoint in the path execution sequence, accurate to the millisecond level. Execution timing logic refers to the set of spatiotemporal constraint rules that integrate the pathpoint sequence and timestamps, specifying the arrival time and dwell time of each waypoint.

[0103] In this embodiment, the time window constraints in the real-time environmental state are first read, including the target prediction disappearance time limit and the maximum mission duration; then, the motion dynamics parameters of the UAV are obtained, covering the maximum turning angle limit, cruise speed threshold and acceleration limit; then, the priority information of different sub-regions in the dynamic segmentation scheme is converted into the execution order of the path point sequence according to the time window and dynamic constraints; finally, a precise timestamp is assigned to each path point to generate the execution timing logic that can be parsed by the flight control system.

[0104] Step a3: Integrate the spatial coordinate mapping table and the execution timing logic to construct a structured decision data flow.

[0105] Among them, structured decision data flow refers to a set of machine instructions organized according to specific data specifications, which includes a three-dimensional nested data structure of spatial coordinates, timestamps, and priorities.

[0106] In this embodiment, the sequence of geographic coordinate points in the spatial coordinate mapping table is first aligned with the timestamps in the execution timing logic according to the path order; secondly, each coordinate point is bound to a corresponding timestamp and priority label through a data structure nesting and fusion method; finally, a structured decision data stream containing spatial location, execution timing and task weight is constructed to form a machine-readable instruction set.

[0107] Step a4: Encode the structured decision data stream to generate reinforcement learning results for UAV autonomous decision-making.

[0108] In this embodiment, the structured decision data stream is first serialized using a binary encoding protocol, converting coordinate points, timestamps, and priority tags into a low-latency transmission format; secondly, track check codes and command check bits are added to ensure data integrity; finally, an autonomous decision reinforcement learning result that can be directly input into the UAV flight control system is generated, which includes an encrypted track point sequence and execution timing commands.

[0109] Here is a specific example: A drone scans an already inspected forest area using its sensors, detecting abnormal hotspots and moving targets, and collecting raw data. Then, based on the location of the hotspots and radar trajectory points, it generates a sequence of predicted movement trajectories for suspicious individuals, creating a high-probability hotspot distribution map of the eastern side of the forest. Through Doppler shift analysis of the radar data, it discovers that the target group is moving towards an uninspected valley, generating a velocity pattern pointing northwest. Next, these hotspot maps, velocity patterns, and the drone's current location information are input into a reinforcement learning model, outputting a dynamic segmentation scheme for the uninspected valley. Then, based on the boundary information provided by this scheme, the geometric vertex coordinates of the uninspected area are extracted, and these coordinates are compared with the real-time environmental conditions. The system matches the actual geographic coordinates in the environment to establish a location mapping relationship and generate a spatial coordinate mapping table. Based on the patrol time window constraints set in the real-time environment, such as patrol time period requirements, and the UAV's motion capability parameters, such as maximum speed and minimum turning angle, the priority of the dynamic segmentation scheme is assigned, for example, the time point marker sequence for converting the valley entrance into a path execution sequence, generating execution timing logic. Then, the spatial coordinate mapping table and execution timing logic are merged and integrated to construct a structured decision data stream. Finally, the data stream is encoded to generate the UAV's autonomous decision reinforcement learning results, and these results are applied to command generation, such as flight path control that first blocks the valley entrance and then covers the slope, to achieve the coordinated completion of dynamic target tracking and area coverage tasks.

[0110] In another specific example, in step a1, the boundary control points of the dynamic segmentation scheme are 100, 200, 500 in the local coordinate system. These are mapped to the real-time geographic coordinates: 116.5°E (x=1000) and 40.2°N (y=2000). Through an affine transformation matrix, the geographic coordinates are obtained as 116.505°E, 40.198°N, and an altitude of 500 meters. A spatial coordinate mapping table is generated, recording the transformation results of all control points. In step a2, the time window constraint is that the target may leave the monitoring range before 10:20, and the UAV has a flight time of 2 hours. The motion dynamics parameters include a maximum turning angle of 30° and a cruising speed of 8-12 m / s. The priority sub-regions Entrance 1, Slope 2, and Woodland 3 are converted into path sequences. Combined with speed calculation timestamps, the entrance is 800 meters from the current location, requiring 80 seconds; adding 0:01:20 to 10:00:00 gives an arrival time of 10:01:20. The slope is 500 meters from the entrance, requiring 50 seconds; arriving at 10:02:10. This generates the execution timing logic. In step a3, the entrance coordinates 116.505°, 40.198° in the mapping table are bound to the timestamp 10:01:20 (priority 1); the slope coordinates are bound to 10:02:10 (priority 2), constructing a structured decision data stream in the format [coordinates, timestamp, priority]. In step a4, the data stream is binary encoded, with each coordinate represented by a 32-bit floating-point number, the timestamp by a 64-bit integer accurate to milliseconds, and the priority by an 8-bit integer. After adding a checksum, a 2KB instruction parseable by the UAV is generated and sent to the flight control system via the onboard bus.

[0111] By executing steps a1 to a4, this embodiment of the application establishes a precise binding mechanism between the decision-making scheme and the physical environment, solving the problem of the disconnect between trajectory planning and real-time status in traditional methods, realizing the lossless conversion of dynamic segmentation schemes into flight control commands, and improving the spatiotemporal accuracy and system reliability of mission execution in complex environments.

[0112] In one possible embodiment, step 143, based on the segmentation parameter set, performs a multidimensional evaluation using a reward function for multi-objective optimization to generate a multidimensional evaluation result, including:

[0113] Step b1: Based on the regional boundary control points in the segmentation parameter set, calculate the spatial coverage blind zone distribution of the unpatched area using the first term in the reward function, and generate the coverage scalar corresponding to the first target, which is the regional coverage rate.

[0114] Among them, the regional boundary control points refer to the set of key coordinate points that define the geometric contour of the unpatched area in the dynamic segmentation scheme, including polygon vertices and curvature control points. The coverage scalar is a quantitative indicator reflecting the completeness of the unpatched area detection, which is the normalized value of the difference between the theoretical coverage area and the actual blind zone area. The spatial coverage blind zone distribution refers to the geographic spatial location and range of the unpatched area not covered by the polygon to be patrolled when the UAV connects the regional boundary control points in the segmentation scheme. Specifically, when the coordinate sequence of the boundary control points is connected in sequence to form a closed polygon, the polygon area represents the effective coverage area that the UAV will patrol. All continuous sub-regions outside the polygon in the unpatched area constitute the spatial coverage blind zone. Its distribution pattern is directly determined by the spatial positional relationship of the boundary control points. The larger the blind zone area, the wider the uncovered geographic range. The blind zone distribution range can be quantified by calculating the total area of ​​the unpatched area minus the area of ​​the polygon enclosed by the boundary control points. This distribution result is used for the first term calculation of the reward function, generating the core indicator reflecting the level of regional coverage, namely the coverage scalar.

[0115] In this embodiment, the minimum circumscribed polygon area of ​​the uninspected area is first calculated based on the coordinates of the regional boundary control points in the segmentation parameter set; then, the polygon area is divided by the total area of ​​the target area to obtain the theoretical coverage rate; next, the difference between the theoretical coverage rate and the actual sensor coverage range is calculated to generate the spatial coverage blind zone distribution; finally, the blind zone area is normalized by taking a negative value through the first term of the reward function to generate the first target coverage scalar reflecting the regional coverage rate.

[0116] Step b2: Based on the partitioning priority weights in the segmentation parameter set, combined with the high-probability target distribution in the movement probability heatmap and the consistency of the motion direction of the target velocity vector field, the matching degree scalar corresponding to the second target is generated using the second term in the reward function. The second target is the probability of successful tracking of the suspicious target.

[0117] Among them, the priority weight refers to the numerical importance level assigned to different sub-regions in the dynamic segmentation scheme, and the higher the value, the more tasks are executed in order. The matching degree scalar refers to the composite evaluation value that integrates the spatial distribution probability of the target and the consistency of the movement direction, reflecting the degree of fit between the tracking strategy and the target's behavior pattern.

[0118] In this embodiment, the priority weights in the segmentation parameter set are first obtained; then, the spatial coordinate density of high-probability target distribution and the motion direction consistency coefficient of the target velocity vector field are extracted from the motion probability heatmap; then, the priority weights are multiplied by the high-probability density value and then by the direction consistency coefficient through the second term of the reward function to generate a second target matching degree scalar that reflects the probability of successful tracking of suspicious targets.

[0119] Step b3: Based on the spatial density of boundary control points in the segmentation parameter set and the current position of the UAV, generate the energy consumption scalar corresponding to the third objective using the third term in the reward function. The third objective is the path energy consumption.

[0120] Among them, the energy consumption scalar refers to the estimated energy consumption calculated based on path length, turning angle and flight speed, including distance factor and maneuver complexity factor.

[0121] In this embodiment of the application, the spatial density of the boundary control points in the segmentation parameter set is first calculated, that is, the number of control points per unit area; then, the Euclidean distance from the current position of the UAV to the nearest control point is obtained; then, the spatial density is multiplied by the distance value and then by the path curvature coefficient through the third term of the reward function to generate a third target energy consumption scalar that reflects the path energy consumption.

[0122] Step b4: Linearly superimpose the coverage scalar, matching degree scalar, and energy consumption scalar to generate a multidimensional evaluation result.

[0123] Linear superposition refers to the calculation process of summing multiple scalars according to preset weight coefficients, where the weights reflect the relative importance of different optimization objectives.

[0124] In this embodiment, the coverage scalar is first multiplied by a first weighting coefficient to obtain a weighted coverage value; then the matching scalar is multiplied by a second weighting coefficient to obtain a weighted matching value; subsequently, the energy consumption scalar is multiplied by a third weighting coefficient to obtain a weighted energy consumption value; finally, the three weighted values ​​are added together to generate a multidimensional evaluation result.

[0125] Here's a specific example: The drone first scans the forest area using infrared and radar equipment, detecting abnormal heat sources and moving objects. It records the locations of these heat sources and radar trajectory points, predicts the movement paths of suspicious individuals, and constructs a high-probability target distribution map on the eastern side of the forest. Next, it analyzes frequency changes in the radar data, discovering that the target group is moving towards a valley in the northwest, generating a velocity vector map in the northwest direction to represent the target's movement trend. Simultaneously, the drone acquires its own location information in real time. After this data is input into the intelligent model, the model outputs a segmentation scheme for the unpatrolled valley, which includes parameters such as control points for the area boundaries and segmentation priority weights. Based on this segmentation scheme, the model begins to calculate the multidimensional evaluation results: First, it analyzes the spatial coverage blind spot distribution in the unpatched area using boundary control points, calculating a coverage value using the first term of the reward function. This value reflects the coverage level of the area, calculated as the ratio of the blind spot area to the total area. Second, combining the priority weights, the high-probability target distribution map, and the motion direction consistency of the velocity vector map, it calculates a matching degree value using the second term of the reward function. This value represents the probability of successfully tracking a suspicious target, calculated by integrating priority weights, target distribution probability, and motion direction matching degree. Third, based on the spatial density of the boundary control points and the UAV's current position, it calculates an energy consumption value using the third term of the reward function. This value represents the energy consumption of the UAV's flight path, estimated by multiplying the control point density by the distance from the UAV to the boundary points. Finally, the coverage value, matching degree value, and energy consumption value are linearly added to generate a multidimensional evaluation result. The evaluation results were used to optimize the segmentation scheme. The scheme boundary was converted into geographic coordinates. Combined with the drone's turning angle limit, flight commands were generated: first, block the valley entrance to prevent the target from escaping, and then cover the slope to ensure area patrol, thus achieving a collaborative task of dynamic tracking and area coverage.

[0126] In a specific example, in step b1, the polygon area enclosed by the control points of the region boundary in the segmentation parameters is 5000 square meters, and the total area of ​​the unpatched area is 6000 square meters. The theoretical coverage is approximately 5000 / 6000≈0.83. The actual sensor coverage is reduced by 5% due to terrain, resulting in a coverage scalar of 0.83×(1-0.05)=0.79. Multiplying this by 0.3 in the first term of the reward function (coverage scalar) yields 0.237. In step b2, the priority weights for the entry point are 1.0 and for slopes 0.7. The high probability area for the entry point accounts for 60% of the movement probability heatmap, and the velocity vector field direction consistency is 85%. The matching degree scalar = 1.0×0.6×0.85 + 0.7×0.3×0.7≈0.51 +0.147=0.657. Multiplying this by 0.4 yields 0.263. In step b3, the spatial density of boundary control points is 5 per 1000 square meters. The distance from the UAV to the entrance is 800 meters, and the path curvature coefficient is 1.2. Since two turns are required, the energy consumption scalar is 5 × 800 × 1.2 = 4800, which is normalized to 0.2. A lower value is better. This is calculated by multiplying the third term by 0.3 and taking the negative value -0.06. In step b4, the multidimensional evaluation result is 0.237 + 0.263 - 0.06 = 0.44, which serves as the basis for model updates.

[0127] By executing steps b1 to b4, the embodiments of this application solve the problem of balancing coverage, tracking, and energy consumption objectives in traditional segmentation schemes through a multi-dimensional quantitative evaluation mechanism, thereby achieving interpretability of the reinforcement learning decision-making process and multi-objective collaborative optimization capability, and improving task adaptability and resource allocation rationality in dynamic environments.

[0128] In one possible embodiment, S12, constructing a movement probability heatmap based on the movement trajectory prediction sequence, includes:

[0129] Step 121: Decompose the predicted movement trajectory sequence into multiple discrete trajectory points containing spatial coordinates and observation times according to geographic grid units to form a discrete trajectory point set.

[0130] Among them, a geographic grid cell refers to a square geographic block that divides the target area into fixed-size regions. Each cell contains information on the longitude span, latitude span, and altitude range. A discrete trajectory point set refers to a set of points with spatiotemporal labels formed by deconstructing the predicted movement trajectory sequence. Each point contains latitude and longitude coordinates and the observation time accurate to the second.

[0131] In this embodiment, the target area is first divided into geographic grid cells of fixed size, each cell having a unique geographic coordinate range; then, each trajectory point in the movement trajectory prediction sequence is assigned to the corresponding geographic grid cell according to its spatial coordinates, and the observation time of the point is recorded; finally, a discrete trajectory point set containing the spatial coordinates and observation times of all trajectory points is formed.

[0132] Step 122: Calculate the spatial distance weight based on the preset spatial weight kernel function and the spatial coordinates of the discrete trajectory points, and calculate the time decay weight based on the preset time decay coefficient and the observation time of the discrete trajectory points.

[0133] The spatial weight kernel function refers to a weight calculation rule designed based on the distance decay law, such as a Gaussian function, where the closer to the grid center, the higher the weight. The time decay coefficient is a parameter that controls the rate at which the importance of historical trajectory points decays; a larger value indicates a higher weight for recent data.

[0134] In this embodiment, a preset spatial weight kernel function is first used to calculate the distance between each point in the discrete trajectory point set and the center of its grid cell. The larger the distance, the smaller the weight. The spatial distance weight is obtained by dividing the distance by a preset attenuation radius. Secondly, based on the difference between the observation time of the discrete trajectory point and the current time, a preset time attenuation coefficient is used. The larger the time difference, the smaller the weight. The time attenuation weight is obtained by multiplying the difference by the attenuation coefficient.

[0135] Step 123: Based on spatial distance weight and time decay weight, the spatial coordinates and observation time are fused to generate the initial thermal values ​​of the geographic grid cells.

[0136] The initial thermal value refers to the grid cell probability benchmark value calculated after integrating spatial and temporal weights, reflecting the spatiotemporal density of historical trajectory points in the region.

[0137] In this embodiment, firstly, for all discrete trajectory points within each geographic grid cell, their spatial distance weight and time decay weight are multiplied together to obtain a comprehensive weight; secondly, the comprehensive weights of all trajectory points within the cell are summed; finally, the sum is divided by the number of trajectory points within the cell to generate the initial thermal value of the geographic grid cell.

[0138] Step 124: Convert the initial heat values ​​of all geographic grid cells into color gradient distributions to form a motion probability heatmap.

[0139] Color gradient distribution refers to a visualization scheme that maps heat values ​​to shades of color, typically using red to represent high-probability areas.

[0140] In this embodiment, the initial heat values ​​of all geographic grid cells are first normalized to the range of 0 to 1; then the normalized values ​​are mapped to a preset color gradient, with higher heat values ​​corresponding to darker warning colors; finally, a motion probability heat map reflecting the probability distribution of target appearance is formed.

[0141] Here is a specific example: A drone scans a forest area using infrared and radar equipment, collecting raw data on abnormal heat sources and moving objects. Based on the location of the heat sources and radar trajectory points, it generates a predicted sequence of movement trajectories for suspicious individuals. Then, following step 121, this sequence is divided into geographic grid units, decomposing the target area into multiple discrete trajectory points. Each point contains spatial coordinates and an observation time, forming a set of discrete trajectory points. Next, in step 122, based on a preset spatial weighting function such as a Gaussian function, the spatial distance weight from each discrete trajectory point to the center of the geographic grid unit is calculated. Simultaneously, based on a preset time decay coefficient, the time decay weight of the time difference between the observation time and a reference time, such as the current time, is calculated. Then, in step 123, based on spatial distance weights and time decay weights, and by fusing spatial coordinates and observation time, an initial heat map is calculated for each geographic grid cell. The calculation formula is that the initial heat map is equal to the weighted sum of the spatial distance weights multiplied by the time decay weights of all discrete trajectory points. Finally, in step 124, the initial heat map values ​​of all geographic grid cells are converted into color gradient distributions to form a motion probability heat map. This heat map is then input into a reinforcement learning model, which, combined with the velocity vector field and the real-time position of the UAV, outputs a segmentation scheme for unpatrolled valleys. After the scheme boundary is converted into geographic coordinates, flight commands are generated by combining the UAV's turning angle constraints. The valley entrance is first blocked, and then the slope is covered, realizing a collaborative task of dynamic tracking and regional coverage.

[0142] In another specific example, in step 121, the target area is a 2km × 2km forest area, divided into 200 × 200 10 × 10 meter geographic grid units. The movement trajectory prediction sequence contains 100 trajectory points, each assigned to a corresponding grid, with recorded observation times such as 9:50:00, 9:51:00, etc., forming a discrete trajectory point set. In step 122, a Gaussian spatial weight kernel function is used, with a weight of 1.0 within 5 meters of the grid center and 0.1 beyond 10 meters. For a trajectory point 3 meters from the center, the spatial distance weight is calculated to be 0.8. The time decay coefficient is 0.9; if the observation time of this point differs from the current 10:00 by 5 minutes, the time decay weight is 0.9^5 ≈ 0.59. In step 123, a certain grid contains 5 trajectory points with comprehensive weights of 0.8×0.59, 0.7×0.65, etc., totaling 2.8. The initial heat value is 2.8 / 5 = 0.56. In step 124, all grid heat values ​​are normalized to 0-1. Values ​​above 0.8 are displayed in dark red, 0.5-0.8 in orange, 0.2-0.5 in yellow, and below in blue. A motion probability heat map is generated, clearly showing the high-probability area at the valley entrance.

[0143] By executing steps 121 to 124, this embodiment of the application solves the problems of ignoring the timeliness of historical trajectories and the fragmentation of spatial continuity in traditional heat maps through a spatiotemporal dual-weight fusion mechanism, thereby achieving a precise quantitative expression of the probability of target occurrence and improving the credibility of predicting suspicious target behavior in dynamic environments.

[0144] In one possible embodiment, S13, performing Doppler frequency shift analysis on the radar data to generate a target velocity vector field, includes:

[0145] Step 131: Deconstruct the radar data into a set of discrete echo cells, each of which contains azimuth, elevation and original frequency shift.

[0146] The discrete echo cell set refers to the smallest set of processing units that divide radar scan data according to azimuth and elevation angles. Each unit contains angle information and raw frequency shift data. Azimuth refers to the horizontal angle between the target and the UAV's true north direction, with clockwise angle values ​​ranging from 0 to 360 degrees. Elevation refers to the vertical angle between the line connecting the target and the UAV and the horizontal plane, with upward values ​​being positive. Raw frequency shift refers to the initial difference between the radar echo frequency and the transmitted frequency, including velocity information caused by the Doppler effect and measurement noise.

[0147] In this embodiment, the raw echo data received by the radar is first divided into independent units according to the scanning azimuth and elevation angle. Each unit contains the target's horizontal azimuth angle, vertical tilt angle and unprocessed frequency offset relative to the UAV, forming a discrete echo unit set.

[0148] Step 132: Perform periodic folding correction on the original frequency shift to generate the true Doppler frequency shift value.

[0149] Periodic folding correction refers to a technique for eliminating radar velocity ambiguity. When the frequency shift exceeds the maximum unambiguous frequency shift, an integer multiple of the wavelength is added or subtracted to bring it within the measurable range. The true Doppler frequency shift refers to the frequency offset that reflects the true radial velocity of the target after correction.

[0150] In this embodiment, it is first identified whether the original frequency shift exceeds the maximum measurable frequency shift corresponding to half the radar wavelength. When it exceeds this value, the frequency shift period value corresponding to an integer multiple of the wavelength is added or subtracted to correct the frequency shift and eliminate measurement ambiguity. Then, the corrected true Doppler frequency shift value is output, which reflects the true relative speed between the target and the UAV.

[0151] Step 133: Convert the true Doppler frequency shift value into radial velocity components, and calculate the three-dimensional velocity projection by combining the azimuth and elevation angles.

[0152] The radial velocity component refers to the velocity component of the target along the radar beam direction, calculated by multiplying the frequency shift by the wavelength and dividing by twice the radar frequency. Three-dimensional velocity projection decomposes the radial velocity into velocity components in three orthogonal directions: east and west, south and north, and up and down, based on azimuth and elevation angles.

[0153] In this embodiment, the radial velocity component is first calculated by dividing the product of the radar wavelength and the actual Doppler frequency shift by twice the radar frequency; then, the velocity components of the target in the east-west, north-south, and vertical directions are calculated by combining the azimuth and elevation angles recorded by the unit through trigonometric functions; finally, a three-dimensional velocity projection is synthesized.

[0154] Step 134: Perform motion consistency aggregation on the three-dimensional velocity projection to obtain the target velocity vector field.

[0155] Among them, motion consistency aggregation refers to the process of clustering spatially adjacent velocity vectors with similar motion directions into a group of motion vectors, which includes direction consistency detection and vector weighted fusion.

[0156] In this embodiment, firstly, the directional similarity detection of the three-dimensional velocity projections of neighboring units is performed, and projections with a moving direction angle less than a preset threshold are classified into the same motion group; secondly, the velocity vectors of the same type of projections are weighted and averaged, with the weight determined by the signal strength; finally, a target velocity vector field reflecting the motion trend of the group is generated.

[0157] Here is a specific example: After the UAV collects raw data of a moving target by scanning a forest area using its airborne radar, step 131 decomposes the radar data into multiple discrete echo units. Each unit contains azimuth angle information, elevation angle information, and the original frequency shift value. Then, in step 132, the original frequency shift is periodically folded for correction. When the original frequency shift exceeds the maximum measurable frequency shift range of the radar, it is adjusted using the formula: the true Doppler frequency shift value equals the original frequency shift value minus twice the maximum unambiguous frequency shift multiplied by an integer adjustment factor k. The integer adjustment factor k is determined based on the echo continuity, thus generating the true Doppler frequency shift value. Then, in step 133, the true Doppler frequency shift value is converted into a radial velocity component. The conversion formula is: the radial velocity component equals the true Doppler frequency shift value multiplied by the speed of light divided by twice the radar transmission frequency. Combining this with the azimuth angle θ and elevation angle φ, the target's east-west velocity component v_e is calculated, which is equal to the radial velocity component multiplied by cosθ. Multiplying by sinφ, the north-south velocity component v_n is equal to the radial velocity component multiplied by sinθ and sinφ, and the vertical velocity component v_u is equal to the radial velocity component multiplied by cosφ, forming a three-dimensional velocity projection. Finally, in step 134, motion consistency aggregation is performed on the three-dimensional velocity projection. A spatial weighted average method is used to filter and weight all velocity vectors within a preset radius around each location point based on directional similarity. The weights are determined by the square of the negative distance Gaussian function exp divided by twice the spatial scale parameter sigma squared. Finally, a target velocity vector field is generated, which shows that the target group is moving as a whole towards the northwest in an unpatrolled valley. This result, together with the heat map and the UAV position, is input into the reinforcement learning model to output a segmentation scheme for the unpatrolled valley. The scheme boundary is converted into geographic coordinates and the UAV turning angle limitation is taken into account to generate a flight path instruction that first blocks the valley entrance and then covers the slope, realizing the collaborative task of dynamic tracking and regional coverage.

[0158] In another specific example, in step 131, the radar data is deconstructed into 1000 discrete echo cells. Each cell records the azimuth angle (e.g., 35°), elevation angle (-1°), and original frequency shift (e.g., +1200Hz), corresponding to a velocity exceeding the radar's maximum measurable value of 10m / s. In step 132, with a radar wavelength of 0.1 meters, the maximum unambiguous frequency shift is calculated as (2 × transmission frequency × 10m / s) / 0.1 = 200 × transmission frequency. The original frequency shift of +1200Hz exceeds this value and is corrected to 1200 - 2 × 1000 = -800Hz. Assuming a maximum frequency shift of 1000Hz, the actual Doppler frequency shift is -800Hz. In step 133, the radial velocity is calculated as (-800Hz × 0.1m) / (2 × transmission frequency). If the transmission frequency is 10GHz, the radial velocity is approximately -1.2m / s. Combining the azimuth angle of 35° and the elevation angle of -1°, the three-dimensional velocity projection is calculated: Eastward = -1.2×sin35°≈-0.68m / s, Northward = -1.2×cos35°≈-0.98m / s, and Vertical ≈0. In step 134, all three-dimensional velocity projections are clustered, with those having an eastward deviation <0.2m / s and a northward deviation <0.2m / s grouped together, accounting for 80%. The average velocity is -0.7m / s eastward and -1.0m / s northward. A target velocity vector field is generated, with the arrow pointing northwest and the length representing the velocity magnitude.

[0159] By executing steps 131 to 134, this embodiment of the application solves the problems of directional ambiguity and multi-target velocity dispersion in traditional radar velocity measurement through periodic folding correction and motion consistency aggregation, realizes accurate extraction of group motion vectors, and improves the reliability of target behavior pattern analysis in complex scenarios.

[0160] In one possible embodiment, S12, based on infrared data and radar data, generates a predicted sequence of the movement trajectory of the suspected target within the target area, including:

[0161] Step c1: Based on the temperature gradient distribution corresponding to the infrared data, extract the center coordinates of abnormal hot spots in the inspected area where the temperature gradient distribution is higher than the preset temperature difference threshold. Sort all the center coordinates of abnormal hot spots according to the acquisition timestamp to generate a target hot spot location sequence.

[0162] Among them, the temperature gradient distribution refers to the distribution of the rate of temperature change between adjacent pixels in the infrared image, reflecting the trend of heat transfer on the object's surface. The preset temperature difference threshold is a manually set value for judging sudden temperature changes; when the temperature difference between adjacent areas exceeds this value, an abnormal alarm is triggered. The coordinates of the abnormal hot spot center refer to the geometric center point location data of the temperature change area, including longitude, latitude, and altitude. The acquisition timestamp refers to the precise time record when the sensor detected the hot spot, in the format of year, month, day, hour, minute, second, and millisecond. The target hot spot position sequence refers to the set of abnormal hot spot center coordinates ordered by time, forming a discrete position chain of the target's movement.

[0163] In this embodiment, the distribution of temperature change rate at each location in the infrared data is first analyzed, and the coordinates of the center point of the abnormal high temperature area with temperature change exceeding the preset warning value are found in the area scanned by the UAV. Then, the coordinates of all the center points are arranged in the order of the time when these high temperature points were detected, and a target hot spot location sequence with time stamp is generated.

[0164] Step c2: Within a preset time window, fuse the spatial coordinates of the target hotspot location sequence with the trajectory segments of the radar data to generate a fused trajectory point set.

[0165] In this context, a trajectory segment refers to a line segment of the target's motion path detected by the radar within a single scan cycle. A fused trajectory point set refers to a unified coordinate dataset formed by matching the coordinates of infrared hotspots with the radar trajectory coordinates. The difference between a fused trajectory point set and a discrete trajectory point set is that a discrete trajectory point set is a primary product of single-sensor data, generated solely from the time-stamped coordinates of the infrared anomalous hotspot centers. Each point only contains its spatial location and observation time, without any interaction or verification with other sensor data. In contrast, a fused trajectory point set is the result of collaborative processing by multiple sensors. It is generated by spatially matching and fusing the target hotspot position sequence generated by infrared sensors with the trajectory segments detected by radar within a preset time window. Each point must simultaneously satisfy the spatiotemporal consistency condition, essentially representing a cross-validation and enhancement of the discrete trajectory point set.

[0166] In this embodiment, a fixed time interval is first set, and the coordinates of the target hotspot position sequence are matched with the coordinates of the moving target trajectory detected by the radar within the interval. Then, the average value of the successfully matched coordinates is taken, and the original data of the unmatched points is retained. Finally, a fused trajectory point set containing both infrared and radar data is generated.

[0167] Step c3: Connect the fused trajectory point sets according to the timestamp order to form a trajectory chain.

[0168] The trajectory chain refers to a sequence of fused trajectory points connected in chronological order, describing the continuous movement path of the target.

[0169] In this embodiment, all coordinate points in the fused trajectory point set are first sorted from earliest to latest according to the detection time; then adjacent coordinate points are connected by straight lines in chronological order; finally, a trajectory chain describing the continuous movement path of the target is formed.

[0170] Step c4: Based on the consistency of velocity vector direction, extend the trajectory according to the trajectory chain and output the predicted sequence of the movement trajectory of the suspicious target in the target area.

[0171] Trajectory extension refers to a prediction method that extends the trajectory chain into undetected areas based on the inertia of historical motion directions.

[0172] In this embodiment, the direction change angle of adjacent line segments in the trajectory chain is first analyzed, and continuous line segments with basically the same movement direction are selected; then the trajectory chain is extended along this direction to the boundary of the target area; finally, the predicted movement trajectory sequence of the suspicious target in the future area is output.

[0173] Here is a specific example: In step c1, based on the temperature gradient distribution of the infrared data, a preset temperature difference threshold of 5℃ is used to extract the center coordinates of three abnormal hot spots: 116.5°E, 40.2°N, 116.502°E, 40.199°N, and 116.503°E, 40.198°N. These coordinates are then sorted according to the acquisition timestamps 9:50, 9:52, and 9:54 to generate a target hot spot location sequence. In step c2, within a preset time window of 1 minute, between 9:50 and 9:51, hot spot A is found to be within 5 meters of the radar trajectory point 116.501°E, 40.199°N, indicating a successful match. The average coordinates are used as the fusion point. Unmatched points are retained, generating a fused trajectory point set containing A, B, C, and two radar-unique points. In step c3, the trajectory points are connected and merged according to the timestamps 9:50→9:51→9:52→9:54 to form a trajectory chain, presenting a continuous path from southeast to northwest, with an average movement speed of 1.2 m / s. In step c4, the movement direction of the trajectory chain's end point at 9:54 is northwest, with an azimuth angle of 315°. The directional deviation of the first three points is less than 5°, indicating high consistency. Based on this direction and speed, the trajectory points for the next 10 minutes are predicted: from 9:56 to 116.504°E, 40.197°N, and from 10:00 to the valley entrance at 116.505°E, 40.196°N, forming a predicted movement trajectory sequence.

[0174] By executing steps c1 to c4, this embodiment of the application solves the problems of trajectory breakage and prediction blind spots of a single sensor through the spatiotemporal fusion of infrared and radar data and the extension of motion inertia, and realizes continuous reconstruction of target movement path and behavior prediction in complex environments.

[0175] Figure 2A schematic diagram of a multi-objective optimization-based autonomous decision-making reinforcement learning system for unmanned aerial vehicles (UAVs) is provided as an embodiment of this application, such as... Figure 2 As shown, the system includes:

[0176] The data acquisition module 21 is used to acquire infrared and radar data in the patrolled area through an airborne sensor array deployed on the UAV.

[0177] Module 22 is used to generate a movement trajectory prediction sequence of a suspicious target within the target area based on infrared and radar data, and to construct a movement probability heat map based on the movement trajectory prediction sequence. The target area includes the patrolled area.

[0178] The generation module 23 is used to perform Doppler frequency shift analysis on radar data and generate the target velocity vector field.

[0179] Output module 24 is used to take the motion probability heatmap and target velocity vector field as input features, and combine them with the UAV's position data. Through a pre-trained region re-partitioning reinforcement learning model based on a reward function for multi-objective optimization, it outputs the UAV autonomous decision reinforcement learning results containing the dynamic segmentation scheme of unpatched areas. The reward function is used to reflect the maximization of region coverage, the maximization of target tracking success probability, and the minimization of path energy consumption.

[0180] Figure 2 The aforementioned multi-objective optimization-based UAV autonomous decision-making reinforcement learning system can execute... Figure 1 The implementation principle and technical effects of the UAV autonomous decision-making reinforcement learning method based on multi-objective optimization described in the illustrated embodiment will not be repeated here. The specific methods by which each module and unit of the UAV autonomous decision-making reinforcement learning system based on multi-objective optimization in the above embodiments are described in detail in the embodiments related to this method, and will not be elaborated upon here.

[0181] In one possible design, Figure 2 The illustrated embodiment of the UAV autonomous decision-making reinforcement learning system based on multi-objective optimization can be implemented as a computing device, such as... Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32.

[0182] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are invoked and executed by the processing component 32.

[0183] The processing component 32 is used to: collect infrared and radar data within the patrolled area using an airborne sensor array deployed on the UAV; generate a predicted trajectory sequence of a suspicious target within the target area based on the infrared and radar data; construct a motion probability heatmap based on the motion trajectory prediction sequence, with the target area including the patrolled area; perform Doppler frequency shift analysis on the radar data to generate a target velocity vector field; use the motion probability heatmap and the target velocity vector field as input features, and combine them with the UAV's position data, outputting the UAV's autonomous decision-making reinforcement learning result, which includes a dynamic segmentation scheme for unpatched areas, through a pre-trained region re-segmentation reinforcement learning model based on a reward function for multi-objective optimization; the reward function is used to maximize area coverage, maximize the probability of successful target tracking, and minimize path energy consumption.

[0184] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above-described method. Alternatively, the processing component may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0185] Storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as Random Access Memory (RAM), Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0186] Of course, computing devices may also include other components, such as input / output interfaces, display components, communication components, etc.

[0187] Input / output interfaces provide interfaces between processing components and peripheral interface modules, which can be output devices, input devices, etc.

[0188] The communication components are configured to facilitate wired or wireless communication between computing devices and other devices.

[0189] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server, and the aforementioned processing components, storage components, etc., can be basic server resources rented or purchased from the cloud computing platform.

[0190] This application also provides a computer storage medium storing a computer program, which, when executed by a computer, can perform the above-described functions. Figure 1 The embodiment shown is a reinforcement learning method for autonomous decision-making of unmanned aerial vehicles based on multi-objective optimization.

[0191] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0192] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0193] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A reinforcement learning method for autonomous decision-making in unmanned aerial vehicles (UAVs) based on multi-objective optimization, characterized in that, include: Infrared and radar data are collected from the patrolled area using an airborne sensor array deployed on the drone. Based on the infrared data and the radar data, a movement trajectory prediction sequence of the suspicious target within the target area is generated, and a movement probability heat map is constructed based on the movement trajectory prediction sequence. The target area includes the patrolled area. Doppler frequency shift analysis is performed on the radar data to generate the target velocity vector field; Using the motion probability heatmap and the target velocity vector field as input features, and combined with the UAV's position data, a pre-trained region repartition reinforcement learning model is used to output the UAV autonomous decision reinforcement learning result with a dynamic segmentation scheme for unpatched areas based on a reward function for multi-objective optimization. The reward function is used to reflect maximizing region coverage, maximizing the probability of successful target tracking, and minimizing path energy consumption. The process involves using the motion probability heatmap and the target velocity vector field as input features, combined with the UAV's position data, and employing a pre-trained region re-partitioning reinforcement learning model based on a reward function for multi-objective optimization to output a UAV autonomous decision-making reinforcement learning result that includes a dynamic segmentation scheme for uninspected areas. This includes: Extract the probability distribution features of the movement probability heatmap and the probability distribution features of the target velocity vector field, and then perform spatiotemporal correlation and fusion of the probability distribution features and the UAV position data to generate a joint state feature vector; The joint state feature vector is input into the region repartition reinforcement learning model, and the policy network outputs the segmentation parameter set of the uninspected region. Based on the set of segmentation parameters, a multidimensional evaluation is performed using the reward function for multi-objective optimization to generate a multidimensional evaluation result. Based on the multidimensional evaluation results, the policy network of the region re-partitioning reinforcement learning model is driven to update the weight parameters and output a dynamic segmentation scheme for the uninspected region. The dynamic segmentation scheme is bound to the real-time environmental state to generate UAV autonomous decision reinforcement learning results that include the dynamic segmentation scheme for uninspected areas. The construction of the movement probability heatmap based on the movement trajectory prediction sequence includes: The predicted movement trajectory sequence is divided into geographic grid units, and the target area is decomposed into multiple discrete trajectory points containing spatial coordinates and observation time to form a discrete trajectory point set. The spatial distance weight is calculated based on the preset spatial weight kernel function and the spatial coordinates of the discrete trajectory points, and the time decay weight is calculated based on the preset time decay coefficient and the observation time of the discrete trajectory points. Based on the spatial distance weight and time decay weight, the spatial coordinates and the observation time are fused to generate the initial thermal value of the geographic grid cell; The initial heat values ​​of all the geographic grid cells are converted into color gradient distributions to form a motion probability heatmap; The step of performing Doppler frequency shift analysis on the radar data to generate the target velocity vector field includes: The radar data is deconstructed into a set of discrete echo cells, each of which contains azimuth, elevation and original frequency shift. The original frequency shift is periodically folded to generate the true Doppler frequency shift value; The true Doppler frequency shift value is converted into a radial velocity component, and combined with the azimuth angle and the elevation angle, the three-dimensional velocity projection is calculated. Motion consistency aggregation is performed on the three-dimensional velocity projection to obtain the target velocity vector field.

2. The UAV autonomous decision-making reinforcement learning method based on multi-objective optimization according to claim 1, characterized in that, The step of binding the dynamic segmentation scheme with the real-time environmental state to generate UAV autonomous decision reinforcement learning results that include the dynamic segmentation scheme for uninspected areas includes: Based on the boundary information of the dynamic segmentation scheme, the geometric vertex coordinates of the unpatched area and the geospatial coordinates in the real-time environmental state are extracted, a mapping relationship between the geometric vertex coordinates and the geospatial coordinates is established, and a spatial coordinate mapping table is generated. Based on the time window constraints in the real-time environmental state and the motion dynamics parameters of the UAV, the priority information of the dynamic segmentation scheme is converted into timestamp markers of the path execution sequence to generate execution timing logic; The spatial coordinate mapping table and execution timing logic are integrated to construct a structured decision data flow; The structured decision data stream is encoded to generate reinforcement learning results for UAV autonomous decision-making.

3. The UAV autonomous decision-making reinforcement learning method based on multi-objective optimization according to claim 1, characterized in that, The step of performing multidimensional evaluation based on the segmentation parameter set and using the reward function for multi-objective optimization to generate multidimensional evaluation results includes: Based on the regional boundary control points in the segmentation parameter set, the spatial coverage blind zone distribution of the unpatched area is calculated using the first term in the reward function, and a coverage scalar corresponding to the first target is generated, where the first target is the regional coverage rate. Based on the partitioning priority weights in the segmentation parameter set, combined with the high-probability target distribution of the moving probability heatmap and the consistency of the motion direction of the target velocity vector field, the matching degree scalar corresponding to the second target is generated using the second term in the reward function. The second target is the probability of successful tracking of a suspicious target. Based on the spatial density of boundary control points in the segmentation parameter set and the current position of the UAV, the energy consumption scalar corresponding to the third target is generated using the third term in the reward function, where the third target is path energy consumption. The coverage scalar, matching degree scalar, and energy consumption scalar are linearly superimposed to generate the multidimensional evaluation result.

4. The UAV autonomous decision-making reinforcement learning method based on multi-objective optimization according to claim 1, characterized in that, The step of generating a predicted trajectory sequence of a suspicious target within the target area based on the infrared data and the radar data includes: Based on the temperature gradient distribution corresponding to the infrared data, the center coordinates of abnormal hot spots whose temperature gradient distribution is higher than the preset temperature difference threshold are extracted in the inspected area. All abnormal hot spot center coordinates are sorted according to the acquisition timestamp to generate a target hot spot location sequence. Within a preset time window, the spatial coordinates of the target hotspot location sequence and the trajectory segments of radar data are fused to generate a fused trajectory point set; The fused trajectory point set is connected in timestamp order to form a trajectory chain; Based on the consistency of velocity vector direction, the trajectory is extended according to the trajectory chain, and a predicted sequence of the movement trajectory of the suspicious target within the target area is output.

5. A reinforcement learning system for autonomous decision-making in unmanned aerial vehicles (UAVs) based on multi-objective optimization, characterized in that, include: The data acquisition module is used to collect infrared and radar data in the patrolled area through an airborne sensor array deployed on the UAV. The module is used to generate a movement trajectory prediction sequence of a suspicious target within a target area based on the infrared data and the radar data, and to construct a movement probability heatmap based on the movement trajectory prediction sequence. The target area includes the patrolled area. The generation module is used to perform Doppler frequency shift analysis on the radar data and generate a target velocity vector field; The output module is used to take the movement probability heatmap and the target velocity vector field as input features, and combine them with the UAV's position data. Through a pre-trained region re-partitioning reinforcement learning model based on a reward function for multi-objective optimization, it outputs the UAV autonomous decision reinforcement learning results containing the dynamic segmentation scheme of unpatched areas. The reward function is used to reflect the maximization of region coverage, the maximization of target tracking success probability, and the minimization of path energy consumption. The process involves using the motion probability heatmap and the target velocity vector field as input features, combined with the UAV's position data, and employing a pre-trained region re-partitioning reinforcement learning model based on a reward function for multi-objective optimization to output a UAV autonomous decision-making reinforcement learning result that includes a dynamic segmentation scheme for uninspected areas. This includes: Extract the probability distribution features of the movement probability heatmap and the probability distribution features of the target velocity vector field, and then perform spatiotemporal correlation and fusion of the probability distribution features and the UAV position data to generate a joint state feature vector; The joint state feature vector is input into the region repartition reinforcement learning model, and the policy network outputs the segmentation parameter set of the uninspected region. Based on the set of segmentation parameters, a multidimensional evaluation is performed using the reward function for multi-objective optimization to generate a multidimensional evaluation result. Based on the multidimensional evaluation results, the policy network of the region re-partitioning reinforcement learning model is driven to update the weight parameters and output a dynamic segmentation scheme for the uninspected region. The dynamic segmentation scheme is bound to the real-time environmental state to generate UAV autonomous decision reinforcement learning results that include the dynamic segmentation scheme for uninspected areas. The construction of the movement probability heatmap based on the movement trajectory prediction sequence includes: The predicted movement trajectory sequence is divided into geographic grid units, and the target area is decomposed into multiple discrete trajectory points containing spatial coordinates and observation time to form a discrete trajectory point set. The spatial distance weight is calculated based on the preset spatial weight kernel function and the spatial coordinates of the discrete trajectory points, and the time decay weight is calculated based on the preset time decay coefficient and the observation time of the discrete trajectory points. Based on the spatial distance weight and time decay weight, the spatial coordinates and the observation time are fused to generate the initial thermal value of the geographic grid cell; The initial heat values ​​of all the geographic grid cells are converted into color gradient distributions to form a motion probability heatmap; The step of performing Doppler frequency shift analysis on the radar data to generate the target velocity vector field includes: The radar data is deconstructed into a set of discrete echo cells, each of which contains azimuth, elevation and original frequency shift. The original frequency shift is periodically folded to generate the true Doppler frequency shift value; The true Doppler frequency shift value is converted into a radial velocity component, and combined with the azimuth angle and the elevation angle, the three-dimensional velocity projection is calculated. Motion consistency aggregation is performed on the three-dimensional velocity projection to obtain the target velocity vector field.

6. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a multi-objective optimization-based UAV autonomous decision reinforcement learning method as described in any one of claims 1-4.

7. A computer storage medium, characterized in that, The system contains a computer program that, when executed by a computer, implements a multi-objective optimization-based reinforcement learning method for autonomous decision-making of unmanned aerial vehicles as described in any one of claims 1-4.

Images