Unmanned aerial vehicle autonomous decision reinforcement learning method, system and device based on multi-objective optimization, and storage medium

Abstract

The invention relates to the technical field of unmanned aerial vehicle autonomous decision, provides an unmanned aerial vehicle autonomous decision reinforcement learning method, system and device based on multi-objective optimization, and a storage medium, and solves the problems of limited dynamic task distribution capability and poor decision capability of an unmanned aerial vehicle. The method comprises the steps that based on infrared data and radar data, a moving track prediction sequence of a suspicious target in a target area is generated, a moving probability thermodynamic diagram is constructed, and the target area comprises a patrolled area; performing Doppler frequency shift analysis on the radar data to generate a target velocity vector field; and in combination with the position data of the unmanned aerial vehicle, outputting an unmanned aerial vehicle autonomous decision reinforcement learning result of a dynamic segmentation scheme containing an unpatrolled region through a region re-division reinforcement learning model based on a reward function for multi-objective optimization. The reward function is used for reflecting area coverage rate maximization, target tracking success probability maximization and path energy consumption minimization. According to the invention, autonomous intelligent decision-making of the unmanned aerial vehicle on the unpatrolled area is realized.

Description

Technical Field

[0001] The present application relates to the technical field of autonomous decision-making for unmanned aerial vehicles (UAVs), and in particular to a reinforcement learning method and system for autonomous decision-making for UAVs based on multi-objective optimization. Background Art

[0002] In scenarios such as disaster search and rescue and border patrols, multiple drone systems must collaborate to achieve dynamic target tracking and area coverage. This task presents three core challenges. First, infrared and radar sensors must be integrated to detect targets with weak heat signatures or obscured areas in complex environments. Second, target movement is highly random, requiring real-time motion modeling to improve tracking success rates. Third, area coverage, target capture probability, and drone energy consumption must be simultaneously optimized, with trade-offs existing between these objectives.

[0003] The current solution utilizes an action mask and multi-agent proximal policy optimization algorithm, combining wide-area detection by high-altitude drones with detailed scanning by low-altitude drones. Reinforcement learning optimizes path planning to maximize target capture and minimize regional uncertainty. Action masking rules are introduced to constrain collision paths, and a field-of-view encoding state representation handles input dimensionality variations, accelerating algorithm convergence. A reward function integrates area coverage, target capture, and path length to drive the agents to balance exploration and exploitation.

[0004] Existing solutions still rely on real-time sensor data and fail to integrate historical patrol information to build a target movement probability model, resulting in weak prediction capabilities for recurring hidden targets. They ignore Doppler shift analysis, making it impossible to generate a target velocity vector field and struggling to cope with sudden maneuvers. The reward function focuses on target capture and area coverage, failing to incorporate path energy consumption into the optimization objective, and lacks dynamic task allocation capabilities. Regional segmentation relies on a preset geographic grid and fails to re-partition unpatrolled areas in real time based on target probability heat maps, resulting in inefficient resource allocation. Summary of the Invention

[0005] The present application provides a reinforcement learning method and system for autonomous decision-making of unmanned aerial vehicles (UAVs) based on multi-objective optimization, which is used to solve the problems in the existing technology such as insufficient historical information fusion leading to target trajectory prediction deviation, lack of dynamic target velocity field modeling, incomplete multi-objective optimization dimensions and limited dynamic task allocation capabilities, and rigid regional resource allocation mechanisms, which ultimately lead to poor autonomous decision-making capabilities of UAVs.

[0006] In the first aspect, the present application provides a reinforcement learning method for autonomous decision-making of drones based on multi-objective optimization, comprising: Collect infrared and radar data from the patrolled area through the airborne sensor array deployed on the drone; generating a movement trajectory prediction sequence of a suspicious target within a target area based on the infrared data and the radar data, and constructing a movement probability heat map based on the movement trajectory prediction sequence, wherein the target area includes an inspected area; Performing Doppler shift analysis on the radar data to generate a target velocity vector field; Taking the movement probability heat map and the target velocity vector field as input features and combining them with the UAV's position data, a pre-trained area redivision reinforcement learning model is used to output the UAV's autonomous decision-making reinforcement learning results, which include a dynamic segmentation scheme for uninspected areas, based on a reward function for multi-objective optimization. The reward function is used to maximize area coverage, maximize the probability of target tracking success, and minimize path energy consumption.

[0007] Optionally, the method uses the movement probability heat map and the target velocity vector field as input features, and combines them with the position data of the UAV, and outputs the UAV autonomous decision-making reinforcement learning results including a dynamic segmentation scheme for uninspected areas based on a reward function for multi-objective optimization through a pre-trained region redivision reinforcement learning model, including: Extracting probability distribution features of the movement probability heat map and probability distribution features of the target velocity vector field, and performing spatiotemporal correlation fusion of the probability distribution features and the position data of the UAV to generate a joint state feature vector; Inputting the joint state feature vector into the region redivision reinforcement learning model, and outputting a set of segmentation parameters for the uninspected region through a policy network; Based on the segmentation parameter set, performing a multi-dimensional evaluation using the reward function for multi-objective optimization to generate a multi-dimensional evaluation result; Based on the multi-dimensional evaluation results, the policy network of the area redivision reinforcement learning model is driven to update weight parameters and output a dynamic segmentation scheme for the uninspected area; The dynamic segmentation scheme is bound to the real-time environmental status to generate the autonomous decision-making reinforcement learning results of the UAV including the dynamic segmentation scheme of the uninspected area.

[0008] Optionally, the step of binding the dynamic segmentation scheme with the real-time environmental state to generate a UAV autonomous decision-making reinforcement learning result including the dynamic segmentation scheme for the uninspected area includes: Extracting the geometric vertex coordinates of the uninspected area and the geographic space coordinates in the real-time environment state based on the boundary information of the dynamic segmentation scheme, establishing a mapping relationship between the geometric vertex coordinates and the geographic space coordinates, and generating a spatial coordinate mapping table; According to the time window constraint in the real-time environment state and the motion dynamics parameters of the UAV, the priority information of the dynamic segmentation scheme is converted into a timestamp tag of the path execution sequence to generate an execution timing logic; The spatial coordinate mapping table is integrated with the execution timing logic to construct a structured decision data flow; The structured decision data stream is encoded to generate a UAV autonomous decision reinforcement learning result.

[0009] Optionally, performing a multi-dimensional evaluation based on the segmentation parameter set and utilizing the reward function for multi-objective optimization to generate a multi-dimensional evaluation result includes: Based on the area boundary control points in the segmentation parameter set, using the first term in the reward function to calculate the spatial coverage blind area distribution of the uninspected area, and generate a coverage scalar corresponding to a first target, where the first target is the area coverage rate; Based on the segmentation priority weights in the segmentation parameter set, combined with the high-probability target distribution in the movement probability heat map and the motion direction consistency of the target velocity vector field, a matching degree scalar corresponding to a second target is generated using the second term in the reward function, where the second target is the probability of successful tracking of the suspicious target; Based on the spatial density of the boundary control points in the segmentation parameter set and the current position of the UAV, using the third term in the reward function to generate an energy consumption scalar corresponding to a third goal, wherein the third goal is the path energy consumption; The coverage scalar, matching scalar and energy consumption scalar are linearly superimposed to generate the multi-dimensional evaluation result.

[0010] Optionally, constructing a movement probability heat map based on the movement trajectory prediction sequence includes: Deconstructing the target area into a plurality of discrete trajectory points including spatial coordinates and observation times according to the movement trajectory prediction sequence according to geographic grid units to form a discrete trajectory point set; Calculating a spatial distance weight based on a preset spatial weight kernel function and the spatial coordinates of the discrete trajectory points, and calculating a time decay weight based on a preset time decay coefficient and the observation time of the discrete trajectory points; Based on the spatial distance weight and the time decay weight, the spatial coordinates and the observation time are fused to generate an initial thermal value of the geographic grid cell; The initial thermal values ​​of all the geographic grid cells are converted into a color gradient distribution to form a movement probability heat map.

[0011] Optionally, performing Doppler frequency shift analysis on the radar data to generate a target velocity vector field includes: Deconstructing the radar data into a set of discrete echo units, each discrete echo unit including an azimuth angle, an elevation angle, and an original frequency shift; Performing periodic folding correction on the original frequency shift to generate a true Doppler frequency shift value; Converting the true Doppler frequency shift value into a radial velocity component, and calculating a three-dimensional velocity projection by combining the azimuth angle and the elevation angle; Perform motion consistency aggregation on the three-dimensional velocity projections to obtain a target velocity vector field.

[0012] Optionally, generating a movement trajectory prediction sequence of a suspicious target within a target area based on the infrared data and the radar data includes: Based on the temperature gradient distribution corresponding to the infrared data, the center coordinates of abnormal hot spots whose temperature gradient distribution is higher than a preset temperature difference threshold are extracted in the inspected area, and the center coordinates of all abnormal hot spots are sorted according to the acquisition timestamp to generate a target hot spot position sequence; Within a preset time window, the target hot spot position sequence is fused with the spatial coordinates of the trajectory segment of the radar data to generate a fused trajectory point set; Connecting the fused trajectory point set in the order of the timestamps to form a trajectory chain; According to the consistency of the velocity vector direction, the trajectory is extended based on the trajectory chain to output a movement trajectory prediction sequence of the suspicious target in the target area.

[0013] Secondly, this application provides a drone autonomous decision-making reinforcement learning system based on multi-objective optimization, including: The acquisition module is used to collect infrared data and radar data within the patrolled area through the airborne sensor array deployed on the UAV; A construction module is configured to generate a movement trajectory prediction sequence of a suspicious target within a target area based on the infrared data and the radar data, and to construct a movement probability heat map based on the movement trajectory prediction sequence, wherein the target area includes an inspected area; A generation module for performing Doppler frequency shift analysis on the radar data to generate a target velocity vector field; The output module is used to take the movement probability heat map and the target velocity vector field as input features, and combine them with the position data of the drone, and output the drone autonomous decision-making reinforcement learning results including the dynamic segmentation scheme of the uninspected area based on the reward function for multi-objective optimization through a pre-trained area redivision reinforcement learning model. The reward function is used to reflect the maximization of area coverage, the maximization of the probability of target tracking success, and the minimization of path energy consumption.

[0014] In a third aspect, the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a multi-objective optimization-based drone autonomous decision-making reinforcement learning method as described in any one of the first aspects.

[0015] In a fourth aspect, the present application provides a computer storage medium storing a computer program. When the computer program is executed by a computer, it implements a drone autonomous decision-making reinforcement learning method based on multi-objective optimization as described in any one of the first aspects.

[0016] In the present application, a reinforcement learning method for autonomous decision-making of a UAV based on multi-objective optimization is provided, the method comprising: collecting infrared data and radar data in an inspected area through an airborne sensor array deployed on a UAV; generating a movement trajectory prediction sequence of a suspicious target in a target area based on the infrared data and radar data, and constructing a movement probability heat map based on the movement trajectory prediction sequence, wherein the target area includes the inspected area; performing Doppler frequency shift analysis on the radar data to generate a target velocity vector field; using the movement probability heat map and the target velocity vector field as input features, and combining them with the position data of the UAV, a pre-trained region redivision reinforcement learning model is used based on a reward function for multi-objective optimization to output a reinforcement learning result for autonomous decision-making of the UAV including a dynamic segmentation scheme for an uninspected area, wherein the reward function is used to reflect the maximization of area coverage, the maximization of the probability of successful target tracking, and the minimization of path energy consumption.

[0017] Beneficial effects of this application: This application realizes the fusion of multi-source heterogeneous perception, breaks through the detection blind spots of a single sensor in complex environments, and improves the reliability of initial target positioning; predicts the long-term behavior patterns of the target through time series modeling, converts discrete detection points into continuous probability distribution, and solves the problem of sudden target trajectory interruption; quantifies the target movement direction and instantaneous speed, identifies sharp turns, sudden acceleration and other maneuvering behaviors, and improves the robustness of dynamic target tracking; and integrates heat maps and velocity fields to realize joint modeling of the environment, outputs dynamic segmentation schemes through multi-objective reward functions, breaks through the resource rigidity caused by fixed partitions, and ultimately improves the autonomous decision-making ability of drones.

[0018] Furthermore, by deeply extracting the probability distribution features of the movement probability heat map and the target velocity vector field, this is spatially and temporally fused with the drone's position data to generate a joint state feature vector. This is then fed into the policy network of the region redivision reinforcement learning model, which outputs a set of segmentation parameters. A multi-objective reward function is used to perform a multi-dimensional evaluation of the segmentation scheme and drive network weight updates, ultimately binding the dynamic segmentation scheme to the real-time environmental state. A spatial coordinate mapping table is generated by establishing a mapping relationship between geometric vertices and geographic spatial coordinates. Priorities are converted into temporal logic by combining time window constraints and drone dynamic parameters. This fusion constructs a structured decision data stream and encodes it into an executable autonomous decision result. Through deep fusion of probability distribution features, the target's motion trend is converted into a quantitative decision factor, avoiding the lag in response to sudden trajectory changes associated with traditional rule-based segmentation. Simultaneously, a structured data stream is used to achieve real-time binding of the segmentation scheme to the drone's physical environment, overcoming the execution error accumulation and response lag caused by manual analysis.

[0019] These and other aspects of the present application will become more readily apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 A flowchart of a multi-objective optimization-based autonomous decision-making reinforcement learning method for drones provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of a multi-objective optimization-based drone autonomous decision-making reinforcement learning system provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0023] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 11, 12, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0024] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0025] In order to solve the problems in the existing technology that ultimately lead to poor autonomous decision-making ability of drones due to insufficient historical information fusion, target trajectory prediction deviation, lack of dynamic target velocity field modeling, incomplete multi-objective optimization dimensions, limited dynamic task allocation capabilities, and rigid regional resource allocation mechanisms, the present application embodiment provides a drone autonomous decision-making reinforcement learning method based on multi-objective optimization. The method adopts the following ideas: first, use airborne infrared and radar sensors to collect patrol area data in real time; generate a movement trajectory prediction sequence of suspicious targets through fusion processing, and construct a spatial distribution heat map reflecting the probability of target appearance based on this; at the same time, perform Doppler frequency shift analysis on the radar data to extract the target motion velocity vector field; then jointly input the heat map, velocity vector field and real-time position data of the drone into a pre-trained reinforcement learning model. The model designs a reward function based on the multi-objective optimization criteria of maximizing regional coverage, maximizing the probability of target tracking success and minimizing flight path energy consumption. Through interactive autonomous decision-making in a dynamic environment, it finally outputs a dynamic path planning result including an intelligent segmentation scheme for uninspected areas, realizing an autonomous closed loop from perception, prediction to decision-making.

[0026] Figure 1 A flowchart of a multi-objective optimization-based autonomous decision-making reinforcement learning method for drones provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes: S11. Collect infrared data and radar data within the patrolled area through the airborne sensor array deployed on the drone.

[0027] The "airborne sensor array" refers to a combination of multimodal detection equipment carried by a drone, including infrared and radar sensors, used to simultaneously collect information on ambient temperature distribution and target motion. The "inspected area" refers to the geographic area that the drone has completed scanning and detection, with its boundaries determined by historical flight paths. Infrared data refers to two-dimensional matrix data obtained by infrared sensors that reflects the surface temperature gradient of an object. Radar data refers to a three-dimensional point cloud dataset generated from radar echoes, containing target range, azimuth, and Doppler shift.

[0028] In an embodiment of the present application, the patrolled area is first scanned by an airborne sensor array deployed on a drone, and infrared data reflecting the temperature distribution of objects and radar data detecting the position and motion status of objects are simultaneously collected to provide original perception input for subsequent analysis.

[0029] S12. Generate a movement trajectory prediction sequence of the suspicious target in the target area based on the infrared data and radar data, and construct a movement probability heat map based on the movement trajectory prediction sequence. The target area includes the patrolled area.

[0030] Suspicious targets are defined as moving objects that meet pre-defined abnormal characteristics in infrared or radar data, such as sudden temperature changes or high-speed movements. A target area is a continuous geographic space encompassing both patrolled areas and unpatrolled areas awaiting expanded exploration. A motion trajectory prediction sequence is a time series of target future locations generated through linear interpolation or machine learning prediction based on historical trajectory points. A motion probability heat map is a two-dimensional distribution map that maps a geographic grid to a color gradient based on the probability of target occurrence. The probability values ​​are calculated by weighting the spatiotemporal density of trajectory points.

[0031] In the embodiment of the present application, the motion trajectory points of the suspicious target are first identified based on infrared data and radar data, and a spatiotemporal correlation algorithm is used to generate a predicted sequence of the target's motion trajectory within the target area. Secondly, the probability weights are calculated based on the spatiotemporal distribution density of the trajectory points in the sequence, combined with the spatial weight kernel function and time decay coefficient of the geographic grid unit. Finally, a motion probability heat map is generated that reflects the probability distribution of the target's appearance. It should be noted that the embodiment of the present application does not specifically limit the expression form of the motion trajectory points, the expression form of the spatial weight kernel function, or the size of the time decay coefficient.

[0032] S13. Perform Doppler frequency shift analysis on the radar data to generate a target velocity vector field.

[0033] Doppler shift analysis is the process of performing periodic folding corrections and radial velocity calculations on the radar echo frequency offset. The target velocity vector field is a group motion direction distribution field generated by aggregating the three-dimensional velocity vectors of discrete targets.

[0034] In an embodiment of the present application, a Doppler frequency shift analysis is first performed on each discrete echo unit in the radar data, and the original frequency shift amount is corrected by periodic folding to obtain the true Doppler frequency shift value; secondly, the frequency shift value is converted into a radial velocity component, and the three-dimensional velocity projection is calculated by combining the azimuth and elevation angles; finally, a target velocity vector field that reveals the movement direction of the target group is generated through motion consistency clustering.

[0035] S14. Taking the movement probability heat map and target velocity vector field as input features and combining them with the UAV's position data, a pre-trained area redivision reinforcement learning model is used based on a reward function for multi-objective optimization to output the UAV's autonomous decision-making reinforcement learning results, which include a dynamic segmentation scheme for uninspected areas. The reward function is used to maximize the area coverage, maximize the probability of successful target tracking, and minimize the path energy consumption.

[0036] Position data refers to the three-dimensional geographic coordinates and attitude angle data output by the drone's real-time positioning system. The region redivision reinforcement learning model refers to a deep reinforcement learning framework that takes environmental features as input and outputs dynamic partitioning parameters through a policy network. The reward function for multi-objective optimization is an evaluation function that simultaneously quantifies the incremental region coverage, target tracking matching, and path energy cost. The unpatrolled area refers to the set of undetected subregions within the target area that are not covered by sensor scans. The dynamic segmentation scheme refers to the set of geometric boundary partitioning parameters for the unpatrolled area generated based on the real-time environmental state. The drone's autonomous decision-making reinforcement learning results refer to the encoding of the dynamic segmentation scheme into an executable instruction set containing a sequence of track points and timestamps. Maximizing region coverage refers to the optimization objective of minimizing the detection blind spot area of ​​the unpatrolled area. Maximizing the probability of target tracking success refers to the optimization objective of giving the highest detection priority to high-probability target areas. Minimizing path energy consumption refers to the optimization objective of minimizing the total length of the drone's track and the steering energy cost.

[0037] The value R of the reward function can be calculated using the following formula: ,in, For regional coverage bonus items, Track rewards for your goals, is the path energy consumption penalty term, is a preset non-negative weighted value of the corresponding item. The embodiment of the present application does not specifically limit the size of each weighted value. For example, , , , and this embodiment does not specifically limit the specific expressions of each item.

[0038] In an embodiment of the present application, the probability distribution characteristics of the movement probability heat map and the probability distribution characteristics of the target velocity vector field are first temporally and spatially correlated with the drone position data to form a joint state feature vector; secondly, the vector is input into the policy network of the region redivision reinforcement learning model, and a set of segmentation parameters for the uninspected area is output; then, the multi-objective optimization reward function is used to evaluate the scalar values ​​of coverage, tracking success probability and energy consumption, driving the network weight update to generate a dynamic segmentation scheme; finally, the scheme is bound to the real-time geographic coordinates, and the drone autonomous decision-making reinforcement learning results are generated through the spatial coordinate mapping table and temporal logic coding.

[0039] The following is a specific example: First, a drone uses its onboard infrared and radar sensors to scan a patrolled forest area, collecting raw data on anomalous hot spots and moving targets. Next, a predicted sequence of movement trajectories of stowaways is generated based on the hot spot coordinates and radar track points. This sequence is then used to construct a high-probability heat map of the eastern side of the forest area. Doppler shift analysis of the radar data reveals that the target group is moving toward an unpatrolled valley, generating a northwest-oriented velocity vector field. The heat map, vector field, and real-time drone position are then input into a reinforcement learning model, which outputs a segmentation scheme for the unpatrolled valley. Finally, the proposed boundary is mapped to geographic coordinates, and combined with the drone's steering angle constraints, a trajectory command is generated to first block the valley entrance and then cover the slope, completing the coordinated task of dynamic tracking and area coverage.

[0040] Specifically, during a drone patrol mission in a border forest area, the drone's onboard infrared and radar sensors were activated simultaneously. The infrared sensor collected temperature distribution data in the patrolled area at a rate of 30 frames per second, generating a two-dimensional temperature matrix with a resolution of 512×512. Three areas were identified where temperatures were significantly above the ambient average by approximately 25°C, reaching a maximum of 36°C. Simultaneously, the radar sensor scanned at a 120° horizontal and 5° vertical angle, generating a 3D point cloud containing 5,000 points. These high-temperature areas corresponded to objects located approximately 800-1200 meters from the drone, with azimuths between 30° and 45°. This infrared and radar data were stored in real time in the drone's onboard storage module and served as raw input for subsequent analysis. Based on the data collected by the S11, the system first identified three suspicious targets due to their abnormal temperatures and associated movement. By analyzing the movement trajectories of the infrared hot spots and the positional changes of the radar echoes, the historical trajectory points of each target were obtained. For example, the trajectory points of target A over the past 5 minutes are (X1, Y1, T1), (X2, Y2, T2), … (X5, Y5, T5). These trajectory points are extended using linear interpolation to predict the target's movement sequence over the next 10 minutes within the target area, encompassing the patrolled area and the unpatrolled valley to the west. For example, target A might pass through (X6, Y6, T6) … (X15, Y15, T15). The target area is then divided into 10m x 10m geographic grids, and the spatiotemporal density of trajectory points within each grid is calculated. Spatially, a Gaussian kernel function is used to weight distance to the grid center, with closer distances receiving higher weights. Temporally, a time decay coefficient of 0.9 is set, giving greater weight to recent trajectory points. Finally, this aggregation generates a movement probability heatmap, where the grid at the valley entrance has a probability value of 0.8, appearing dark red, while areas farther from the trajectory have a probability value of less than 0.2, appearing light blue. Radar data acquired by the S11 is processed and decomposed into 1000 discrete echo units. Each unit contains an azimuth angle of 30°-45°, an elevation angle of -2°-1°, and a raw frequency shift. Since the maximum unambiguous radar frequency shift corresponds to a velocity of 10 m / s, some raw frequency shifts are found to exceed this range. This is corrected by period folding, for example, correcting the frequency shift corresponding to +15 m / s to -5 m / s. The 20 m / s period is then subtracted to obtain the true Doppler shift. The radial velocity component of each unit is calculated using the formula "radial velocity = (frequency shift value × wavelength) / (2 × transmit frequency)." Combining the azimuth and elevation angles, a three-dimensional velocity projection is then decomposed. For example, target A has an eastward velocity of -1.2 m / s and a northward velocity of -0.8 m / s, indicating northwestward movement. Through motion consistency clustering, velocity vectors with direction deviations less than 10° are classified into one category, and the target velocity vector field is finally generated, showing that 85% of the target vectors point to the northwest with an average speed of about 1.5 m / s.The drone's current location is 116.5° east longitude, 40.2° north latitude, and 500 meters above sea level. The movement probability heat map generated by S12 and the target velocity vector field generated by S13 are fed into a pre-trained region repartitioning reinforcement learning model along with this location data. The model's reward function is set with a weight of 0.3 for region coverage, 0.4 for target tracking success probability, and 0.3 for path energy consumption. After policy network analysis, a dynamic segmentation solution is output, dividing the unpatrolled area into three sub-areas: priority 1 for the valley entrance, priority 2 for the western slope, and priority 3 for the northern woodland. The coordinates of the boundary control points for each area are determined. Constrained by the drone's maximum steering angle of 30° and cruising speed of 10 m / s, an autonomous decision is generated, resulting in a track point sequence with the entry coordinates at 10:00, the slope coordinates at 10:05, and the woodland coordinates at 10:15. This command is encoded and sent to the flight control system, ensuring that high-probability areas are prioritized for blocking while minimizing flight energy consumption.

[0041] By executing S11 to S14, the embodiment of the present application solves the problems of delayed dynamic environmental response and multi-target optimization fragmentation in traditional inspections through deep fusion of multi-source perception data and dynamic decision-making through reinforcement learning, realizes adaptive segmentation of uninspected areas and precise scheduling of drone resources, and simultaneously improves target tracking reliability, regional coverage completeness and energy utilization efficiency in complex scenarios.

[0042] In one possible embodiment, S14 uses the movement probability heat map and the target velocity vector field as input features, and combines them with the drone's position data. A pre-trained region redivision reinforcement learning model is used based on a reward function for multi-objective optimization to output a drone autonomous decision-making reinforcement learning result including a dynamic segmentation scheme for uninspected areas, including: Step 141: extract the probability distribution features of the movement probability heat map and the probability distribution features of the target velocity vector field, and perform spatiotemporal correlation fusion of the probability distribution features and the position data of the UAV to generate a joint state feature vector.

[0043] Among them, probability distribution features refer to the probability density distribution characteristics of grid cells in the motion probability heat map, including probability value variance, gradient change rate, and the concentration of vector directions and velocity modulus distribution range in the target velocity vector field. Spatiotemporal correlation fusion refers to the process of aligning the target probability distribution features with the drone position data by timestamp and geographic coordinates, and combining the drone observation angle weight with the target motion trend weight through a weighted fusion function. The joint state feature vector is a multidimensional tensor that fuses the environment probability features and the drone posture data. Its dimensions include the heat map probability value, the vector field direction angle, the drone's latitude and longitude, and the pitch angle.

[0044] In an embodiment of the present application, the spatial probability distribution characteristics of the motion probability heat map are first extracted, including the probability density peak and the gradient change trend; at the same time, the motion direction probability distribution characteristics of the target velocity vector field are extracted, including the vector direction consistency and the velocity modulus distribution; secondly, the two types of probability distribution characteristics are spatially and temporally aligned with the real-time position data of the drone, and the drone observation perspective and the target motion trend are fused through the spatiotemporal attention mechanism; finally, a joint state feature vector is generated that characterizes the coupling relationship between the environmental dynamics and the drone state.

[0045] Step 142: Input the joint state feature vector into the region redivision reinforcement learning model, and output the segmentation parameter set of the uninspected area through the policy network.

[0046] The policy network, a deep neural network-based decision-making component within the region repartitioning reinforcement learning model, takes as input a joint state vector and outputs a weight matrix of segmentation parameters. The segmentation parameter set refers to the set of unpatrolled region partitioning parameters generated by the policy network, including the three-dimensional coordinates of boundary control points, sub-region priority coefficients, and boundary smoothness coefficients. The policy network, as the core decision-making component within the region repartitioning reinforcement learning model, directly receives as input a joint state feature vector generated by fusing movement probability heat maps, target velocity vector fields, and drone position data. Using the computational rules within the neural network, it analyzes the current environmental state and mission objectives and outputs a dynamic segmentation parameter set for the unpatrolled region in real time. This set includes key parameters such as boundary control point coordinates and segmentation priority weights. The region repartitioning reinforcement learning model drives segmentation solution generation through the policy network's action outputs. It also uses a reward function to perform multi-objective optimization evaluation of the segmentation parameters output by the policy network. By continuously iteratively updating the policy network's parameter weights, the model's final segmentation solution is designed to synergistically maximize region coverage, track success probability, and minimize path energy consumption.

[0047] In an embodiment of the present application, the joint state feature vector is first input into the policy network of the region redivision reinforcement learning model, which is composed of multiple layers of fully connected neurons; secondly, the forward propagation calculation outputs the segmentation parameter set of the uninspected area, including the coordinates of the region boundary control points, the division priority weights and the boundary density parameters.

[0048] Step 143: Based on the segmentation parameter set, a multi-dimensional evaluation is performed using a reward function for multi-objective optimization to generate a multi-dimensional evaluation result.

[0049] Among them, the multidimensional evaluation result refers to the weighted sum of three types of scalar values ​​calculated through the multi-objective reward function, including area coverage capability evaluation, target tracking effect evaluation and path energy efficiency evaluation.

[0050] In an embodiment of the present application, first, based on the area boundary control points in the segmentation parameter set, the first term of the reward function is used to calculate the coverage scalar of the uninspected area; secondly, the target tracking matching scalar is calculated according to the division priority weight combined with the high probability distribution of the heat map and the direction consistency of the vector field; then, based on the spatial density of the boundary control points and the current position of the drone, the path energy consumption scalar is calculated; finally, the three types of scalars are linearly superimposed to generate a multi-dimensional evaluation result.

[0051] Step 144: Based on the multi-dimensional evaluation results, the policy network of the area redivision reinforcement learning model is driven to update the weight parameters and output a dynamic segmentation scheme for the uninspected area.

[0052] The weight parameters refer to the weights of the neuron connections in the policy network, which are iteratively updated using a gradient descent algorithm to optimize decision-making. The real-time environmental state refers to the set of dynamic constraints imposed on the drone during mission execution, including geospatial coordinates, remaining flight time, maximum steering angle, and speed threshold.

[0053] In an embodiment of the present application, the gradient direction of the policy network is first calculated based on the multi-dimensional evaluation results; secondly, the network weight parameters are updated through the back propagation algorithm; and then the dynamic segmentation scheme of the uninspected area is regenerated based on the updated policy network, which includes optimized boundary control points and priority parameters.

[0054] Step 145: Bind the dynamic segmentation scheme to the real-time environmental status to generate a UAV autonomous decision-making reinforcement learning result including the dynamic segmentation scheme of the uninspected area.

[0055] In an embodiment of the present application, first, the geometric vertex coordinates are extracted based on the boundary information of the dynamic segmentation scheme; secondly, a mapping relationship is established with the geographic spatial coordinates in the real-time environment to generate a spatial coordinate mapping table; then, combined with the time window constraints and the maximum steering angle and speed parameters of the drone, the scheme priority is converted into a timestamp-marked execution timing logic; finally, the coordinate mapping table and the timing logic are integrated to encode and generate autonomous decision-making reinforcement learning results that can be executed by the drone.

[0056] The following is a specific example: The drone first analyzes the distribution patterns of human clusters in the forest heat map, including areas with the highest hotspot density and the fastest temperature changes. It also analyzes the movement patterns of groups of people in the radar velocity field, such as the majority moving northwest at similar speeds. These distribution patterns are then combined with the drone's current coordinates and flight altitude to generate a data packet that comprehensively describes the environment and the drone's status. This data packet is then input into the intelligent segmentation model. The model, through its built-in decision network, outputs the parameters for the segmentation plan for unpatrolled areas, including the coordinates of valley boundary points requiring priority blockade and the priority numbers for each area. Based on these parameters, the system evaluates the effectiveness of the plan: it calculates the coverage of the blind spot area, calculates a tracking and matching score based on the hotspot distribution and human movement directions, and estimates the flight energy consumption score based on the density of boundary points and the drone's current position. After the evaluation, the system automatically adjusts the decision network parameters based on these three scores to generate a more optimal segmentation plan, for example, assigning the valley entrance as the highest priority blockade area and the slope as a secondary coverage area. Finally, the boundary point coordinates of the new plan were converted to actual geographic coordinates. Combined with the drone's maximum turning angle, specific instructions were generated: the drone was required to reach the valley entrance coordinates before 10:00 AM to implement the blockade, and to complete the coverage inspection of the slope coordinates before 10:30 AM, forming a directly executable flight mission. The coverage score was calculated by subtracting the actual blind spot area from the intended coverage area; the tracking and matching score was calculated by multiplying the density of people in the hotspot by the consistency ratio of their movement directions; and the energy consumption score was determined by multiplying the straight-line distance from the drone to the boundary point by the path curvature coefficient.

[0057] In another specific example, in step 141, features of the movement probability heat map are extracted, including a probability peak at the valley entrance and a gradient rate of change of 0.3 / meter. Features of the velocity vector field are also extracted, including directional consistency of 85% and a velocity modulus of 1-2 m / s. These features are then aligned with the drone's location at 116.5° East longitude, 40.2° North latitude, and a pitch angle of 5°, at timestamp T=10:00, and with geographic coordinates. Using a spatiotemporal attention mechanism, features within the drone's observation range are given a higher weight of 0.7, generating a 128-dimensional joint state feature vector containing information such as heat values, azimuth angles, and longitude and latitude. In step 142, this joint state feature vector is input into the policy network, which consists of three fully connected layers. Through forward propagation, the network outputs a set of segmentation parameters, including the coordinates of five valley entrance boundary control points, a priority coefficient of 1.0, and a boundary smoothness coefficient of 0.8, as well as four slope control points with a priority coefficient of 0.7. Step 143 calculates the multi-dimensional evaluation results based on the parameters: coverage scalar 0.8, blind spot area ratio 20%, matching scalar 0.9, high-probability area matching the target's movement direction, energy consumption scalar 0.3, short flight distance with few turns, and a weighted score of 0.8 × 0.3 + 0.9 × 0.4 + 0.3 × 0.3 = 0.75. Step 144 adjusts the policy network weights based on the evaluation results through backpropagation, for example, adjusting the weight of the heat map feature layer from 0.2 to 0.3. After the update, a more optimized segmentation solution is output, which densifies the valley entrance control points and improves boundary accuracy. Step 145 converts the boundary points of the new solution into actual geographic coordinates, such as the entrance center point at 116.51° East longitude and 40.19° North latitude. Based on the time window of 10:00-10:20 and the drone's performance, an execution sequence is generated: arrival at the entrance at 10:00 and slope coverage starting at 10:05. This is encoded into binary instructions containing coordinates and timestamps, which serve as the final decision result.

[0058] By executing steps 141 to 145, the embodiment of the present application solves the problems of environmental response lag and execution disconnection in dynamic segmentation through deep feature fusion and reinforcement learning closed-loop optimization, realizes adaptive binding of uninspected area division and drone physical constraints, and improves the real-time performance of target tracking and the energy efficiency balance of coverage paths in complex scenarios.

[0059] In one possible embodiment, step 145, binding the dynamic segmentation scheme with the real-time environmental state to generate a UAV autonomous decision-making reinforcement learning result including the dynamic segmentation scheme of the uninspected area, includes: Step a1: extract the geometric vertex coordinates of the uninspected area and the geographic space coordinates in the real-time environment state according to the boundary information of the dynamic segmentation scheme, establish a mapping relationship between the geometric vertex coordinates and the geographic space coordinates, and generate a spatial coordinate mapping table.

[0060] Among them, boundary information refers to the geometric outline description data of the unpatrolled area defined in the dynamic segmentation scheme, including the vertex connection relationship and topological structure of the polygon boundary. Geometric vertex coordinates refer to the three-dimensional position data of the boundary points of the unpatrolled area described in the local coordinate system, usually stored in the form of Cartesian coordinates. Geographic spatial coordinates refer to the global coordinate system position data based on the earth ellipsoid model, including longitude, latitude and altitude values. The mapping relationship refers to the conversion correspondence rules between geometric vertex coordinates and geographic spatial coordinates, and the coordinate system alignment is achieved through the affine transformation matrix. The spatial coordinate mapping table refers to a two-dimensional relationship table that records the conversion results of all boundary points from local coordinates to geographic coordinates, which contains the original vertex number and the converted coordinate value.

[0061] In an embodiment of the present application, first, the geometric vertex coordinates of the uninspected area are extracted based on the boundary information of the dynamic segmentation scheme, and the coordinates exist in the form of a polygon vertex sequence; secondly, the geographic space coordinates in the real-time environmental state are obtained, and the coordinates are derived from the high-precision three-dimensional coordinate system of the geographic information system; then, a one-to-one mapping relationship between the geometric vertex coordinates and the geographic space coordinates is established, and the conversion from the local coordinate system to the global coordinate system is realized through the coordinate transformation matrix; finally, a spatial coordinate mapping table containing all vertex mapping relationships is generated.

[0062] Step a2: According to the time window constraints in the real-time environment state and the motion dynamics parameters of the UAV, the priority information of the dynamic segmentation scheme is converted into the timestamp mark of the path execution sequence to generate the execution timing logic.

[0063] Among them, the time window constraint refers to the set of time restriction conditions for task execution, including the effective target tracking time limit, the maximum endurance time of the drone, and the time threshold of key actions. The motion dynamics parameters refer to the quantitative indicators of the drone's physical motion capabilities, covering the maximum pitch angle, roll angle limit, cruising speed range, and the upper limit of sharp turn acceleration. Priority information refers to the importance classification data of different sub-areas in the dynamic segmentation scheme, and the high and low values ​​reflect the urgency of task execution. The path execution sequence refers to an ordered set of track points sorted by task priority, which defines the order of the drone's spatial movement. The timestamp tag refers to the absolute time identifier assigned to each track point in the path execution sequence, with execution time accurate to milliseconds. The execution timing logic refers to the set of spatiotemporal constraint rules that integrate the path point sequence and the timestamp, which stipulates the arrival time and residence time of the track points.

[0064] In an embodiment of the present application, the time window constraints in the real-time environmental state are first read, including the target prediction disappearance time limit and the maximum mission duration; secondly, the motion dynamics parameters of the UAV are obtained, covering the maximum steering angle limit, the cruising speed threshold and the acceleration upper limit; then, the priority information of different sub-areas in the dynamic segmentation scheme is converted into the execution order of the path point sequence according to the time window and dynamic constraints; finally, an accurate timestamp mark is assigned to each path point to generate an execution timing logic that can be parsed by the flight control system.

[0065] Step a3: Integrate the spatial coordinate mapping table and the execution timing logic to build a structured decision data flow.

[0066] Among them, structured decision data stream refers to a set of machine instructions organized according to specific data specifications, which contains a three-dimensional nested data structure of spatial coordinates, timestamps, and priorities.

[0067] In an embodiment of the present application, the sequence of geographic coordinate points in the spatial coordinate mapping table is first aligned with the timestamp tags in the execution timing logic in path order; secondly, each coordinate point is bound to the corresponding timestamp and priority label through the data structure nesting fusion method; finally, a structured decision data flow containing spatial position, execution timing and task weight is constructed to form a machine-readable instruction set.

[0068] Step a4: Encode the structured decision data stream to generate the UAV autonomous decision reinforcement learning results.

[0069] In an embodiment of the present application, a binary coding protocol is first used to serialize the structured decision data stream, and the coordinate points, timestamps, and priority tags are converted into a low-latency transmission format; secondly, track check codes and instruction check bits are added to ensure data integrity; finally, an autonomous decision reinforcement learning result is generated that can be directly input into the drone flight control system, and the result contains an encrypted track point sequence and execution timing instructions.

[0070] The following is a specific example: the drone uses its own sensors to scan the inspected forest area, detects abnormal hot spots and moving targets, and collects raw data; then, based on the location of the hot spots and radar track points, it generates a prediction sequence of the movement trajectory of suspicious persons, and uses this to create a high-probability hot spot distribution map on the east side of the forest; through Doppler frequency shift analysis of the radar data, it is found that the target group is moving towards the uninspected valley, and a movement speed pattern pointing to the northwest is generated; then these heat maps, speed patterns and the current position information of the drone are input into the reinforcement learning model, and a dynamic segmentation scheme for the uninspected valley is output; then, based on the boundary information provided by the scheme, the geometric vertex position coordinates of the uninspected area are extracted, and these coordinates are compared with the real-time environmental conditions. The system matches the actual geographic location coordinates in the state, establishes a position mapping relationship, and generates a spatial coordinate mapping table. According to the inspection time window restrictions set in the real-time environmental state, such as the inspection time period requirements, and the UAV's motion capability parameters such as maximum speed and minimum turning angle, the priority of the dynamic segmentation scheme, such as prioritizing the valley entrance, is converted into a time point marking sequence of the path execution sequence to generate the execution timing logic. The spatial coordinate mapping table and the execution timing logic are then merged and integrated to construct a structured decision data stream. Finally, the data stream is encoded and processed to generate the UAV's autonomous decision-making reinforcement learning results, which are then applied to command generation. For example, the flight path control of first blocking the valley entrance and then covering the slope is realized, realizing the coordinated completion of dynamic tracking targets and area coverage tasks.

[0071] In another specific example, in step a1, the boundary control points of the dynamic segmentation scheme are 100, 200, and 500 in the local coordinate system. These points are mapped to the real-time geographic coordinates of 116.5° East longitude corresponding to x=1000 and 40.2° North latitude corresponding to y=2000. Using an affine transformation matrix, the geographic coordinates are 116.505° East longitude, 40.198° North latitude, and an altitude of 500 meters. A spatial coordinate mapping table is generated, recording the transformation results of all control points. In step a2, the time window constraint is that the target may leave the monitoring range before 10:20, and the drone has a flight time of 2 hours. Motion dynamics parameters include a maximum steering angle of 30° and a cruising speed of 8-12 m / s. The priority sub-areas, Entrance 1, Slope 2, and Woodland 3, are converted into a path sequence. Timestamps are calculated based on speed. The entrance is 800 meters from the current location and requires 80 seconds. Adding 0:01:20 to 10:00:00 equals 10:01:20. The slope is 500 meters from the entrance and requires 50 seconds, resulting in 10:02:10. This generates execution sequential logic. In step a3, the entry coordinates 116.505°, 40.198° from the mapping table are bound to the timestamp 10:01:20, giving priority 1. The slope coordinates are bound to 10:02:10, giving priority 2. This constructs a structured decision data stream with the format [coordinate, timestamp, priority]. In step a4, the data stream is binary-encoded, with each coordinate represented as a 32-bit floating-point number, the timestamp as a 64-bit integer with millisecond accuracy, and the priority as an 8-bit integer. After adding a checksum, a 2KB command is generated that can be parsed by the drone and sent to the flight control system via the onboard bus.

[0072] By executing steps a1 to a4, the embodiment of the present application solves the problem of disconnection between trajectory planning and real-time status in traditional methods by establishing a precise binding mechanism between the decision-making plan and the physical environment, realizes lossless conversion of dynamic segmentation plan to flight control instructions, and improves the spatiotemporal accuracy and system reliability of task execution in complex environments.

[0073] In one possible embodiment, step 143, based on the segmentation parameter set, performing a multi-dimensional evaluation using a reward function for multi-objective optimization to generate a multi-dimensional evaluation result, includes: Step b1: Based on the area boundary control points in the segmentation parameter set, use the first term in the reward function to calculate the spatial coverage blind area distribution of the uninspected area and generate a coverage scalar corresponding to the first target, which is the area coverage rate.

[0074] The regional boundary control points refer to the set of key coordinate points in the dynamic segmentation scheme that define the geometric outline of the unpatrolled area, including polygon vertices and curvature control points. The coverage scalar is a quantitative indicator of the completeness of detection in the unpatrolled area, calculated by calculating the normalized difference between the theoretical coverage area and the actual blind spot area. The spatial coverage blind spot distribution refers to the geographic locations and their extent within the unpatrolled area that are not covered by the polygonal area to be patrolled, formed by connecting the regional boundary control points in the segmentation scheme. Specifically, when the coordinate sequence of the boundary control points is connected in sequence to form a closed polygon, the polygon represents the effective coverage area to be patrolled by the drone. All contiguous sub-areas of the unpatrolled area that lie outside the polygon constitute the spatial coverage blind spots. The distribution of these blind spots is directly determined by the spatial positional relationship of the boundary control points. A larger blind spot indicates a wider uncovered geographic area. The blind spot distribution is quantified by subtracting the area of ​​the polygon enclosed by the boundary control points from the total area of ​​the unpatrolled area. This distribution is used in the first calculation of the reward function to generate the core indicator, the coverage scalar, that reflects the level of regional coverage.

[0075] In an embodiment of the present application, first, the minimum circumscribed polygon area of ​​the uninspected area is calculated based on the coordinates of the area boundary control points in the segmentation parameter set; secondly, the polygon area is divided by the total area of ​​the target area to obtain the theoretical coverage rate; then, the difference between the theoretical coverage rate and the actual sensor coverage range is calculated to generate the spatial coverage blind spot distribution; finally, the blind spot area is normalized by taking a negative value through the first term of the reward function to generate a first target coverage scalar reflecting the area coverage rate.

[0076] Step b2: Based on the segmentation priority weights in the segmentation parameter set, combined with the high-probability target distribution in the movement probability heat map and the motion direction consistency of the target velocity vector field, the second term in the reward function is used to generate a matching degree scalar corresponding to the second target. The second target is the probability of successful tracking of the suspicious target.

[0077] The priority weights are numerically assigned to different sub-regions within the dynamic segmentation scheme, and their values ​​determine the order in which tasks are executed. The matching scalar is a composite evaluation that combines the probability of the target's spatial distribution and the consistency of its motion direction, reflecting the degree of fit between the tracking strategy and the target's behavior pattern.

[0078] In an embodiment of the present application, first, the division priority weight in the segmentation parameter set is obtained; secondly, the spatial coordinate density of the high probability target distribution and the motion direction consistency coefficient of the target velocity vector field in the movement probability heat map are extracted; then, through the second term of the reward function, the priority weight is multiplied by the high probability density value and then by the direction consistency coefficient to generate a second target matching scalar reflecting the probability of successful tracking of the suspicious target.

[0079] Step b3: Based on the spatial density of the boundary control points in the segmentation parameter set and the current position of the UAV, use the third term in the reward function to generate an energy consumption scalar corresponding to the third goal, which is the path energy consumption.

[0080] Among them, the energy consumption scalar refers to the estimated energy consumption calculated based on the path length, turning angle and flight speed, which includes the distance factor and maneuver complexity factor.

[0081] In an embodiment of the present application, the spatial density of the boundary control points in the segmentation parameter set, that is, the number of control points per unit area, is first calculated; secondly, the Euclidean distance from the current position of the drone to the nearest control point is obtained; then, the spatial density is multiplied by the distance value and then by the path curvature coefficient through the third term of the reward function to generate a third target energy consumption scalar reflecting the path energy consumption.

[0082] Step b4: linearly superimpose the coverage scalar, matching scalar and energy consumption scalar to generate a multi-dimensional evaluation result.

[0083] Among them, linear superposition refers to the calculation process of weighting and summing multiple scalars according to preset weight coefficients, and the weights reflect the relative importance of different optimization objectives.

[0084] In an embodiment of the present application, the coverage scalar is first multiplied by the first weight coefficient to obtain a weighted coverage value; secondly, the matching scalar is multiplied by the second weight coefficient to obtain a weighted matching value; then, the energy consumption scalar is multiplied by the third weight coefficient to obtain a weighted energy consumption value; finally, the three types of weighted values ​​are added together to generate a multidimensional evaluation result.

[0085] Here's a specific example: A drone first uses infrared and radar equipment to scan a forest area, detecting unusual heat sources and moving objects. It then records the locations of these heat sources and their radar track points, predicting the movement paths of suspicious individuals and constructing a high-probability target distribution map on the eastern side of the forest. It then analyzes frequency variations in the radar data and identifies the target group moving toward a northwest valley. A northwest velocity vector diagram is generated, representing the target's movement. Simultaneously, the drone acquires its own location information in real time. This data is fed into an intelligent model, which then outputs a segmentation scheme for the unpatrolled valley, including parameters such as control points for the area boundaries and priority weights for the segments. Based on this segmentation scheme, the model begins to calculate multidimensional evaluation results. In the first step, boundary control points are used to analyze the distribution of spatial coverage blind spots in the uninspected area. The first term of the reward function is used to calculate a coverage value, which reflects the coverage rate of the area. The coverage is calculated based on the ratio of the blind spot area to the total area. In the second step, the second term of the reward function is used to calculate a matching value, which represents the probability of successfully tracking a suspicious target, by combining the priority weights, the high-probability target distribution map, and the consistency of the motion direction of the velocity vector map. The calculation process is generated by integrating the priority weights, the target distribution probability, and the motion direction matching. In the third step, based on the spatial density of the boundary control points and the current position of the drone, the third term of the reward function is used to calculate an energy consumption value. This value represents the energy consumption of the drone's flight path. The calculation basis is the product of the control point density and the distance from the drone to the boundary point to estimate the energy consumption. Finally, the coverage value, matching value, and energy consumption value are linearly added to generate a multidimensional evaluation result. This evaluation result is used to optimize the segmentation plan. The plan boundaries are converted into geographic coordinates and combined with the drone's steering angle limit to generate flight instructions: first block the valley entrance to prevent the target from escaping, then cover the slope to ensure regional inspections, realizing the collaborative tasks of dynamic tracking and regional coverage.

[0086] In a specific example, in step b1, the polygon enclosed by the region boundary control points in the segmentation parameters is 5000 square meters, and the total area of ​​the unpatrolled area is 6000 square meters. The theoretical coverage ratio is approximately 5000 / 6000, which is approximately 0.83. Actual sensor coverage is lost by 5% due to terrain, resulting in a coverage scalar of 0.83 × (1 - 0.05) = 0.79. The first term of the reward function, covering the scalar by multiplying it by 0.3, yields 0.237. In step b2, the partitioning priority weights are 1.0 for entrances and 0.7 for slopes. The high-probability entrance area in the movement probability heat map accounts for 60%, and the velocity vector field has 85% directional consistency. The matching scalar is 1.0 × 0.6 × 0.85 + 0.7 × 0.3 × 0.7 ≈ 0.51 + 0.147 = 0.657. The second term, covering the scalar by multiplying it by 0.4, yields 0.263. In step b3, the spatial density of boundary control points is 5 per 1000 square meters, the distance from the drone to the entrance is 800 meters, the path curvature coefficient is 1.2, and because two turns are required, the energy consumption scalar is 5 × 800 × 1.2 = 4800. After normalization, it is 0.2. The lower the value, the better. The third term is multiplied by 0.3 and negated to -0.06. In step b4, the multidimensional evaluation result is 0.237 + 0.263 - 0.06 = 0.44, which serves as the basis for model update.

[0087] By executing steps b1 to b4, the embodiment of the present application solves the problem of balancing coverage, tracking, and energy consumption targets in traditional segmentation schemes through a multi-dimensional quantitative evaluation mechanism, realizes the interpretability of the reinforcement learning decision-making process and the multi-objective collaborative optimization capability, and improves the task adaptability and resource allocation rationality in a dynamic environment.

[0088] In a possible embodiment, S12, constructing a movement probability heat map based on the movement trajectory prediction sequence, includes: Step 121 : Deconstruct the target area of ​​the mobile trajectory prediction sequence into multiple discrete trajectory points including spatial coordinates and observation times according to geographic grid units to form a discrete trajectory point set.

[0089] A geographic grid cell is a square geographic block that divides the target area into fixed-size blocks. Each cell contains information about the longitude span, latitude span, and altitude range. A discrete trajectory point set is a collection of spatiotemporally labeled points formed by decomposing the trajectory prediction sequence. Each point contains latitude and longitude coordinates and the observation time accurate to the second.

[0090] In the embodiment of the present application, the target area is first divided into fixed-size geographic grid cells, each of which has a unique geographic coordinate range; secondly, each trajectory point in the mobile trajectory prediction sequence is assigned to the corresponding geographic grid cell according to its spatial coordinates, and the observation time of the point is recorded; finally, a discrete trajectory point set containing the spatial coordinates and observation times of all trajectory points is formed.

[0091] Step 122: Calculate the spatial distance weight based on the preset spatial weight kernel function and the spatial coordinates of the discrete trajectory points, and calculate the time decay weight based on the preset time decay coefficient and the observation time of the discrete trajectory points.

[0092] The spatial weight kernel function is a weight calculation rule designed based on the law of distance decay, such as the Gaussian function, where the closer to the grid center, the higher the weight. The temporal decay coefficient is a parameter that controls the rate at which the importance of historical trajectory points decays; a larger value indicates a higher weight for recent data.

[0093] In an embodiment of the present application, a preset spatial weight kernel function is first used to calculate the distance between each point in the discrete trajectory point set and the center of the grid unit in which it is located. The larger the distance, the smaller the weight. The spatial distance weight is obtained by dividing the distance by the preset attenuation radius. Secondly, based on the difference between the observation time of the discrete trajectory point and the current time, combined with the preset time attenuation coefficient, the larger the time difference, the smaller the weight. The time attenuation weight is obtained by multiplying the difference by the attenuation coefficient.

[0094] Step 123: Based on the spatial distance weight and the time decay weight, the spatial coordinates and the observation time are fused to generate the initial thermal value of the geographic grid cell.

[0095] Among them, the initial thermal value refers to the grid unit probability benchmark value calculated after integrating the spatial weight and time weight, reflecting the spatiotemporal density of the historical trajectory points in the area.

[0096] In an embodiment of the present application, first, for all discrete trajectory points in each geographic grid cell, their spatial distance weight and time attenuation weight are multiplied to obtain a comprehensive weight; secondly, the comprehensive weights of all trajectory points in the cell are added together; finally, the sum is divided by the number of trajectory points in the cell to generate the initial thermal value of the geographic grid cell.

[0097] Step 124: Convert the initial thermal values ​​of all geographic grid cells into a color gradient distribution to form a movement probability heat map.

[0098] Among them, color gradient distribution refers to a visualization scheme that maps thermal values ​​to color depth, usually using red to represent high-probability areas.

[0099] In an embodiment of the present application, the initial thermal values ​​of all geographic grid cells are first normalized to the range of 0 to 1; secondly, the normalized values ​​are mapped to a preset color gradient, with higher thermal values ​​corresponding to darker warning colors; finally, a moving probability heat map is formed that reflects the probability distribution of target occurrence.

[0100] The following is a specific example: a drone scans a forest area using infrared and radar equipment, collects raw data on abnormal heat sources and moving objects, and generates a predicted sequence of the movement trajectory of suspicious persons based on the heat source location and radar trajectory points; then, according to step 121, this sequence is divided according to geographic grid units, and the target area is decomposed into multiple discrete trajectory points, each of which contains spatial coordinates and observation time, forming a discrete trajectory point set; then, in step 122, based on a preset spatial weight function such as a Gaussian function, the spatial distance weight of each discrete trajectory point to the center of the geographic grid unit is calculated, and based on a preset time decay coefficient, the time decay weight of the time difference between the observation time of each point and a reference time such as the current time is calculated; then Then, in step 123, based on the spatial distance weight and time decay weight, the spatial coordinates and observation time are integrated to calculate the initial thermal value for each geographic grid cell. The calculation formula is that the initial thermal value is equal to the weighted sum of the spatial distance weight of all discrete trajectory points multiplied by the time decay weight; finally, in step 124, the initial thermal values ​​of all geographic grid cells are converted into a color gradient distribution to form a mobile probability heat map. The heat map is subsequently input into the reinforcement learning model, combined with the velocity vector field and the real-time position of the UAV, and the segmentation scheme of the unpatrolled valley is output. After the scheme boundary is converted into geographic coordinates, the flight command is generated in combination with the UAV steering angle limit, first blocking the valley entrance and then covering the slope, realizing the collaborative task of dynamic tracking and area coverage.

[0101] In another specific example, in step 121, the target area is a 2 km × 2 km forest area, divided into 200 × 200 10 × 10 meter geographic grid cells. The predicted trajectory sequence contains 100 trajectory points, each assigned to a corresponding grid cell. The observation times, such as 9:50:00 and 9:51:00, are recorded to form a discrete trajectory point set. In step 122, a Gaussian spatial weight kernel function is used, with a weight of 1.0 for points within 5 meters of the grid center and 0.1 for points beyond 10 meters. For a trajectory point 3 meters from the center, the spatial distance weight is 0.8. The time decay coefficient is 0.9, and the observation time of this point is 5 minutes away from the current time of 10:00. The time decay weight is 0.9^5≈0.59. In step 123, a grid contains five trajectory points, with weights of 0.8 × 0.59, 0.7 × 0.65, and so on, totaling 2.8. The initial thermal value is 2.8 / 5 = 0.56. In step 124, all grid thermal values ​​are normalized to a scale of 0–1, with values ​​above 0.8 colored dark red, 0.5–0.8 colored orange, 0.2–0.5 colored yellow, and values ​​below blue. This generates a movement probability heatmap, clearly showing the high-probability areas at the valley entrance.

[0102] By executing steps 121 to 124, the embodiment of the present application solves the problems of neglecting the timeliness of historical trajectories and breaking spatial continuity in traditional heat maps through a dual-weight fusion mechanism of time and space, achieves accurate quantitative expression of the probability of target occurrence, and improves the credibility of the prediction of suspicious target behavior in a dynamic environment.

[0103] In a possible embodiment, S13, performing Doppler shift analysis on radar data to generate a target velocity vector field, includes: Step 131: Deconstruct the radar data into a set of discrete echo units, each of which contains an azimuth angle, an elevation angle, and an original frequency shift.

[0104] A discrete echo unit set is the smallest processing unit of radar scan data, divided by azimuth and elevation. Each unit contains angle information and raw frequency shift data. Azimuth refers to the horizontal angle of the target relative to the drone's true north, with clockwise values ​​ranging from 0 to 360 degrees. Elevation refers to the vertical angle between the line connecting the target and the drone and the horizontal plane, with positive angles pointing upward. The raw frequency shift is the initial difference between the radar echo frequency and the transmit frequency, including velocity information caused by the Doppler effect and measurement noise.

[0105] In an embodiment of the present application, the original echo data received by the radar is first divided into independent units according to the scanning azimuth and pitch angle. Each unit contains the horizontal direction angle, vertical tilt angle and unprocessed frequency offset of the target relative to the drone, forming a discrete echo unit set.

[0106] Step 132: Perform period folding correction on the original frequency shift to generate a true Doppler frequency shift value.

[0107] Period folding correction is a technique used to eliminate ambiguity in radar velocity measurements. When the frequency shift exceeds the maximum unambiguous frequency shift, an integer multiple of the wavelength is added or subtracted to bring it within the measurable range. The true Doppler shift value is the corrected frequency offset that reflects the target's true radial velocity.

[0108] In an embodiment of the present application, it is first identified whether the original frequency shift exceeds the maximum measurable frequency shift corresponding to half the radar wavelength. If it exceeds the limit, the frequency shift period value corresponding to the frequency shift plus or minus an integer multiple of the wavelength is corrected to eliminate measurement ambiguity; secondly, the corrected true Doppler frequency shift value is output, which reflects the true relative speed between the target and the drone.

[0109] Step 133: Convert the true Doppler frequency shift value into a radial velocity component, and calculate the three-dimensional velocity projection by combining the azimuth angle and the elevation angle.

[0110] The radial velocity component refers to the target's velocity along the radar beam, calculated by multiplying the frequency shift by the wavelength and dividing it by twice the radar frequency. Three-dimensional velocity projection decomposes the radial velocity into three orthogonal velocity components: east and west, south and north, and up and down, based on azimuth and elevation.

[0111] In the embodiment of the present application, the radial velocity component is first calculated according to the formula of the product of the radar wavelength and the true Doppler frequency shift value divided by twice the radar frequency; secondly, the azimuth and elevation angles recorded by the unit are combined to calculate the velocity components of the target in the east-west, north-south and vertical directions through trigonometric function relationships; finally, a three-dimensional velocity projection is synthesized.

[0112] Step 134: Perform motion consistency aggregation on the three-dimensional velocity projection to obtain a target velocity vector field.

[0113] Among them, motion consistency aggregation refers to the process of clustering spatially adjacent velocity vectors with similar motion directions into group motion vectors, which includes direction consistency detection and vector weighted fusion.

[0114] In an embodiment of the present application, the three-dimensional velocity projections of adjacent units are firstly tested for directional similarity, and projections with a moving direction angle less than a preset threshold are classified as the same moving group; secondly, a weighted average of the velocity vectors of the same type of projections is taken, with the weight determined by the signal strength; finally, a target velocity vector field is generated that reflects the group movement trend.

[0115] The following is a specific example: After the drone collects raw data of a moving target by scanning a forest area with an airborne radar, the radar data is decomposed into multiple discrete echo units according to step 131, where each unit contains azimuth angle information, elevation angle information, and a raw frequency shift value; then, in step 132, the raw frequency shift is subjected to periodic folding correction. When the raw frequency shift exceeds the maximum frequency shift range measurable by the radar, the true Doppler frequency shift value is adjusted by the formula: the raw frequency shift value is equal to the raw frequency shift value minus twice the maximum unambiguous frequency shift multiplied by an integer adjustment coefficient k, where the integer adjustment coefficient k is determined based on echo continuity, thereby generating a true Doppler frequency shift value; then, in step 133, the true Doppler frequency shift value is converted into a radial velocity component, where the conversion formula is: the radial velocity component is equal to the true Doppler frequency shift value multiplied by the speed of light divided by twice the radar transmission frequency, and then, combined with the azimuth angle θ and the elevation angle φ, the target's velocity component v_e in the east-west direction is calculated to be equal to the radial velocity component multiplied by cosθ Multiplied by sinφ, the north-south velocity component v_n is equal to the radial velocity component multiplied by sinθ multiplied by sinφ, and the vertical velocity component v_u is equal to the radial velocity component multiplied by cosφ, forming a three-dimensional velocity projection. Finally, in step 134, the three-dimensional velocity projection is subjected to motion consistency aggregation. A spatial weighted average method is used to screen and weightedly fuse all velocity vectors within a preset radius around each location point based on directional similarity. The weight is determined by the square of the negative distance of the spatial distance Gaussian function exp divided by twice the square of the spatial scale parameter sigma. Finally, a target velocity vector field is generated, which shows that the target group as a whole is moving toward the unpatrolled valley in the northwest direction. This result is input into the reinforcement learning model together with the heat map and the drone position, and a segmentation scheme for the unpatrolled valley is output. The scheme boundary is converted into geographic coordinates and the drone's steering angle limit is considered. A track instruction is generated to first block the valley entrance and then cover the slope, realizing the collaborative task of dynamic tracking and area coverage.

[0116] In another specific example, in step 131, radar data is deconstructed into 1000 discrete echo units. Each unit records an azimuth angle of, for example, 35°, an elevation angle of -1°, and an original frequency shift of, for example, +1200 Hz. This corresponds to a velocity exceeding the radar's maximum measurable value of 10 m / s. In step 132, the radar wavelength is 0.1 meters. The maximum unambiguous frequency shift = (2 × transmit frequency × 10 m / s) / 0.1 = 200 × transmit frequency. The original frequency shift of +1200 Hz exceeds this value and is corrected to 1200 - 2 × 1000 = -800 Hz. Assuming a maximum frequency shift of 1000 Hz, the true Doppler shift value is -800 Hz. In step 133, the radial velocity = (-800 Hz × 0.1 m) / (2 × transmit frequency). If the transmit frequency is 10 GHz, the radial velocity is approximately -1.2 m / s. Combining an azimuth of 35° and an elevation of -1°, the 3D velocity projection is calculated: Easting = -1.2 × sin35° ≈ -0.68 m / s, Northing = -1.2 × cos35° ≈ -0.98 m / s, and vertical ≈ 0. In step 134, all 3D velocity projections are clustered, and those with easting deviations <0.2 m / s and northing deviations <0.2 m / s are grouped together, accounting for 80%. This category has an average velocity of -0.7 m / s in the east and -1.0 m / s in the north. This generates the target velocity vector field, with arrows pointing northwest and lengths indicating velocity magnitude.

[0117] By executing steps 131 to 134, the embodiment of the present application solves the problems of directional ambiguity and multi-target speed dispersion in traditional radar speed measurement through periodic folding correction and motion consistency aggregation, realizes accurate extraction of group motion vectors, and improves the reliability of target behavior pattern analysis in complex scenarios.

[0118] In a possible embodiment, S12, generating a movement trajectory prediction sequence of a suspicious target within a target area based on infrared data and radar data, includes: Step c1: Based on the temperature gradient distribution corresponding to the infrared data, extract the center coordinates of abnormal hot spots with temperature gradient distribution higher than the preset temperature difference threshold in the inspected area, sort all the center coordinates of abnormal hot spots according to the acquisition timestamp, and generate a target hot spot position sequence.

[0119] Among them, the temperature gradient distribution refers to the distribution diagram of the temperature change rate between adjacent pixels in the infrared image, which reflects the heat transfer trend of the object surface. The preset temperature difference threshold refers to the artificially set temperature mutation judgment value. When the temperature difference between adjacent areas exceeds this value, an abnormal alarm is triggered. The center coordinates of the abnormal hot spot refer to the position data of the geometric center point of the temperature mutation area, including longitude, latitude and altitude values. The acquisition timestamp refers to the precise time record when the sensor detects the hot spot, in the format of year, month, day, hour, minute, second and millisecond. The target hot spot position sequence refers to the set of abnormal hot spot center coordinates sorted by time, which constitutes a discrete position chain of target movement.

[0120] In an embodiment of the present application, the temperature change rate distribution of each position in the infrared data is first analyzed, and the coordinates of the center point of the abnormally high temperature area where the temperature change amplitude exceeds the preset warning value in the area scanned by the drone are found; secondly, all the center point coordinates are arranged in the chronological order of the detection of these high temperature points to generate a target hot spot position sequence with time marks.

[0121] Step c2: Within a preset time window, the target hot spot position sequence is fused with the spatial coordinates of the trajectory segment of the radar data to generate a fused trajectory point set.

[0122] Among them, the trajectory segment refers to the target motion path segment detected by the radar within a single scanning cycle. The fused trajectory point set refers to the unified coordinate data set formed by matching the infrared hot spot coordinates with the radar trajectory coordinates. The difference between the fused trajectory point set and the discrete trajectory point set is that the discrete trajectory point set is the primary product of a single sensor data, which is generated only by sorting the center coordinates of the infrared abnormal hot spot by timestamp. Each point only contains the spatial position and observation time, and is not interactively verified with other sensor data. The fused trajectory point set is the result of collaborative processing of multiple source sensors. It is generated by spatially matching and fusing the target hot spot position sequence generated by infrared with the trajectory segment detected by radar within a preset time window. Each point must meet the spatiotemporal consistency conditions at the same time. In essence, it is a cross-validation and enhancement of the discrete trajectory point set.

[0123] In an embodiment of the present application, a time interval of fixed length is first set, within which the coordinate points in the target hot spot position sequence are matched with the coordinate points of the moving target trajectory detected by the radar; secondly, the average value of the successfully matched coordinate points is taken, and the original data of the unmatched points is retained; finally, a fused trajectory point set containing both infrared and radar data is generated.

[0124] Step c3: Connect the fused trajectory point set in the order of timestamps to form a trajectory chain.

[0125] Among them, a trajectory chain refers to a sequence of fused trajectory points connected in time order, describing the continuous movement path of the target.

[0126] In an embodiment of the present application, first, all coordinate points in the fusion trajectory point set are sorted from early to late according to the detection time; secondly, adjacent coordinate points are connected with straight lines in chronological order; finally, a trajectory chain describing the continuous movement path of the target is formed.

[0127] Step c4: according to the consistency of the velocity vector direction, the trajectory is extended based on the trajectory chain, and a predicted sequence of the movement trajectory of the suspicious target in the target area is output.

[0128] Among them, trajectory extension refers to a prediction method that extends the trajectory chain to unexplored areas based on the inertia of historical movement direction.

[0129] In an embodiment of the present application, the direction change angles of adjacent line segments in the trajectory chain are first analyzed to screen continuous line segments with basically consistent movement directions; secondly, the trajectory chain is extended along this direction to the boundary of the target area; finally, a predicted movement trajectory sequence of the suspicious target in the future area is output.

[0130] The following is a specific example: In step c1, a temperature difference threshold of 5°C is set in the temperature gradient distribution of the infrared data. The center coordinates of three anomalous hot spots are extracted: 116.5°E, 40.2°N, 116.502°E, 40.199°N, and 116.503°E, 40.198°N. These are sorted by the acquisition timestamps of 9:50, 9:52, and 9:54 to generate a target hot spot location sequence. In step c2, a 1-minute time window is set. Between 9:50 and 9:51, if the spatial distance between hot spot A and the radar track point 116.501°E, 40.199°N is less than 5 meters, a successful match is achieved. The average coordinate is taken as the fusion point. Unmatched points are retained to generate a fused track point set consisting of A, B, C, and two unique radar points. In step c3, the fused trajectory points are connected by timestamps 9:50 → 9:51 → 9:52 → 9:54 to form a trajectory chain, showing a continuous path from southeast to northwest with an average movement speed of 1.2 m / s. In step c4, the end point of the trajectory chain, 9:54, shows a northwest direction with an azimuth of 315°. The direction deviation of the previous three points is less than 5°, indicating high consistency. Based on this direction and speed, trajectory points are predicted for the next 10 minutes: 9:56 to 116.504°E, 40.197°N, and 10:00 to the valley entrance at 116.505°E, 40.196°N, forming a predicted movement trajectory sequence.

[0131] By executing steps c1 to c4, the embodiment of the present application solves the problems of single sensor trajectory breakage and prediction blind spots through the spatiotemporal fusion and motion inertia extension of infrared and radar data, and realizes continuous reconstruction of target movement paths and behavior prediction in complex environments.

[0132] Figure 2A schematic diagram of the structure of a multi-objective optimization-based autonomous decision-making reinforcement learning system for drones provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the system includes: The acquisition module 21 is used to collect infrared data and radar data in the patrolled area through the airborne sensor array deployed on the UAV.

[0133] The construction module 22 is used to generate a movement trajectory prediction sequence of a suspicious target in a target area based on infrared data and radar data, and to construct a movement probability heat map based on the movement trajectory prediction sequence. The target area includes the inspected area.

[0134] The generating module 23 is used to perform Doppler frequency shift analysis on the radar data to generate a target velocity vector field.

[0135] The output module 24 is used to take the movement probability heat map and the target velocity vector field as input features, and combine them with the position data of the UAV, and output the UAV autonomous decision-making reinforcement learning results including the dynamic segmentation scheme of the uninspected area based on the reward function for multi-objective optimization through a pre-trained area redivision reinforcement learning model. The reward function is used to reflect the maximization of area coverage, the maximization of the probability of target tracking success, and the minimization of path energy consumption.

[0136] Figure 2 The UAV autonomous decision-making reinforcement learning system based on multi-objective optimization can be executed Figure 1 The implementation principles and technical effects of the multi-objective optimization-based reinforcement learning method for autonomous decision-making by drones described in the illustrated embodiment are not further elaborated. The specific manner in which each module and unit performs operations in the multi-objective optimization-based reinforcement learning system for autonomous decision-making by drones in the aforementioned embodiment has been described in detail in the relevant embodiments of the method and will not be further elaborated here.

[0137] In one possible design, Figure 2 The embodiment shown is a multi-objective optimization-based autonomous decision-making reinforcement learning system for drones that can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32 .

[0138] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0139] The processing component 32 is used to collect infrared data and radar data from the patrolled area through the airborne sensor array deployed on the drone. Based on the infrared and radar data, a prediction sequence of the movement trajectory of the suspicious target within the target area is generated, and a movement probability heat map is constructed based on the movement trajectory prediction sequence. The target area includes the patrolled area. Doppler shift analysis is performed on the radar data to generate a target velocity vector field. The movement probability heat map and target velocity vector field are used as input features. In combination with the drone's position data, a pre-trained region redivision reinforcement learning model is used to output the drone's autonomous decision-making reinforcement learning results, including a dynamic segmentation scheme for the unpatrolled area, based on a reward function for multi-objective optimization. The reward function is used to maximize regional coverage, maximize the probability of successful target tracking, and minimize path energy consumption.

[0140] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0141] The storage component 31 is configured to store various types of data to support operations on the terminal. The storage component can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as random access memory (RAM), static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0142] Of course, a computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0143] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0144] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0145] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0146] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The illustrated embodiment is a reinforcement learning method for autonomous decision-making of a UAV based on multi-objective optimization.

[0147] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0148] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0149] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A reinforcement learning method for autonomous decision-making of UAV based on multi-objective optimization, characterized by: include: Collect infrared and radar data from the patrolled area through the airborne sensor array deployed on the drone; generating a movement trajectory prediction sequence of a suspicious target within a target area based on the infrared data and the radar data, and constructing a movement probability heat map based on the movement trajectory prediction sequence, wherein the target area includes an inspected area; Performing Doppler frequency shift analysis on the radar data to generate a target velocity vector field; Taking the movement probability heat map and the target velocity vector field as input features and combining them with the UAV's position data, a pre-trained area redivision reinforcement learning model is used to output the UAV's autonomous decision-making reinforcement learning results, which include a dynamic segmentation scheme for uninspected areas, based on a reward function for multi-objective optimization. The reward function is used to maximize area coverage, maximize the probability of target tracking success, and minimize path energy consumption.

2. The UAV autonomous decision-making reinforcement learning method based on multi-objective optimization according to claim 1 is characterized in that: The method uses the movement probability heat map and the target velocity vector field as input features, and combines them with the drone's position data. A pre-trained region redivision reinforcement learning model is used based on a reward function for multi-objective optimization to output a drone autonomous decision-making reinforcement learning result that includes a dynamic segmentation scheme for uninspected areas, including: Extracting probability distribution features of the movement probability heat map and probability distribution features of the target velocity vector field, and performing spatiotemporal correlation fusion of the probability distribution features and the position data of the UAV to generate a joint state feature vector; Inputting the joint state feature vector into the region redivision reinforcement learning model, and outputting a set of segmentation parameters for the uninspected region through a policy network; Based on the segmentation parameter set, performing a multi-dimensional evaluation using the reward function for multi-objective optimization to generate a multi-dimensional evaluation result; Based on the multi-dimensional evaluation results, the policy network of the area redivision reinforcement learning model is driven to update weight parameters and output a dynamic segmentation scheme for the uninspected area; The dynamic segmentation scheme is bound to the real-time environmental status to generate the autonomous decision-making reinforcement learning results of the UAV including the dynamic segmentation scheme of the uninspected area.

3. The UAV autonomous decision-making reinforcement learning method based on multi-objective optimization according to claim 2 is characterized in that: The method of binding the dynamic segmentation scheme with the real-time environmental state to generate a UAV autonomous decision-making reinforcement learning result of the dynamic segmentation scheme including the uninspected area includes: Extracting the geometric vertex coordinates of the uninspected area and the geographic space coordinates in the real-time environment state based on the boundary information of the dynamic segmentation scheme, establishing a mapping relationship between the geometric vertex coordinates and the geographic space coordinates, and generating a spatial coordinate mapping table; According to the time window constraint in the real-time environment state and the motion dynamics parameters of the UAV, the priority information of the dynamic segmentation scheme is converted into a timestamp tag of the path execution sequence to generate an execution timing logic; The spatial coordinate mapping table is integrated with the execution timing logic to construct a structured decision data flow; The structured decision data stream is encoded to generate a UAV autonomous decision reinforcement learning result.

4. The UAV autonomous decision-making reinforcement learning method based on multi-objective optimization according to claim 2 is characterized in that: The step of performing a multi-dimensional evaluation based on the segmentation parameter set and utilizing the reward function for multi-objective optimization to generate a multi-dimensional evaluation result includes: Based on the area boundary control points in the segmentation parameter set, using the first term in the reward function to calculate the spatial coverage blind area distribution of the uninspected area, and generate a coverage scalar corresponding to a first target, where the first target is the area coverage rate; Based on the segmentation priority weights in the segmentation parameter set, combined with the high-probability target distribution in the movement probability heat map and the motion direction consistency of the target velocity vector field, a matching degree scalar corresponding to a second target is generated using the second term in the reward function, where the second target is the probability of successful tracking of the suspicious target; Based on the spatial density of the boundary control points in the segmentation parameter set and the current position of the UAV, using the third term in the reward function to generate an energy consumption scalar corresponding to a third goal, wherein the third goal is the path energy consumption; The coverage scalar, matching scalar and energy consumption scalar are linearly superimposed to generate the multi-dimensional evaluation result.

5. The UAV autonomous decision-making reinforcement learning method based on multi-objective optimization according to claim 1 is characterized in that: The constructing of a movement probability heat map based on the movement trajectory prediction sequence includes: Deconstructing the target area into a plurality of discrete trajectory points including spatial coordinates and observation times according to the movement trajectory prediction sequence according to geographic grid units to form a discrete trajectory point set; Calculating a spatial distance weight based on a preset spatial weight kernel function and the spatial coordinates of the discrete trajectory points, and calculating a time decay weight based on a preset time decay coefficient and the observation time of the discrete trajectory points; Based on the spatial distance weight and the time decay weight, the spatial coordinates and the observation time are fused to generate an initial thermal value of the geographic grid cell; The initial thermal values ​​of all the geographic grid cells are converted into a color gradient distribution to form a movement probability heat map.

6. The UAV autonomous decision-making reinforcement learning method based on multi-objective optimization according to claim 1 is characterized in that: The performing Doppler frequency shift analysis on the radar data to generate a target velocity vector field includes: Deconstructing the radar data into a set of discrete echo units, each discrete echo unit including an azimuth angle, an elevation angle, and an original frequency shift; Performing periodic folding correction on the original frequency shift to generate a true Doppler frequency shift value; Converting the true Doppler frequency shift value into a radial velocity component, and calculating a three-dimensional velocity projection by combining the azimuth angle and the elevation angle; Perform motion consistency aggregation on the three-dimensional velocity projections to obtain a target velocity vector field.

7. The UAV autonomous decision-making reinforcement learning method based on multi-objective optimization according to claim 1 is characterized in that: The generating, based on the infrared data and the radar data, a movement trajectory prediction sequence of the suspicious target within the target area, comprises: Based on the temperature gradient distribution corresponding to the infrared data, the center coordinates of abnormal hot spots whose temperature gradient distribution is higher than a preset temperature difference threshold are extracted in the inspected area, and the center coordinates of all abnormal hot spots are sorted according to the acquisition timestamp to generate a target hot spot position sequence; Within a preset time window, the target hot spot position sequence is fused with the spatial coordinates of the trajectory segment of the radar data to generate a fused trajectory point set; Connecting the fused trajectory point set in the order of timestamps to form a trajectory chain; According to the consistency of the velocity vector direction, the trajectory is extended based on the trajectory chain to output a movement trajectory prediction sequence of the suspicious target in the target area.

8. A multi-objective optimization-based autonomous decision-making reinforcement learning system for unmanned aerial vehicles, characterized by: include: The acquisition module is used to collect infrared data and radar data within the patrolled area through the airborne sensor array deployed on the UAV; A construction module is configured to generate a movement trajectory prediction sequence of a suspicious target within a target area based on the infrared data and the radar data, and to construct a movement probability heat map based on the movement trajectory prediction sequence, wherein the target area includes an inspected area; A generating module, configured to perform Doppler frequency shift analysis on the radar data to generate a target velocity vector field; The output module is used to take the movement probability heat map and the target velocity vector field as input features, and combine them with the position data of the drone, and output the drone autonomous decision-making reinforcement learning results including the dynamic segmentation scheme of the uninspected area based on the reward function for multi-objective optimization through a pre-trained area redivision reinforcement learning model. The reward function is used to reflect the maximization of area coverage, the maximization of the probability of target tracking success, and the minimization of path energy consumption.

9. A computing device, characterized in that It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a multi-objective optimization-based autonomous decision-making reinforcement learning method for unmanned aerial vehicles as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that A computer program is stored, and when the computer program is executed by a computer, the method for autonomous decision-making reinforcement learning of a drone based on multi-objective optimization is implemented as described in any one of claims 1 to 7.

Images