Rotorcraft unmanned vehicle hazard identification management method and system for complex terrain

Abstract

The application discloses a rotor unmanned aerial vehicle hazard identification management method and system for complex terrain, and belongs to the technical field of unmanned aerial vehicle monitoring. The system comprises a three-dimensional model generation module, a hovering point acquisition module, a data acquisition module, a hazard identification module, a hazard level sorting module and a data management module. The three-dimensional terrain model is generated in real time through a terrain complexity quantification algorithm, so that the preset path planning algorithm dynamically selects the hovering detection point on the optimal cruising path; the visible light, infrared and acoustic multi-modal data are synchronously collected at the hovering point, the hazard feature vector is extracted through YOLOv7 network fusion, the risk level is generated and sorted, and finally the management report containing the position, level and disposal suggestion is output, so that the detection coverage range is expanded, the identification confidence is improved, and the emergency response time is shortened under the complex terrain, and the problems of the expansion of the detection blind area, the decline of the identification confidence and the extension of the response time under the complex terrain are solved.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) monitoring technology, and in particular to a method and system for identifying and managing hazards in rotary-wing UAVs used in complex terrain. Background Technology

[0002] With the continued growth in demand for public safety, emergency rescue, and infrastructure inspection, the rapid identification and intelligent management of hazardous materials such as explosives, leak sources, and crash debris in complex terrain by rotary-wing UAVs has become a core aspect of low-altitude safe operations.

[0003] Existing technologies, such as the AI-based smart construction site safety supervision method and system published in Chinese Patent Publication No. CN119152148A, acquire drone aerial images and point cloud data at fixed time intervals, generate a digital elevation model, extract elevation data, and integrate it with the features and edge characteristics of the aerial images to form a three-dimensional image pair, ultimately constructing a digital map of the construction site, and then outputting safety risk coefficients based on personnel locations and hazard factors; another example is the laser point cloud-based curtain wall deformation safety detection method and device published in Chinese Patent Publication No. CN120162985A, which first performs coordinate unification processing on the overall point cloud of the curtain wall to generate a digital twin model of the curtain wall surface, then extracts the surface normal vector and combines it with wind pressure and temperature field information to construct a dynamic flexural deformation calculation model, outputting global and local deformation parameters to achieve high-precision dynamic monitoring of the flatness of the exterior wall.

[0004] In existing technologies, the former focuses on the construction of static maps of construction scenarios, and the resulting maps are static results. Path planning and sensor configuration are executed according to a predetermined script. Terrain undulations are only used for elevation display and do not participate in real-time decision-making. The latter targets the unified coordinates of point clouds on the entire surface of the curtain wall, constructs a twin model, and then integrates wind pressure-temperature field to extract deformation parameters. The evaluation results are limited to the flatness of the building's exterior walls, and the task objective is singular. Neither of these technologies can meet the comprehensive requirements of high coverage, high confidence, and rapid response in emergency scenarios. They also have problems such as expanded detection blind spots, decreased recognition confidence, and extended response time under complex terrain. Summary of the Invention

[0005] This application provides a method and system for identifying and managing hazards from rotary-wing UAVs in complex terrain. It solves the problems of increased detection blind spots, decreased identification confidence, and prolonged response time in the prior art under complex terrain, and realizes the expansion of detection coverage, improvement of identification confidence, and shortening of emergency response time under complex terrain.

[0006] This application provides a method for hazard identification and management of rotary-wing UAVs in complex terrain, applied in a hazard identification and management system for rotary-wing UAVs in complex terrain, including:

[0007] Acquire lidar point cloud data of complex terrain, calculate terrain undulation parameters through terrain complexity quantification algorithm, and generate 3D terrain model data;

[0008] Based on 3D terrain model data, the optimal cruise path coordinate sequence is calculated using a preset path planning algorithm, and hovering detection point location data is obtained according to terrain gradient changes.

[0009] At each hovering detection point, multimodal sensors are activated to synchronously acquire multimodal sensor data, including visible light image data, infrared thermal imaging data, and acoustic feature data.

[0010] Multimodal sensor data is input into a preset YOLOv7 hazard identification network to extract hazard feature vectors and generate hazard location coordinates and category identification data;

[0011] Based on the feature vectors of hazards, the risk level values ​​of each hazard are calculated using a threat assessment algorithm, and the hazards are prioritized according to their risk level values.

[0012] The sorted hazardous materials data are stored in the hazardous materials management database to generate a hazardous materials management report that includes location information, threat level, and handling recommendations.

[0013] Furthermore, the terrain complexity quantification algorithm includes:

[0014] The lidar point cloud data is preprocessed to remove noise points and then a regular grid is constructed.

[0015] Calculate the elevation variance and slope gradient of each point within the grid using a terrain complexity quantification formula:

[0016] ;

[0017] in, This represents the terrain complexity value. The variance of the elevation values ​​of all points within the grid. The magnitude of the slope gradient vector. For the roughness parameters of the mesh, The point cloud density within the grid. and These are the weighting coefficients;

[0018] Sort all the terrain complexity values ​​in ascending order and divide them into three intervals using the ternary method. The terrain complexity value is not greater than the first complexity threshold and is flat. The terrain complexity value is greater than the first complexity threshold but not greater than the second complexity threshold and is medium complex. The terrain complexity value is greater than the second complexity threshold and is high complex.

[0019] Furthermore, the preset path planning algorithm includes:

[0020] The optimal planning path is derived from the total cost function, and the total cost function of the path nodes is:

[0021] ;

[0022] in, From the mission launch point to the node The actual cost already incurred. For the node Heuristic cost to reaching the end of the mission. This comes at the cost of terrain adaptability.

[0023] Terrain Adaptation Cost The calculation formula is: ,in As the terrain complexity weight, For nodes The terrain complexity value at that location. The weighting is determined by the impact of obstacles. For nodes The density of obstacles at that location.

[0024] Furthermore, the steps for acquiring the hovering detection point position data include:

[0025] Based on the optimal cruise path coordinate sequence, obtain the terrain gradient change rate at each point on the path.

[0026] When the rate of change of terrain gradient exceeds a preset threshold When this happens, mark it as a potential hovering point;

[0027] Optimize the view coverage algorithm:

[0028] ;

[0029] in, The score is given for the field of view coverage at the hovering point. The number of observable areas. For the first The area of ​​each region For the angle of observation, For observation distance, The occlusion factor;

[0030] Calculated using ray tracing algorithm ,in For the first The volume of the occlusion. Total observed volume;

[0031] choose The point with the largest value is taken as the final hover detection point position.

[0032] Furthermore, the synchronous acquisition by the multimodal sensor includes:

[0033] Establish a multi-sensor time synchronization mechanism and set a unified timestamp benchmark;

[0034] The system uses a visible light camera to acquire RGB image data at a fixed frame rate, an infrared thermal imager to acquire temperature distribution data at a fixed frame rate, and an acoustic sensor to acquire audio signals at a fixed sampling rate.

[0035] A time alignment algorithm is used to unify sensor data of different frequencies into the same time coordinate system;

[0036] Each sensor is independently calibrated with intrinsic parameters to obtain its own pixel-physical coordinate mapping parameters. Then, extrinsic parameter calibration is used to obtain the transformation matrix from visible light pixel coordinates, infrared pixel coordinates, acoustic array coordinates to world coordinates, thus completing spatial alignment.

[0037] The acquired raw data is preprocessed, including denoising the visible light image, calibrating the infrared thermal image, and bandpass filtering the acoustic signal.

[0038] Output a time-space aligned and dimensionally uniform multimodal data set.

[0039] Furthermore, the preset YOLOv7 hazard identification network includes:

[0040] Image features at different scales are extracted using a feature pyramid network, and a joint feature vector is constructed by combining thermal imaging features and acoustic features.

[0041] Feature fusion weights are dynamically adjusted through a self-attention mechanism:

[0042] ;

[0043] in, To fuse the weight matrix, , , These are respectively query, key-value, and numerical matrix. It is the transpose of the key matrix. For feature dimensions;

[0044] The network outputs the bounding box coordinates, confidence score, and class probability of the hazard, while generating a hazard feature vector that includes shape, texture, and thermal radiation features.

[0045] Furthermore, the roughness parameter is obtained by means of:

[0046] For the point cloud data within the regular grid, calculate the absolute value of the elevation difference between adjacent points and substitute it into the roughness parameter quantization formula;

[0047]

[0048] in, The number of points within the grid. For the first Elevation values ​​of each point This represents the average elevation of the points within the grid.

[0049] Furthermore, the threat assessment algorithm includes:

[0050] A multi-dimensional threat assessment model is constructed based on the feature vectors of hazardous substances;

[0051] The formula for calculating threat level is:

[0052] ;

[0053] in, To calculate the overall threat level, This is an assessment value for the explosive power. This is the location hazard assessment value. Distance is the influencing factor. This represents the linearly normalized value of the terrain complexity. , , , These are the weighting coefficients.

[0054] Furthermore, the calculation of the distance influence factor includes:

[0055] Measure the straight-line distance between the hazard and the current position of the drone. The effective handling distance is calculated through accessibility analysis:

[0056] ;

[0057] in, Standard handling distance, For the first The height of the obstruction, The width of the obstruction, Where is the total shading height, e is the natural constant, and V is the total number of obstructions.

[0058] This application provides a hazard identification and management system for rotary-wing unmanned aerial vehicles (UAVs) in complex terrain, which implements a hazard identification and management method for rotary-wing UAVs in complex terrain, including:

[0059] The 3D model generation module is used to acquire lidar point cloud data of complex terrain, calculate terrain undulation parameters through terrain complexity quantification algorithm, and generate 3D terrain model data.

[0060] The hovering point acquisition module is used to calculate the optimal cruise path coordinate sequence based on 3D terrain model data and a preset path planning algorithm, and to obtain hovering detection point position data according to terrain gradient changes.

[0061] The data acquisition module is used to initiate synchronous acquisition of multimodal sensor data at each hovering detection point, acquiring multimodal sensor data, including visible light image data, infrared thermal imaging data, and acoustic feature data;

[0062] The hazard identification module is used to input multimodal sensor data into a preset YOLOv7 hazard identification network, extract hazard feature vectors, and generate hazard location coordinates and category identification data;

[0063] The hazard level ranking module is used to calculate the risk level value of each hazard based on the hazard feature vector and the threat assessment algorithm, and to prioritize the hazard according to the risk level value.

[0064] The data management module is used to store the sorted hazardous materials data in the hazardous materials management database and generate a hazardous materials management report that includes location information, threat level, and disposal recommendations.

[0065] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0066] 1. The terrain undulation parameters are calculated and three-dimensional terrain model data is generated by the terrain complexity quantification algorithm. Based on the three-dimensional terrain model data, the optimal cruise path coordinate sequence is calculated by the preset path planning algorithm, and the hovering detection point position data is obtained according to the terrain gradient change. This reduces the detection blind zone under complex terrain and effectively solves the problem of the expansion of the detection blind zone under complex terrain in the existing technology.

[0067] 2. By activating multimodal sensors at each hovering detection point to synchronously acquire visible light image data, infrared thermal imaging data, and acoustic feature data, and inputting the multimodal sensor data into a preset YOLOv7 hazard identification network to extract hazard feature vectors, a joint feature vector is constructed by combining thermal imaging features and acoustic features, thereby improving the identification confidence and effectively solving the problem of decreased identification confidence in existing technologies.

[0068] 3. By constructing a multi-dimensional threat assessment model based on the feature vectors of hazardous materials, the comprehensive threat level is calculated and the hazardous materials are prioritized according to their risk level values. The prioritized hazardous materials data is then stored in the hazardous materials management database to generate a hazardous materials management report containing location information, threat level, and disposal recommendations. This shortens the response time and effectively solves the problem of prolonged response time in existing technologies. Attached Figure Description

[0069] Figure 1 A flowchart of a hazard identification and management method for rotary-wing unmanned aerial vehicles (UAVs) in complex terrain, provided in an embodiment of this application;

[0070] Figure 2 This is a structural diagram of a rotary-wing UAV hazard identification and management system for complex terrain provided in an embodiment of this application. Detailed Implementation

[0071] This application provides a method and system for hazardous object identification and management of rotary-wing UAVs in complex terrain. It solves the problems of increased detection blind spots, decreased identification confidence, and prolonged response time in existing technologies under complex terrain. The method generates a three-dimensional terrain model in real time through a terrain complexity quantification algorithm, which drives a preset path planning algorithm to dynamically select hovering detection points on the optimal cruise path. Visible light, infrared, and acoustic multimodal data are collected simultaneously at the hovering points, and hazardous object feature vectors are extracted by YOLOv7 network fusion. Then, risk levels are generated and ranked based on a multidimensional threat assessment model, and finally, a management report containing location, level, and disposal recommendations is output. This achieves the goals of expanding detection coverage, improving identification confidence, and shortening emergency response time under complex terrain.

[0072] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0073] like Figure 1 As shown, this application provides a method for identifying and managing hazards of rotary-wing UAVs in complex terrain. The method is applied to a hazard identification and management system for rotary-wing UAVs in complex terrain, including: acquiring lidar point cloud data of complex terrain, calculating terrain undulation parameters through a terrain complexity quantification algorithm, and generating three-dimensional terrain model data.

[0074] Based on 3D terrain model data, the optimal cruise path coordinate sequence is calculated using a preset path planning algorithm, and hovering detection point location data is obtained according to terrain gradient changes.

[0075] At each hovering detection point, multimodal sensors are activated to synchronously acquire multimodal sensor data, including visible light image data, infrared thermal imaging data, and acoustic feature data.

[0076] Multimodal sensor data is input into a preset YOLOv7 hazard identification network to extract hazard feature vectors and generate hazard location coordinates and category identification data;

[0077] Based on the feature vectors of hazards, the risk level values ​​of each hazard are calculated using a threat assessment algorithm, and the hazards are prioritized according to their risk level values.

[0078] The sorted hazardous materials data are stored in the hazardous materials management database to generate a hazardous materials management report that includes location information, threat level, and handling recommendations.

[0079] Furthermore, the terrain complexity quantification algorithm includes:

[0080] The lidar point cloud data is preprocessed to remove noise points and then a regular grid is constructed.

[0081] Calculate the elevation variance and slope gradient of each point within the grid using a terrain complexity quantification formula:

[0082] ;

[0083] in, This represents the terrain complexity value. The variance of the elevation values ​​of all points within the grid. This represents the magnitude of the slope gradient vector, specifically the length of the slope gradient vector at the grid center point. For the roughness parameters of the mesh, The point cloud density within the grid. and These are the weighting coefficients;

[0084] The data was obtained by fitting historical topographic data using the least squares method, and the calculation formula is as follows: ,in The value represents the manually labeled terrain complexity level of the p-th historical sample grid (flat = 1, medium = 2, high complexity = 3). For the first Elevation variance of historical sample grids;

[0085] The optimal value is determined using cross-validation.

[0086] Sort all the terrain complexity values ​​in ascending order and divide them into three intervals using the ternary method. The terrain complexity value is not greater than the first complexity threshold and is flat. The terrain complexity value is greater than the first complexity threshold but not greater than the second complexity threshold and is medium complex. The terrain complexity value is greater than the second complexity threshold and is high complex.

[0087] Furthermore, the preset path planning algorithm includes:

[0088] The optimal planning path is derived from the total cost function, and the total cost function of the path nodes is:

[0089] ;

[0090] in, From the mission launch point to the node The actual cost already incurred. For the node Heuristic cost to reaching the end of the mission. This comes at the cost of terrain adaptability.

[0091] Terrain Adaptation Cost The calculation formula is: ,in As the terrain complexity weight, For nodes The terrain complexity value at that location. The weighting is determined by the impact of obstacles. For nodes Obstacle density at a given location, i.e., the number of LiDAR obstacle points per unit area;

[0092] The correlation analysis between path length and terrain complexity determined the calculation formula as follows: ,in This is a path length vector. Let covariance function be used. It is the variance function;

[0093] The optimal value is determined through obstacle avoidance performance evaluation, which involves generating test scenarios with different obstacle densities in a simulation environment; and traversing candidate obstacles using path length and collision rate as evaluation metrics. Value; select the value that minimizes the path and the collision rate. As the optimal value.

[0094] Furthermore, the steps for acquiring the hovering detection point position data include:

[0095] Based on the optimal cruise path coordinate sequence, obtain the terrain gradient change rate at each point on the path.

[0096] When the rate of change of terrain gradient exceeds a preset threshold When this happens, mark it as a potential hovering point;

[0097] Optimize the view coverage algorithm:

[0098] ;

[0099] in, The score is given for the field of view coverage at the hovering point. The number of observable areas. For the first The area of ​​each region For the angle of observation, For observation distance, The occlusion factor;

[0100] Calculated using ray tracing algorithm ,in For the first The volume of the occlusion. Total observed volume;

[0101] choose The point with the largest value is used as the final hover detection point location to ensure the maximum coverage of hazard identification.

[0102] In this embodiment, the observable area is divided into sectors at equal angular intervals (e.g., every 15°) on the horizontal plane, centered on the hovering point. Each sector is further divided into ring-shaped grids at equal intervals (e.g., every 10m). Each grid represents an observable area, and its number Q is the product of the number of sectors and the number of ring layers. The occlusion volume is numbered with the hovering point as the origin, and the lidar point cloud is voxelized along the observation ray direction. Each voxel pierced by the ray is considered an occlusion unit, numbered sequentially r=1,2,…,R according to the ray direction, and its volume is denoted as… Observation angle Defined as the angle (0°~180°) between the line connecting the hovering point to the center of the qth observable area and the vertical downward direction of the hovering point. The optimal cruise path coordinate sequence is given in the Northeast-Sky (ENU) local coordinate system corresponding to the lidar point cloud, with the origin set at the takeoff point, the X-axis pointing east, the Y-axis pointing north, and the Z-axis pointing to the sky. All coordinates are represented in this coordinate system.

[0103] Furthermore, the synchronous acquisition by the multimodal sensor includes:

[0104] Establish a multi-sensor time synchronization mechanism and set a unified timestamp benchmark;

[0105] The system uses a visible light camera to acquire RGB image data at a fixed frame rate, an infrared thermal imager to acquire temperature distribution data at a fixed frame rate, and an acoustic sensor to acquire audio signals at a fixed sampling rate.

[0106] A time alignment algorithm is used to unify sensor data of different frequencies into the same time coordinate system;

[0107] Each sensor is independently calibrated with intrinsic parameters to obtain its own pixel-physical coordinate mapping parameters. Then, extrinsic parameter calibration is used to obtain the transformation matrix from visible light pixel coordinates, infrared pixel coordinates, acoustic array coordinates to Northeastern Sky (ENU) world coordinates, thus completing spatial alignment.

[0108] The acquired raw data is preprocessed, including denoising the visible light image, calibrating the infrared thermal image, and bandpass filtering the acoustic signal.

[0109] The output is a time-space aligned and dimensionally uniform multimodal data set. Visible light data provides image features, including texture, shape, and color; infrared data provides thermal imaging features, including temperature distribution and thermal gradient; and acoustic data provides acoustic features, including spectrum, energy, and harmonic structure. These three types of features serve as the input sources for the preset YOLOv7 hazard identification network: image features, thermal imaging features, and acoustic features.

[0110] Furthermore, the preset YOLOv7 hazard identification network includes:

[0111] Image features at different scales are extracted using a feature pyramid network, and a joint feature vector is constructed by combining thermal imaging features and acoustic features.

[0112] Feature fusion weights are dynamically adjusted through a self-attention mechanism:

[0113] ;

[0114] in, To fuse the weight matrix, , , These are respectively query, key-value, and numerical matrix. It is the transpose of the key matrix. For feature dimension ( (representing the dimension of the key vector).

[0115] Through linear transformation Obtain, among which For the input feature matrix, To query the weight matrix;

[0116] Similarly, and Calculate using the same method;

[0117] The network outputs the bounding box coordinates, confidence score, and class probability of the hazard, while generating a hazard feature vector that includes shape, texture, and thermal radiation features.

[0118] Furthermore, the roughness parameter is obtained by means of:

[0119] For the point cloud data within the regular grid, calculate the absolute value of the elevation difference between adjacent points and substitute it into the roughness parameter quantization formula;

[0120] ;

[0121] in, The number of points within the grid. For the first Elevation values ​​of each point The average elevation of the points within the grid;

[0122] For each grid, roughness parameters are calculated, and a terrain roughness distribution map is constructed.

[0123] Furthermore, the threat assessment algorithm includes:

[0124] A multi-dimensional threat assessment model is constructed based on the feature vectors of hazardous substances;

[0125] The formula for calculating threat level is:

[0126] ;

[0127] in, To calculate the overall threat level, This is an assessment value for the explosive power. This is the location hazard assessment value. Distance is the influencing factor. This represents the linearly normalized value of the terrain complexity. , , , The weighting coefficients are obtained from the database.

[0128] By directly reading the hazard level corresponding to the maximum probability of hazard category from the database, the database pre-stores three levels: low, medium, and high, which are mapped to discrete values: low=1, medium=2, and high=3. Read the planar coordinates and population density of each key objective from the task database. ( (persons / km²), calculate the Euclidean distance from the hazard to the target. , .

[0129] Furthermore, the calculation of the distance influence factor includes:

[0130] Measure the straight-line distance between the hazard and the current position of the drone. The effective handling distance is calculated through accessibility analysis:

[0131] ;

[0132] in, Standard handling distance, For the first The height of the obstruction, The width of the obstruction, Where is the total shading height, e is the natural constant, and V is the total number of obstructions.

[0133] Obstacle identification is performed using LiDAR data to calculate the distribution of obstacles between the hazard and the drone; based on the calculated distance influence factor, the disposal priority of the hazard is adjusted, with hazards that are closer and more accessible receiving a higher disposal priority.

[0134] like Figure 2 As shown, this application embodiment provides a hazard identification and management system for rotary-wing UAVs in complex terrain, used to implement the hazard identification and management method for rotary-wing UAVs in complex terrain, including: a 3D model generation module, a hovering point acquisition module, a data acquisition module, a hazard identification module, a hazard level sorting module, and a data management module;

[0135] The three-dimensional model generation module is used to acquire lidar point cloud data of complex terrain, calculate terrain undulation parameters through a terrain complexity quantification algorithm, and generate three-dimensional terrain model data.

[0136] The hovering point acquisition module is used to calculate the optimal cruise path coordinate sequence based on three-dimensional terrain model data and a preset path planning algorithm, and to obtain hovering detection point position data according to terrain gradient changes.

[0137] The data acquisition module is used to activate the multimodal sensor for synchronous acquisition at each hovering detection point, and acquire multimodal sensor data, including visible light image data, infrared thermal imaging data and acoustic feature data.

[0138] The hazard identification module is used to input multimodal sensor data into a preset YOLOv7 hazard identification network, extract hazard feature vectors, and generate hazard location coordinates and category identification data;

[0139] The hazard level ranking module is used to calculate the risk level value of each hazard based on the hazard feature vector and through a threat assessment algorithm, and to prioritize the hazard according to the risk level value.

[0140] The data management module is used to store the sorted hazardous materials data in the hazardous materials management database and generate a hazardous materials management report that includes location information, threat level, and disposal recommendations.

[0141] In summary, this application embodiment generates a three-dimensional terrain model in real time through a terrain complexity quantification algorithm, drives a preset path planning algorithm to dynamically select hovering detection points on the optimal cruise path; simultaneously collects visible light, infrared, and acoustic multimodal data at the hovering points, and extracts hazard feature vectors through YOLOv7 network fusion; then generates and sorts risk levels based on a multidimensional threat assessment model, and finally outputs a management report containing location, level, and handling recommendations, thereby expanding the detection coverage, improving identification confidence, and shortening emergency response time in complex terrain.

[0142] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0143] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0145] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0146] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0147] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0148] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for hazard identification and management of rotary-wing unmanned aerial vehicles (UAVs) in complex terrain, characterized in that, Includes the following steps: Acquire lidar point cloud data of complex terrain, calculate terrain undulation parameters through terrain complexity quantification algorithm, and generate 3D terrain model data; The terrain complexity quantification algorithm includes: The lidar point cloud data is preprocessed to remove noise points and then a regular grid is constructed. Calculate the elevation variance and slope gradient of each point within the grid using a terrain complexity quantification formula: ； in, This represents the terrain complexity value. The variance of the elevation values ​​of all points within the grid. The magnitude of the slope gradient vector. For the roughness parameters of the mesh, The point cloud density within the grid. and These are the weighting coefficients; Sort all the terrain complexity values ​​in ascending order and divide them into three intervals using the ternary method: terrain complexity values ​​not greater than the first complexity threshold are flat, terrain complexity values ​​greater than the first complexity threshold but not greater than the second complexity threshold are medium complexity, and terrain complexity values ​​greater than the second complexity threshold are high complexity. Based on 3D terrain model data, the optimal cruise path coordinate sequence is calculated using a preset path planning algorithm, and hovering detection point location data is obtained according to terrain gradient changes. The preset path planning algorithm includes: The optimal planning path is derived from the total cost function, and the total cost function of the path nodes is: ； in, From the mission launch point to the node The actual cost already incurred. For the node Heuristic cost to reaching the end of the mission. This comes at the cost of terrain adaptability. Terrain Adaptation Cost The calculation formula is: ,in As the terrain complexity weight, For nodes The terrain complexity value at that location. The weighting is determined by the impact of obstacles. For nodes Obstacle density at the location; The steps for obtaining the hover detection point location data include: Based on the optimal cruise path coordinate sequence, obtain the terrain gradient change rate at each point on the path. When the rate of change of terrain gradient exceeds a preset threshold When this happens, mark it as a potential hovering point; Optimize the view coverage algorithm: ； in, The score is given for the field of view coverage at the hovering point. The number of observable areas. For the first The area of ​​each region For the angle of observation, For observation distance, The occlusion factor; Calculated using ray tracing algorithm ,in For the first The volume of the occlusion. Total observed volume; choose The point with the largest value is taken as the final hover detection point position; At each hovering detection point, multimodal sensors are activated to synchronously acquire multimodal sensor data, including visible light image data, infrared thermal imaging data, and acoustic feature data. Multimodal sensor data is input into a preset YOLOv7 hazard identification network to extract hazard feature vectors and generate hazard location coordinates and category identification data; Based on the feature vectors of hazards, the risk level values ​​of each hazard are calculated using a threat assessment algorithm, and the hazards are prioritized according to their risk level values. The sorted hazardous materials data are stored in the hazardous materials management database to generate a hazardous materials management report that includes location information, threat level, and handling recommendations.

2. The method for hazard identification and management of rotary-wing unmanned aerial vehicles (UAVs) in complex terrain as described in claim 1, characterized in that, The synchronous acquisition by the multimodal sensor includes: Establish a multi-sensor time synchronization mechanism and set a unified timestamp benchmark; The system uses a visible light camera to acquire RGB image data at a fixed frame rate, an infrared thermal imager to acquire temperature distribution data at a fixed frame rate, and an acoustic sensor to acquire audio signals at a fixed sampling rate. A time alignment algorithm is used to unify sensor data of different frequencies into the same time coordinate system; Each sensor is independently calibrated with intrinsic parameters to obtain its own pixel-physical coordinate mapping parameters. Then, extrinsic parameter calibration is used to obtain the transformation matrix from visible light pixel coordinates, infrared pixel coordinates, acoustic array coordinates to world coordinates, thus completing spatial alignment. The acquired raw data is preprocessed, including denoising the visible light image, calibrating the infrared thermal image, and bandpass filtering the acoustic signal. Output a time-space aligned and dimensionally uniform multimodal data set.

3. The method for hazard identification and management of rotary-wing UAVs in complex terrain as described in claim 2, characterized in that, The preset YOLOv7 hazardous substance identification network includes: Image features at different scales are extracted using a feature pyramid network, and a joint feature vector is constructed by combining thermal imaging features and acoustic features. Feature fusion weights are dynamically adjusted through a self-attention mechanism: ； in, To fuse the weight matrix, , , These are respectively query, key-value, and numerical matrix. It is the transpose of the key matrix. For feature dimensions; The network outputs the bounding box coordinates, confidence score, and class probability of the hazard, while generating a hazard feature vector that includes shape, texture, and thermal radiation features.

4. The method for hazard identification and management of rotary-wing unmanned aerial vehicles (UAVs) in complex terrain as described in claim 1, characterized in that, The roughness parameters are obtained in the following ways: For the point cloud data within the regular grid, calculate the absolute value of the elevation difference between adjacent points and substitute it into the roughness parameter quantization formula; in, The number of points within the grid. For the first Elevation values ​​of each point This represents the average elevation of the points within the grid.

5. The method for hazard identification and management of rotary-wing unmanned aerial vehicles (UAVs) in complex terrain as described in claim 1, characterized in that, The threat assessment algorithm includes: A multi-dimensional threat assessment model is constructed based on the feature vectors of hazardous substances; The formula for calculating threat level is: ； in, To calculate the overall threat level, This is an assessment value for the explosive power. This is the location hazard assessment value. Distance is the influencing factor. This represents the linearly normalized value of the terrain complexity. , , , These are the weighting coefficients.

6. The method for hazard identification and management of rotary-wing unmanned aerial vehicles (UAVs) in complex terrain as described in claim 5, characterized in that, The calculation of the distance influence factor includes: Measure the straight-line distance between the hazard and the current position of the drone. The effective handling distance is calculated through accessibility analysis: ； in, Standard handling distance, For the first The height of the obstruction, The width of the obstruction, Where is the total shading height, e is the natural constant, and V is the total number of obstructions.

7. A hazard identification and management system for rotary-wing unmanned aerial vehicles (UAVs) in complex terrain, used to implement the hazard identification and management method for rotary-wing UAVs in complex terrain as described in any one of claims 1-6, characterized in that, include: The 3D model generation module is used to acquire lidar point cloud data of complex terrain, calculate terrain undulation parameters through terrain complexity quantification algorithm, and generate 3D terrain model data. The hovering point acquisition module is used to calculate the optimal cruise path coordinate sequence based on 3D terrain model data and a preset path planning algorithm, and to obtain hovering detection point position data according to terrain gradient changes. The data acquisition module is used to initiate synchronous acquisition of multimodal sensor data at each hovering detection point, acquiring multimodal sensor data, including visible light image data, infrared thermal imaging data, and acoustic feature data; The hazard identification module is used to input multimodal sensor data into a preset YOLOv7 hazard identification network, extract hazard feature vectors, and generate hazard location coordinates and category identification data; The hazard level ranking module is used to calculate the risk level value of each hazard based on the hazard feature vector and the threat assessment algorithm, and to prioritize the hazard according to the risk level value. The data management module is used to store the sorted hazardous materials data in the hazardous materials management database and generate a hazardous materials management report that includes location information, threat level, and disposal recommendations.

Images