Mountain search and rescue unmanned aerial vehicle and system based on visual recognition and shouting guidance

Abstract

This invention discloses a mountain search and rescue drone and system based on visual recognition and voice guidance. The drone includes a fuselage, a three-axis stabilized gimbal, a folding pole, a protective frame, and a power module. The three-axis stabilized gimbal is fixedly connected to the bottom of the fuselage. This invention employs a multimodal visual perception scheme that fuses visible light images and infrared thermal data, combined with a three-axis stabilized gimbal, to achieve large field of view and high stability image acquisition, avoiding interference from environmental factors and enabling rapid and accurate location of trapped personnel. This invention constructs a closed-loop search and rescue system with two-way voice interaction through a voice guidance unit, which can complete the confirmation of the trapped personnel's status, emotional reassurance, and precise guidance of the rescue route. Through multi-sensor data fusion, the Cartographer algorithm is used to construct a two-dimensional raster map of the mountain scene, and extended Kalman filtering is used to achieve high-precision odometer calculation, solving the positioning drift problem in mountainous areas without satellite signals.

Description

Technical Field

[0001] This invention relates to the field of emergency rescue technology, specifically to a mountain search and rescue drone and system based on visual recognition and verbal guidance. Background Technology

[0002] Mountainous terrain is complex, with rugged terrain, dense vegetation, weak communication signals, and frequent extreme weather, posing significant challenges to emergency rescue operations. Traditional manual search and rescue methods are not only inefficient but also carry high safety risks. Therefore, existing technologies generally employ drones for search and rescue operations in mountainous areas. However, existing drone search and rescue technologies still have the following drawbacks in practical applications: 1. Most existing search and rescue drones use a single visible light or infrared detection scheme. In mountainous areas with dense vegetation, changing light and shadow, rain, fog, and low visibility at night, the target recognition accuracy drops significantly, making it difficult to accurately detect trapped personnel covered by vegetation. At the same time, the gimbal's image stabilization performance is insufficient, resulting in severe image shaking during flight, further affecting the recognition effect and failing to meet the search and rescue needs of complex mountainous environments. 2. Most existing rescue drones are only equipped with a one-way voice broadcast module, which cannot achieve two-way voice interaction with trapped personnel, making it difficult to complete operations such as confirming the trapped personnel's status, calming their emotions, and guiding them to accurate rescue routes. 3. Existing drones are highly dependent on GPS / BeiDou satellite positioning. In mountainous canyons, dense forests, and other areas with vegetation cover and no satellite signals, positioning drift and path inaccuracy are prone to occur, making autonomous navigation and accurate obstacle avoidance impossible. Summary of the Invention

[0003] The purpose of this invention is to provide a mountain search and rescue drone and system based on visual recognition and verbal guidance, so as to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a mountain search and rescue drone and system based on visual recognition and voice guidance, comprising a fuselage, a three-axis stabilization gimbal, folding rods, a protective frame, and a power module. The three-axis stabilization gimbal is fixedly connected to the bottom of the fuselage, and multiple folding rods are fixedly connected to the fuselage. A protective frame is fixedly connected to one end of each folding rod, and a power module is fixedly connected to the protective frame.

[0005] The power module includes a motor and a blade, with the motor fixedly connected to the protective frame and the blade fixedly connected to the output end of the motor, and the blade is set inside the protective frame.

[0006] A system for mountain search and rescue drones based on visual recognition and voice guidance includes a main control module, an execution module, a data acquisition module, a data storage module, a wireless communication module, a power supply module, and a ground terminal. The ground terminal establishes a data connection with the main control module through the wireless communication module. The main control module establishes electrical connections with the execution module, data acquisition module, data storage module, and power supply module. The main control module coordinates the collaborative operation of each module and completes computation and control tasks. The execution module executes the instructions issued by the main control module. The data acquisition module collects information and realizes environmental perception. The data storage module stores data. The wireless communication module realizes wireless data transmission. The power supply module supplies power to the drone. The ground terminal is used for remote command and control of the drone.

[0007] The main control module includes a data preprocessing unit, a scene mapping unit, a navigation and obstacle avoidance unit, an anomaly detection unit, a power-on self-test and fault monitoring unit, and a voice guidance unit. The data preprocessing unit is used to standardize and preprocess the collected raw data. The scene mapping unit constructs a two-dimensional grid map of the environment based on the preprocessed LiDAR point cloud data, and optimizes and completes the grid map by integrating visible light images, infrared thermal imaging data, and IMU attitude data. The navigation and obstacle avoidance unit is used to realize the functions of UAV positioning, route planning, dynamic obstacle avoidance, and stable flight. The anomaly detection unit performs target recognition and risk scene detection based on the preprocessed visible light images, infrared thermal imaging data, and LiDAR point cloud data. The power-on self-test and fault monitoring unit is responsible for monitoring the equipment status and safety protection of the UAV throughout the entire operation cycle. The voice guidance unit is responsible for two-way voice interaction control between the UAV and the stranded personnel.

[0008] The scene mapping unit uses the Cartographer algorithm to construct a two-dimensional raster map, which can be expressed by the following formula:

[0009]

[0010] in, To match the optimal pose, For the two-dimensional pose of the drone, Here is the pose transformation matrix. For lidar One scan point, This represents the probability of sub-map grid occupancy. This represents the number of valid scan points.

[0011] The navigation and obstacle avoidance unit includes a positioning calculation subunit, a path planning subunit, and an attitude control subunit. The positioning calculation subunit is used to locate the UAV, the path planning subunit is used to realize global planning of the UAV's flight path and local dynamic obstacle avoidance, and the attitude control subunit is used to generate control commands for the UAV's flight attitude and gimbal attitude.

[0012] The positioning calculation subunit uses extended Kalman filtering to calculate odometry and adaptive Monte Carlo positioning algorithm to achieve UAV self-localization. The particle weight update formula is as follows:

[0013]

[0014] in, for Time of the first The weight of each particle; for Real-time lidar single-frame scan of point cloud; For the first Candidate poses of each particle; To pre-build a two-dimensional raster map of the mountainous area; To observe the likelihood probability;

[0015] The formula for the observation likelihood probability is as follows:

[0016]

[0017] in, This refers to the number of effective scan points in a single frame of the lidar. For the first The Euclidean distance from each scan point to the nearest obstacle grid cell in the map; To measure the standard deviation of noise.

[0018] The attitude control subunit uses a PID algorithm to achieve attitude control, which can be expressed by the following formula:

[0019]

[0020] in, For motor control output, This refers to the deviation between the target angle and the actual angle of the gimbal. For proportionality coefficient, For integral coefficients, These are the differential coefficients. To control the cycle.

[0021] The execution module includes a motor ESC unit, a gimbal drive unit, and a voice output unit. The motor ESC unit is used to control the power output of the UAV, the gimbal drive unit is used to control the gimbal attitude, and the voice output unit is used to play voice information.

[0022] The data acquisition module includes a lidar unit, an inertial measurement unit (IMU), a visible light camera unit, an infrared thermal imaging unit, a voice acquisition unit, and a positioning unit. The lidar unit is used to acquire three-dimensional point cloud data of the environment, the IMU is used to acquire IMU attitude data of the UAV, the visible light camera unit is used to acquire visible light visual information of the environment, the infrared thermal imaging unit is used to acquire infrared thermal imaging data of the environment, the voice acquisition unit is used to acquire voice signals in the environment, and the positioning unit is used for satellite positioning.

[0023] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention adopts a multimodal visual perception scheme that fuses visible light images and infrared thermal data, combined with a three-axis stabilized gimbal, to achieve large field of view and high stability image acquisition, avoiding interference from environmental factors and enabling rapid and accurate positioning of trapped personnel; This invention constructs a closed-loop search and rescue system with two-way voice interaction through a voice guidance unit, which can complete the confirmation of the trapped personnel's status, emotional reassurance, and precise guidance of the rescue route; Through multi-sensor data fusion, the Cartographer algorithm is used to construct a two-dimensional grid map of the mountain scene, and the extended Kalman filter is combined to achieve high-precision odometer calculation, solving the positioning drift problem in mountainous areas without satellite signals. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the overall three-dimensional structure of the UAV of the present invention;

[0025] Figure 2 This is a schematic diagram of the overall side view structure of the UAV of the present invention;

[0026] Figure 3 This is a system structure block diagram of the present invention;

[0027] Figure 4 This is a block diagram of the main control module of the present invention;

[0028] Figure 5 This is a block diagram of the execution module structure of the present invention;

[0029] Figure 6 This is a block diagram of the data acquisition module of the present invention.

[0030] In the diagram: 1. Body; 11. Three-axis stabilized gimbal; 12. Folding rod; 13. Protective frame; 14. Power module; 2. Main control module; 21. Data preprocessing unit; 22. Scene mapping unit; 23. Navigation and obstacle avoidance unit; 231. Positioning calculation subunit; 232. Path planning subunit; 233. Attitude control subunit; 24. Anomaly detection unit; 25. Power-on self-test and fault monitoring unit; 26. Voice guidance unit; 3. Execution module; 31. Motor ESC unit; 32. Gimbal drive unit; 33. Voice output unit; 4. Data acquisition module; 41. LiDAR unit; 42. IMU inertial measurement unit; 43. Visible light camera unit; 44. Infrared thermal imaging unit; 45. Voice acquisition unit; 46. Positioning unit; 5. Data storage module; 6. Wireless communication module; 7. Power supply module; 8. Ground terminal. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Please see the appendix Figure 1 - Appendix Figure 6 This invention provides an embodiment of a mountain search and rescue drone and system based on visual recognition and voice guidance, comprising a fuselage 1, a three-axis stabilization gimbal 11, folding rods 12, a protective frame 13, and a power module 14. The three-axis stabilization gimbal 11 is fixedly connected to the bottom of the fuselage 1, and multiple folding rods 12 are fixedly connected to the fuselage 1. One end of each folding rod 12 is fixedly connected to the protective frame 13, and the power module 14 is fixedly connected to the protective frame 13. The fuselage 1 is the main body of the drone, the three-axis stabilization gimbal 11 is used to support the equipment, the folding rods 12 are used to enable the folding of the protective frame 13, the protective frame 13 is used to protect the power module 14, and the power module 14 is used to provide power to the drone. The power module 14 includes a motor and propellers, and the motor is fixedly connected to the protective frame 13. The output end of the motor is fixedly connected to the propellers, and the propellers are disposed inside the protective frame 13. The motor is used to drive the propellers.

[0033] The system for a mountain search and rescue drone based on visual recognition and voice guidance includes a main control module 2, an execution module 3, a data acquisition module 4, a data storage module 5, a wireless communication module 6, a power supply module 7, and a ground terminal 8. The ground terminal 8 establishes a data connection with the main control module 2 through the wireless communication module 6. The main control module 2 establishes electrical connections with the execution module 3, data acquisition module 4, data storage module 5, and power supply module 7 respectively. The main control module 2 coordinates the collaborative operation of each module and completes computational and control tasks. The execution module 3 executes the commands issued by the main control module 2. The data acquisition module 4 collects information and achieves environmental perception. The data storage module 5 stores data. The wireless communication module 6 enables wireless data transmission. The power supply module 7 powers the drone. The ground terminal 8 remotely commands and controls the drone. The main control module 2 includes a data preprocessing unit 21, a scene mapping unit 22, a navigation and obstacle avoidance unit 23, and an anomaly detection unit 24. 4. The power-on self-test and fault monitoring unit 25 and the voice guidance unit 26, the data preprocessing unit 21 for standardizing and preprocessing the collected raw data, the scene mapping unit 22 for constructing a two-dimensional grid map of the environment based on the preprocessed lidar point cloud data, and optimizing and completing the grid map by fusing visible light images, infrared thermal imaging data and IMU attitude data, the navigation and obstacle avoidance unit 23 for realizing the functions of UAV positioning, route planning, dynamic obstacle avoidance and stable flight, the anomaly detection unit 24 for target recognition and risk scene detection based on the preprocessed visible light images, infrared thermal imaging data and lidar point cloud data, the power-on self-test and fault monitoring unit 25 for monitoring the equipment status and safety protection of the UAV throughout the entire operation cycle, and the voice guidance unit 26 for two-way voice interaction control between the UAV and the stranded personnel; the scene mapping unit 22 uses the Cartographer algorithm to complete the construction of the two-dimensional grid map, which can be expressed by the following formula:

[0034]

[0035] in, To match the optimal pose, For the two-dimensional pose of the drone, Here is the pose transformation matrix. For lidar One scan point, This represents the probability of sub-map grid occupancy. The effective number of scan points; the navigation and obstacle avoidance unit 23 includes a positioning calculation subunit 231, a path planning subunit 232, and an attitude control subunit 233. The positioning calculation subunit 231 is used to locate the UAV, the path planning subunit 232 is used to realize the global planning of the UAV's flight path and local dynamic obstacle avoidance, and the attitude control subunit 233 is used to generate control commands for the UAV's flight attitude and gimbal attitude. The positioning calculation subunit 231 uses an extended Kalman filter to realize odometry calculation and an adaptive Monte Carlo positioning algorithm to realize the UAV's self-localization. The particle weight update formula is as follows:

[0036]

[0037] in, for Time of the first The weight of each particle; for Real-time lidar single-frame scan of point cloud; For the first Candidate poses of each particle; To pre-build a two-dimensional raster map of the mountainous area; To observe the likelihood probability;

[0038] The formula for the observation likelihood probability is as follows:

[0039]

[0040] in, This refers to the number of effective scan points in a single frame of the lidar. For the first The Euclidean distance from each scan point to the nearest obstacle grid cell in the map; To measure the standard deviation of noise;

[0041] Attitude control subunit 233 uses a PID algorithm to achieve attitude control, which can be expressed by the following formula:

[0042]

[0043] in, For motor control output, This refers to the deviation between the target angle and the actual angle of the gimbal. For proportionality coefficient, For integral coefficients, These are the differential coefficients. To control the cycle; Execution module 3 includes a motor ESC unit 31, a gimbal drive unit 32, and a voice output unit 33. The motor ESC unit 31 is used to control the power output of the UAV, the gimbal drive unit 32 is used to control the gimbal attitude, and the voice output unit 33 is used to play voice information; Data acquisition module 4 includes a lidar unit 41, an IMU inertial measurement unit 42, a visible light camera unit 43, an infrared thermal imaging unit 44, a voice acquisition unit 45, and a positioning unit 46. The lidar unit 41 is used to acquire three-dimensional point cloud data of the environment, the IMU inertial measurement unit 42 is used to acquire the IMU attitude data of the UAV, the visible light camera unit 43 is used to acquire visible light visual information of the environment, the infrared thermal imaging unit 44 is used to acquire infrared thermal imaging data of the environment, the voice acquisition unit 45 is used to acquire voice signals in the environment, and the positioning unit 46 is used for satellite positioning.

[0044] Working Principle: When using this invention for mountain search and rescue, after the system is powered on, the power-on self-test and fault monitoring unit 25 under the main control module 2 completes the initialization of the whole machine hardware driver and the self-test of the status of all modules. Combined with the independent watchdog mechanism, it establishes full-cycle monitoring. After the self-test passes, the system enters the standby state. If a communication, power, or sensor fault is detected, the safety protection mechanism is immediately triggered, and the fault information is reported to the ground terminal 8 through the wireless communication module 6. In the standby state, the power supply module 7 uses a four-electric dual-group power supply architecture with two batteries as a group and two groups connected in parallel, combined with a three-level step-down power management scheme, to output 48V power bus, 24V task load bus, and 12V / 5V logic bus respectively. The system employs a three-tiered, graded isolated power supply to provide stable power to the main unit, payload equipment, and logic circuits. Simultaneously, the data acquisition module 4 is activated. This module includes a lidar unit 41, an IMU inertial measurement unit 42, a visible light camera unit 43, an infrared thermal imaging unit 44, a voice acquisition unit 45, and a positioning unit 46. These units respectively acquire real-time environmental point cloud data of the search and rescue area, UAV attitude and motion data, high-definition visible light images, infrared thermal source data, on-site voice data, and satellite positioning data. All raw data is transmitted to the data preprocessing unit 21 of the main control module 2. Upon receiving the multi-source raw data, the data preprocessing unit 21 first synchronizes the timestamps of all data. The point cloud data from the lidar unit 41 and the inertial data from the IMU inertial measurement unit 42 are then denoised. First-order low-pass filtering and one-dimensional Kalman filtering are applied to key signals such as angle, angular velocity, and current to suppress measurement fluctuations caused by mechanical vibration and electromagnetic interference. Distortion correction, format normalization, and feature extraction are performed on the image data from the visible light camera unit 43 and the infrared thermal imaging unit 44. Noise reduction and gain optimization are performed on the audio data from the voice acquisition unit 45. The pre-processed data are then transmitted to the scene mapping unit 22, the navigation and obstacle avoidance unit 23, and the anomaly detection unit 24, respectively. Upon receiving the pre-processed lidar point cloud data, the scene mapping unit 22 uses Cart... The ographer algorithm completes pure data mapping of the LiDAR, generating a two-dimensional grid map that accurately presents key information such as terrain trends, obstacle distribution, and dangerous area boundaries in the search and rescue area. It simultaneously integrates visible light images, infrared thermal imaging data, and IMU attitude data to optimize and complete the grid map. The positioning calculation subunit 231 of the navigation and obstacle avoidance unit 23 combines the preprocessed IMU and point cloud data, completes high-precision odometry calculation through extended Kalman filtering, and then combines it with the two-dimensional grid map generated by the scene mapping unit 22. Using the adaptive Monte Carlo positioning algorithm, it achieves global high-precision positioning in complex mountainous environments and outputs the UAV's precise coordinates, attitude angles, and heading information in real time.The path planning subunit 232, based on grid maps, real-time positioning data, and search and rescue commands issued by the ground terminal 8, uses the Nav2 navigation framework to complete the globally optimal cruise route planning covering the designated search and rescue area, enabling autonomous search and rescue patrols in designated mountainous areas on a fixed time and route. Simultaneously, it combines real-time environmental point cloud data collected by the lidar unit 41 to complete local dynamic obstacle avoidance and route replanning, adjusting the flight path in real time for sudden obstacles and dangerous terrain encountered during the search and rescue process. The attitude control subunit 233, based on real-time attitude data, uses a PID algorithm to complete closed-loop control of the UAV's flight attitude, achieving smooth control of the UAV's hovering, cruise, turning, climbing, and obstacle avoidance maneuvers. The main control module 2 integrates flight and... The gimbal control command is sent to the execution module 3, and the motor ESC unit 31 controls the motor of the power module 14. The gimbal drive unit 32 adjusts the roll axis, pitch axis, and yaw axis of the three-axis stabilized gimbal 11 according to the gimbal control command. During the cruise, the anomaly detection unit 24 uses a multi-modal fusion recognition algorithm based on pre-processed visible light images, infrared thermal imaging data, and lidar point cloud data to detect and identify trapped personnel and potential hazards such as landslides and fires in real time. After accurately locating the abnormal target, it automatically triggers the alarm mechanism and transmits the image, coordinates, and feature information of the abnormal target back to the ground terminal 8 in real time through the wireless communication module 6. At the same time, it links with the voice guidance unit 26 to start directional rescue guidance operations. The voice output unit 33 of the control execution module 3, controlled by the voice guidance unit 26, provides directional rescue guidance and emotional reassurance. It collects voice feedback from trapped personnel through the voice acquisition unit 45, and after noise reduction and optimization by the data preprocessing unit 21, transmits it in real-time to the ground terminal 8 via the wireless communication module 6, achieving two-way voice interaction and forming a complete search and rescue closed loop of positioning, voice communication, and guidance. Throughout the entire operation, the wireless communication module 6 maintains two-way real-time data interaction between the UAV and the ground terminal 8. The uplink continuously transmits the UAV's flight status data, real-time search and rescue images, location information, abnormal alarm data, and on-site voice data back to the ground terminal 8, allowing rescue personnel to have a comprehensive understanding of the search and rescue situation and make emergency decisions. The downlink... The system receives real-time instructions from the ground terminal 8, including mission planning, flight control, voice broadcasting, and parameter adjustment. The data storage module 5 completes local storage and offline caching of all operational data. The power-on self-test and fault monitoring unit 25 continuously monitors the overall operating status. If a sudden fault such as communication interruption, power abnormality, sensor failure, or program overrun is detected, the safety protection mechanism will be immediately triggered, automatically switching to a safe mode. This mode ensures the safety of the UAV and its onboard equipment by limiting the power output of the power system, locking the gimbal attitude, and initiating automatic return. At the same time, the fault details are reported to the ground terminal 8. The main control board of the main control module 2 integrates an edge computing unit, and all algorithms used are implemented based on the edge computing unit.The hinge of the folding rod 12 is designed with a quick-release locking structure. After the folding rod 12 is unfolded, its end forms an upward angle of 5° with the fuselage 1, causing the power module 14 on the protective frame 13 to be tilted upwards. This utilizes aerodynamics to generate an automatic straightening torque, counteracting the tilting and offset interference of the fuselage 1 caused by mountain gusts and turbulence. The main control module 2, execution module 3, data storage module 5, wireless communication module 6, and power supply module 7 are all deployed on the fuselage 1. The IMU inertial measurement unit 42 and positioning unit 46 are deployed on the fuselage 1. The lidar unit 41, visible light camera unit 43, infrared thermal imaging unit 44, voice acquisition unit 45, and voice output unit 33 are deployed on the three-axis stabilization gimbal 11.

[0045] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A mountain search and rescue drone based on visual recognition and voice guidance, comprising a fuselage (1), a three-axis stabilization gimbal (11), a folding pole (12), a protective frame (13), and a power module (14), characterized in that: The bottom of the fuselage (1) is fixedly connected to a three-axis stabilization gimbal (11), and multiple folding rods (12) are fixedly connected to the fuselage (1). One end of the folding rod (12) is fixedly connected to a protective frame (13), and a power module (14) is fixedly connected to the protective frame (13).

2. The mountain search and rescue drone based on visual recognition and voice guidance according to claim 1, characterized in that: The power module (14) includes a motor and a blade, and the motor is fixedly connected to the protective frame (13). The output end of the motor is fixedly connected to the blade, and the blade is set inside the protective frame (13).

3. A system for mountain search and rescue drones based on visual recognition and voice guidance, comprising a main control module (2), an execution module (3), a data acquisition module (4), a data storage module (5), a wireless communication module (6), a power supply module (7), and a ground terminal (8), characterized in that: The ground terminal (8) establishes a data connection with the main control module (2) through the wireless communication module (6). The main control module (2) establishes electrical connections with the execution module (3), the data acquisition module (4), the data storage module (5), and the power supply module (7) respectively. The main control module (2) is used to coordinate the collaborative operation of each module and complete the calculation and control tasks. The execution module (3) is used to execute the instructions issued by the main control module (2). The data acquisition module (4) is used to collect information and realize environmental perception. The data storage module (5) is used to store data. The wireless communication module (6) is used to realize wireless data transmission. The power supply module (7) is used to power the UAV. The ground terminal (8) is used to remotely command and control the UAV.

4. The system for mountain search and rescue drones based on visual recognition and verbal guidance according to claim 3, characterized in that: The main control module (2) includes a data preprocessing unit (21), a scene mapping unit (22), a navigation and obstacle avoidance unit (23), an anomaly detection unit (24), a power-on self-test and fault monitoring unit (25), and a voice guidance unit (26). The data preprocessing unit (21) is used to standardize and preprocess the collected raw data. The scene mapping unit (22) constructs a two-dimensional grid map of the environment based on the preprocessed lidar point cloud data and optimizes and completes the grid map by integrating visible light images, infrared thermal imaging data, and IMU attitude data. The navigation and obstacle avoidance unit (23) is used to realize the functions of UAV positioning, route planning, dynamic obstacle avoidance, and stable flight. The anomaly detection unit (24) performs target recognition and risk scene detection based on the preprocessed visible light images, infrared thermal imaging data, and lidar point cloud data. The power-on self-test and fault monitoring unit (25) is responsible for monitoring the equipment status and safety protection of the UAV throughout the entire operation cycle. The voice guidance unit (26) is responsible for two-way voice interaction control between the UAV and the trapped personnel.

5. The system for mountain search and rescue drones based on visual recognition and verbal guidance according to claim 4, characterized in that: The scene mapping unit (22) uses the Cartographer algorithm to construct a two-dimensional raster map, which can be expressed by the following formula: in, To match the optimal pose, For the two-dimensional pose of the drone, Here is the pose transformation matrix. For lidar One scan point, This represents the probability of sub-map grid occupancy. This represents the number of valid scan points.

6. The system for mountain search and rescue drones based on visual recognition and verbal guidance according to claim 4, characterized in that: The navigation and obstacle avoidance unit (23) includes a positioning calculation subunit (231), a path planning subunit (232), and an attitude control subunit (233). The positioning calculation subunit (231) is used to locate the UAV, the path planning subunit (232) is used to realize the global planning of the UAV route and local dynamic obstacle avoidance, and the attitude control subunit (233) is used to generate control commands for the UAV flight attitude and gimbal attitude.

7. The system for mountain search and rescue drones based on visual recognition and verbal guidance according to claim 6, characterized in that: The positioning solution subunit (231) uses extended Kalman filtering to perform odometry calculation and adaptive Monte Carlo positioning algorithm to achieve UAV self-localization. The particle weight update formula is as follows: in, for Time of the first The weight of each particle; for Real-time lidar single-frame scan of point cloud; For the first Candidate poses of each particle; To pre-build a two-dimensional raster map of the mountainous area; To observe the likelihood probability; The formula for the observation likelihood probability is as follows: in, This refers to the number of effective scan points in a single frame of the lidar. For the first The Euclidean distance from each scan point to the nearest obstacle grid cell in the map; To measure the standard deviation of noise.

8. The system for mountain search and rescue drones based on visual recognition and verbal guidance according to claim 6, characterized in that: The attitude control subunit (233) uses a PID algorithm to achieve attitude control, which can be expressed by the following formula: in, For motor control output, This refers to the deviation between the target angle and the actual angle of the gimbal. For proportionality coefficient, For integral coefficients, These are the differential coefficients. To control the cycle.

9. The system for mountain search and rescue drones based on visual recognition and verbal guidance according to claim 3, characterized in that: The execution module (3) includes a motor ESC unit (31), a gimbal drive unit (32), and a voice output unit (33). The motor ESC unit (31) is used to control the power output of the UAV, the gimbal drive unit (32) is used to control the gimbal attitude, and the voice output unit (33) is used to play voice information.

10. The system for mountain search and rescue drones based on visual recognition and verbal guidance according to claim 3, characterized in that: The data acquisition module (4) includes a lidar unit (41), an IMU inertial measurement unit (42), a visible light camera unit (43), an infrared thermal imaging unit (44), a voice acquisition unit (45), and a positioning unit (46). The lidar unit (41) is used to acquire three-dimensional point cloud data of the environment, the IMU inertial measurement unit (42) is used to acquire IMU attitude data of the UAV, the visible light camera unit (43) is used to acquire visible light visual information of the environment, the infrared thermal imaging unit (44) is used to acquire infrared thermal imaging data of the environment, the voice acquisition unit (45) is used to acquire voice signals in the environment, and the positioning unit (46) is used for satellite positioning.

Images