Multi-mode unmanned aerial vehicle integrated shooting system

Abstract

The invention relates to the technical field of integrated shooting systems, in particular to a multi-mode unmanned aerial vehicle integrated shooting system which comprises a multi-mode sensing module, an environment sensing and cooperative control unit, a data storage and transmission module, a dynamic path planning module, a central cooperative control unit and a multi-mode shooting unit. The multi-modal sensing module comprises a sensor group, an inertial measurement unit and a data fusion engine; the sensor group integrates a visible light camera, an infrared thermal imager, an ultraviolet imager, a laser radar and a millimeter-wave radar and covers different frequency bands and environmental conditions, the visible light camera clearly captures details of pole and tower hardware, the infrared thermal imager detects overheating hidden dangers of key parts of a wire joint and an insulator, the ultraviolet imager detects a corona discharge phenomenon, and the laser radar covers the laser radar. Positioning hidden dangers of insulator contamination and equipment aging; the system improves the power grid inspection efficiency, achieves the precise collection and fusion of all-scene data, and provides a high-precision, high-reliability and low-cost solution for the intelligent inspection of a power grid.

Description

Technical Field

[0001] This invention relates to the technical field of integrated shooting systems, and in particular to a multimodal unmanned aerial vehicle (UAV) integrated shooting system. Background Technology

[0002] As the world's largest public utility, State Grid Corporation of China bears the core mission of ensuring national energy security and electricity supply for residents. However, traditional transmission line inspection methods face many challenges. Manual inspection requires climbing towers or traversing complex terrain on foot, and is severely limited by weather and terrain conditions. It is difficult to detect minute defects with the naked eye or simple tools. To address these challenges, equipment such as visible light cameras and infrared thermal imagers is used to achieve rapid line inspection and preliminary defect identification. However, existing drone systems still have the following limitations: Limited sensing capabilities: Relying on a single sensor makes it difficult to cover the needs of all scenarios; Insufficient intelligent decision-making: Path planning relies on preset routes and cannot dynamically avoid unexpected obstacles; Data fusion is lagging: multi-source data lacks spatiotemporal synchronization and deep fusion, and defect analysis still requires manual intervention.

[0003] For example, the patent with the existing technology announcement number CN106791634A discloses a method and device for taking panoramic images by a drone based on opto-mechatronics technology. The device includes a central processing unit, the central processing unit is electrically connected to a display unit, and the data acquisition system includes a panoramic shooting data acquisition subsystem and a drone flight data acquisition subsystem.

[0004] Existing technologies have been found to have issues with limited data collection and path planning that relies on pre-defined routes, making it impossible to dynamically avoid unexpected obstacles. Summary of the Invention

[0005] To address the aforementioned technical issues, this invention provides a multimodal drone integrated shooting system that enhances power grid inspection efficiency, enables accurate data collection and fusion across all scenarios, and offers a high-precision, high-reliability, and low-cost solution for intelligent power grid inspection.

[0006] The present invention provides an integrated multimodal drone shooting system, comprising a multimodal perception module, an environmental perception and collaborative control unit, a data storage and transmission module, a dynamic path planning module, a central collaborative control unit, and a multimodal shooting unit; The multimodal sensing module includes a sensor array, an inertial measurement unit, and a data fusion engine; Sensor group: Integrates visible light camera, infrared thermal imager, ultraviolet imager, lidar, and millimeter-wave radar, covering different frequency bands and environmental conditions. The visible light camera clearly captures the details of tower hardware, the infrared thermal imager detects overheating hazards in conductor joints and key parts of insulators, and the ultraviolet imager detects corona discharge phenomena, locates insulator contamination and equipment aging hazards, thereby generating multi-source data. Inertial Measurement Unit: Integrates a three-axis gyroscope and a three-axis accelerometer to measure the three-axis angular velocity and three-axis acceleration of the UAV in real time. It calculates the pitch angle, roll angle and yaw angle of the UAV through integral calculation, providing an attitude reference for the flight control of the UAV. Data fusion engine: It adopts a feature-level fusion algorithm, combines lidar point cloud and visible light image, automatically labels tower number, phase identifier, and equipment type, and fuses infrared thermal imaging and ultraviolet imaging data to generate a comprehensive temperature discharge defect map. Environmental perception and collaborative control unit: Based on synchronous positioning and mapping technology, combined with data from multimodal perception modules, a real-time environmental map is constructed, marking static and dynamic obstacles; Data storage and transmission module: Supports synchronous storage and compressed transmission of multimodal data; Dynamic path planning module: Based on the real-time environmental map and marking of static and dynamic obstacles constructed by the environmental perception and collaborative control unit, it continuously monitors the movement trajectory of obstacles through millimeter-wave radar, dynamically adjusts the path planning to avoid sudden obstacles, and automatically generates and dynamically optimizes the three-dimensional flight path of the UAV according to the input mission objectives and geographical constraints. Central Coordination and Control Unit: Connected to the multimodal perception module and dynamic path planning module, based on the three-dimensional flight path and mission objectives, it calculates and synchronously controls the UAV's flight attitude and the sensor group's shooting parameters, angles and triggering timings in real time, generates control commands and issues them; Multimodal imaging unit: It combines drones and gimbals to carry visible light cameras, infrared thermal imagers, ultraviolet imagers, lidar, and millimeter-wave radar.

[0007] Preferably, the dynamic path planning module includes a task parsing unit, a path generation unit, a dynamic optimization unit, and an energy consumption prediction submodule; Task parsing unit: used to decompose user-defined task objectives into specific waypoints, coverage areas and resolution requirements, cover all towers according to preset routes, focus on detecting joint temperature and insulator condition, and quickly locate fault points according to dispatch instructions; Path generation unit: A three-dimensional spatial path planning algorithm based on B-spline curves is adopted, taking into account the constraints of terrain undulation, obstacle avoidance, sensor field of view and UAV endurance. It prioritizes flying along the power transmission line corridor, reduces lateral deviation to reduce wind drift risk, and generates an initial smooth path. Dynamic optimization unit: Based on real-time acquired lidar point cloud data and visual information, it performs online local replanning of the flight path to avoid unforeseen dynamic or static obstacles; Energy consumption prediction submodule: Calculates remaining range in real time based on battery discharge curve and motor efficiency graph, and dynamically adjusts the number of waypoints when generating the route.

[0008] Preferably, the central coordination control unit executes the following coordination logic: S1. When flying to the preset area, the center point of the tower top is identified by the laser radar, and the flight attitude is automatically adjusted so that the camera is vertically aligned with the target. When shooting the insulator string, the gimbal is controlled to rotate horizontally to cover all the porcelain insulators. S2. When the visible light camera is shooting at high speed, the infrared thermal imager is simultaneously triggered to record temperature data. When the lidar scans the three-dimensional structure of the tower, the ultraviolet imager detects the corona discharge point. S3. When switching between tasks, adjust the gimbal angle in advance to smoothly transition to the next shooting posture.

[0009] Preferably, the central collaborative control unit is connected to the inertial measurement unit. By coordinating the wind speed monitored by the millimeter-wave radar with the data from the inertial measurement unit, the inertial measurement unit calculates the UAV attitude angle in real time and compensates for high-frequency vibrations, complementing the low-frequency motion monitoring of the millimeter-wave radar, and dynamically adjusting the flight attitude to reduce image jitter.

[0010] Preferably, the multimodal sensing module further includes a sensor spatiotemporal synchronization calibration module; Sensor spatiotemporal synchronization calibration module: It adopts the PTP protocol to achieve hardware-level time synchronization of visible light cameras, infrared thermal imagers, lidar and millimeter-wave radar, and calibrates the spatial coordinate system of each sensor through a laser interferometer to ensure spatiotemporal consistency before multi-source data fusion.

[0011] Preferably, the multimodal shooting unit includes a fixing device, an unmanned aerial vehicle body, a gimbal, an integrated housing, a transparent window, a square plug, and a positioning slot; The gimbal is mounted on the outer wall of the drone. The integrated housing houses the aforementioned visible light camera, infrared thermal imager, ultraviolet imager, lidar, and millimeter-wave radar. The transparent window is located on the outer wall of the integrated housing; The square insert is located at the rear end of the integrated housing; Multiple positioning slots are set on the outer wall of the square plug; The fixing device is installed on the moving end of the gimbal. It uses square inserts and multiple positioning slots to quickly secure the integrated housing. A visible light camera, infrared thermal imager, ultraviolet imager, lidar, and millimeter-wave radar are integrated inside the housing. The housing is then mounted on the fixing device using the square inserts and positioning slots. The unmanned aerial vehicle (UAV) moves the gimbal, simultaneously adjusting the angle of the integrated housing to achieve multi-source data acquisition. When loading or unloading the integrated housing is required, the fixing device is controlled and adjusted to enable rapid loading and unloading, improving the unit's maintenance efficiency and operational convenience.

[0012] Preferably, the fixing device includes a positioning component, a fixing plate, a limiting component, a connecting cylinder, a turntable, a locking block, a connecting column, and a guide groove; The top of the fixed plate is connected to the moving end of the gimbal, and a square hole is provided in the middle of the fixed plate, through which a square insert rod passes. Multiple sets of limiting components are set on the front part of the outer wall of the fixed plate, and the rear end of the integrated housing is embedded between the multiple sets of limiting components; The connecting cylinder is located on the rear part of the outer wall of the fixed plate, and a positioning hole is provided on the connecting cylinder; The turntable is mounted on the connecting cylinder in a rotating manner; Multiple sets of locking blocks are slidably installed on the outer wall of the fixed plate; Multiple sets of connecting posts are respectively installed on the outer walls of multiple sets of locking blocks; Multiple sets of guide grooves are respectively set on the inner side wall of the turntable, and multiple sets of connecting columns are respectively slidably set in the multiple sets of guide grooves; The positioning components are installed on the connecting cylinder and the turntable. The positioning components cooperate with the positioning holes to fix the rotation of the turntable. The rear of the integrated housing is embedded between multiple sets of limiting members, and the square insert passes through the square hole of the fixing plate. At this time, the multiple sets of positioning slots move to the side of the multiple sets of locking blocks. By rotating the turntable, the turntable moves the multiple sets of connecting columns closer to each other through the multiple sets of guide slots. This causes the multiple sets of connecting columns to drive the multiple sets of locking blocks to slide and insert into the multiple sets of positioning slots, thereby fixing the position of the positioning slots and locking the integrated housing. When it is necessary to disassemble the integrated housing, the turntable is rotated in the opposite direction. This improves the convenience of loading and unloading the integrated housing and enhances the installation and positioning stability of the integrated housing.

[0013] Preferably, the positioning assembly includes a first sleeve, a first pin, a first spring, a second sleeve, a second pin, a second spring, and a pressing handle; The first sleeve is set on the outer wall of the turntable; The first pin is slidably mounted on the first sleeve; The first spring is located inside the first sleeve; The second sleeve is installed on the outer wall of the connecting sleeve; The second pin is slidably mounted on the second sleeve; The second spring is installed inside the second sleeve; The pressing handle is located at the top of the second pin. After the turntable rotates, it drives the first sleeve to move circumferentially. The circumferential movement of the first sleeve drives the first pin to move. When the first pin moves to the through hole position of the connecting cylinder, the first spring pushes the first pin, causing the first pin to be inserted into the through hole to achieve rotational locking of the turntable, thereby improving the fixing reliability of the integrated housing. When it is necessary to rotate the turntable, the pressing handle is pressed down, which drives the second pin to be inserted into the through hole and pushes the first pin down, thereby separating the first pin from the through hole, thus facilitating the rotation of the turntable and improving the convenience of operation.

[0014] Preferably, it also includes magnets; Magnets are placed on the outer wall of the turntable; the rear end of the square plug is attracted by the magnets, which improves the positioning effect of the integrated housing pre-installed on the fixing plate.

[0015] Preferably, it also includes anti-slip texture; Anti-slip textures are applied to the outer wall of the turntable; these textures improve the ease of operation of the turntable.

[0016] Compared with existing technologies, the beneficial effects of this invention are as follows: through multimodal perception and intelligent collaborative control technology, the efficiency of power grid inspection is significantly improved. The multimodal perception module integrates multiple types of sensors and, combined with spatiotemporal synchronous calibration, achieves accurate data acquisition and fusion across all scenarios. The multimodal imaging unit and collaborative logic support seamless switching between multiple tasks. A single sortie can complete four types of detection: visible light, infrared, ultraviolet, and lidar. This provides a high-precision, high-reliability, and low-cost solution for intelligent power grid inspection. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the isometric structure of the present invention; Figure 2 This is a schematic diagram of the structure of the multimodal sensing module; Figure 3 This is a structural diagram of the dynamic path planning module; Figure 4 This is a partial isometric structural diagram of the connection between the drone body and the gimbal, etc. Figure 5 This is a partial isometric structural diagram of the integrated housing and the connection between the positioning groove, etc. Figure 6 This is a partial isometric structural diagram of the connection between the fixed plate and the connecting cylinder, etc. Figure 7 This is a schematic diagram of the turntable and its anti-slip texture. Figure 8 This is a partial isometric structural diagram of the connection between the turntable and magnets, etc. Figure 9 This is a partial isometric structural diagram of the connection between the turntable and the first sleeve, etc. Figure 10 This is a partial isometric structural diagram of the connection between the connecting sleeve and the second sleeve, etc. Figure 11 It is a schematic diagram of the integrated shell and fixing plate structure.

[0018] The attached diagram is labeled as follows: 101, Unmanned Aerial Vehicle (UAV) body; 102, Gimbal; 103, Integrated housing; 104, Transparent window; 105, Square insert rod; 106, Positioning groove; 201, Fixing plate; 202, Limiting component; 203, Connecting cylinder; 204, Turntable; 205, Locking block; 206, Connecting column; 207, Guide groove; 301, First sleeve; 302, First pin; 303, First spring; 304, Second sleeve; 305, Second pin; 306, Second spring; 307, Press handle; 401, Magnet; 501, Anti-slip texture. Detailed Implementation

[0019] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. The present invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0020] Example 1 like Figures 1 to 3 As shown, the multimodal unmanned aerial vehicle integrated shooting system of the present invention includes a multimodal perception module, an environmental perception and collaborative control unit, a data storage and transmission module, a dynamic path planning module, a central collaborative control unit, and a multimodal shooting unit; The multimodal sensing module includes a sensor array, an inertial measurement unit, and a data fusion engine; Sensor group: Integrates visible light camera, infrared thermal imager, ultraviolet imager, lidar, and millimeter-wave radar, covering different frequency bands and environmental conditions. The visible light camera clearly captures the details of tower hardware, the infrared thermal imager detects overheating hazards in conductor joints and key parts of insulators, and the ultraviolet imager detects corona discharge phenomena, locates insulator contamination and equipment aging hazards, thereby generating multi-source data. Inertial Measurement Unit: Integrates a three-axis gyroscope and a three-axis accelerometer to measure the three-axis angular velocity and three-axis acceleration of the UAV in real time. It calculates the pitch angle, roll angle and yaw angle of the UAV through integral calculation, providing an attitude reference for the flight control of the UAV. Data fusion engine: It adopts a feature-level fusion algorithm, combines lidar point cloud and visible light image, automatically labels tower number, phase identifier, and equipment type, and fuses infrared thermal imaging and ultraviolet imaging data to generate a comprehensive temperature discharge defect map. Environmental perception and collaborative control unit: Based on synchronous positioning and mapping technology, combined with data from multimodal perception modules, a real-time environmental map is constructed, marking static and dynamic obstacles; Data storage and transmission module: Supports synchronous storage and compressed transmission of multimodal data; Dynamic path planning module: Based on the real-time environmental map and marking of static and dynamic obstacles constructed by the environmental perception and collaborative control unit, it continuously monitors the movement trajectory of obstacles through millimeter-wave radar, dynamically adjusts the path planning to avoid sudden obstacles, and automatically generates and dynamically optimizes the three-dimensional flight path of the UAV according to the input mission objectives and geographical constraints. Central Coordination and Control Unit: Connected to the multimodal perception module and dynamic path planning module, based on the three-dimensional flight path and mission objectives, it calculates and synchronously controls the UAV's flight attitude and the sensor group's shooting parameters, angles and triggering timings in real time, generates control commands and issues them; Multimodal imaging unit: It combines drones and gimbals to mount visible light cameras, infrared thermal imagers, ultraviolet imagers, lidar, and millimeter-wave radar; The dynamic path planning module includes a task parsing unit, a path generation unit, a dynamic optimization unit, and an energy consumption prediction submodule. Task parsing unit: used to decompose user-defined task objectives into specific waypoints, coverage areas and resolution requirements, cover all towers according to preset routes, focus on detecting joint temperature and insulator condition, and quickly locate fault points according to dispatch instructions; Path generation unit: A three-dimensional spatial path planning algorithm based on B-spline curves is adopted, taking into account the constraints of terrain undulation, obstacle avoidance, sensor field of view and UAV endurance. It prioritizes flying along the power transmission line corridor, reduces lateral deviation to reduce wind drift risk, and generates an initial smooth path. Dynamic optimization unit: Based on real-time acquired lidar point cloud data and visual information, it performs online local replanning of the flight path to avoid unforeseen dynamic or static obstacles; Energy consumption prediction submodule: Calculates remaining range in real time based on battery discharge curve and motor efficiency graph, and dynamically adjusts the number of waypoints when generating the route; In this embodiment, the efficiency of power grid inspection is significantly improved by using multimodal sensing and intelligent collaborative control technology. The multimodal sensing module integrates multiple types of sensors and combines spatiotemporal synchronous calibration to achieve accurate data acquisition and fusion across all scenarios. The multimodal imaging unit and collaborative logic support seamless switching between multiple tasks. A single flight can complete four types of detection: visible light, infrared, ultraviolet, and lidar. This provides a high-precision, high-reliability, and low-cost solution for intelligent power grid inspection.

[0021] Example 2 Based on Embodiment 1, the present invention provides a multimodal unmanned aerial vehicle (UAV) integrated shooting system, wherein the central collaborative control unit executes the following collaborative logic: S1. When flying to the preset area, the center point of the tower top is identified by the laser radar, and the flight attitude is automatically adjusted so that the camera is vertically aligned with the target. When shooting the insulator string, the gimbal is controlled to rotate horizontally to cover all the porcelain insulators. S2. When the visible light camera is shooting at high speed, the infrared thermal imager is simultaneously triggered to record temperature data. When the lidar scans the three-dimensional structure of the tower, the ultraviolet imager detects the corona discharge point. S3. When switching between tasks, adjust the gimbal angle in advance to smoothly transition to the next shooting posture; The central collaborative control unit is connected to the inertial measurement unit. By coordinating the wind speed monitored by the millimeter-wave radar with the data from the inertial measurement unit, the inertial measurement unit calculates the UAV's attitude angle in real time and compensates for high-frequency vibrations. This complements the low-frequency motion monitoring of the millimeter-wave radar, and dynamically adjusts the flight attitude to reduce image jitter. The multimodal sensing module also includes a sensor spatiotemporal synchronization calibration module; Sensor spatiotemporal synchronization calibration module: It adopts the PTP protocol to achieve hardware-level time synchronization of visible light cameras, infrared thermal imagers, lidar and millimeter-wave radar, and calibrates the spatial coordinate system of each sensor through a laser interferometer to ensure spatiotemporal consistency before multi-source data fusion.

[0022] Example 3 Based on Example 1, the present invention provides a multimodal unmanned aerial vehicle (UAV) integrated shooting system, such as... Figures 4 to 11 As shown, the multimodal shooting unit includes a fixing device, an unmanned aerial vehicle body 101, a gimbal 102, an integrated housing 103, a transparent window 104, a square insert rod 105, and a positioning groove 106; The gimbal 102 is mounted on the outer wall of the unmanned aerial vehicle body 101; The visible light camera, infrared thermal imager, ultraviolet imager, lidar, and millimeter-wave radar are installed inside the integrated housing 103. The transparent window 104 is provided on the outer wall of the integrated housing 103; The square insertion rod 105 is located at the rear end of the integrated housing 103; Multiple positioning grooves 106 are all set on the outer wall of the square insert rod 105; The fixing device is set on the moving end of the gimbal 102. The fixing device is used to quickly fix the integrated housing 103 using the square plug rod 105 and multiple sets of positioning slots 106. The fixing device includes a positioning component, a fixing plate 201, a limiting component 202, a connecting cylinder 203, a turntable 204, a locking block 205, a connecting column 206, and a guide groove 207; The top of the fixed plate 201 is connected to the movable end of the gimbal 102. A square hole is provided in the middle of the fixed plate 201, and the square insertion rod 105 passes through the square hole. Multiple sets of limiting components 202 are all provided on the front part of the outer side wall of the fixed plate 201, and the rear end of the integrated housing 103 is embedded between the multiple sets of limiting components 202. The connecting cylinder 203 is located on the rear part of the outer wall of the fixing plate 201, and the connecting cylinder 203 is provided with positioning holes; Turntable 204 is rotatably mounted on connecting cylinder 203; Multiple sets of locking blocks 205 are slidably installed on the outer wall of the fixing plate 201; Multiple sets of connecting posts 206 are respectively installed on the outer walls of multiple sets of locking blocks 205; Multiple sets of guide grooves 207 are respectively arranged on the inner side wall of the turntable 204, and multiple sets of connecting columns 206 are respectively slidably arranged in the multiple sets of guide grooves 207; The positioning components are installed on the connecting cylinder 203 and the turntable 204. The positioning components cooperate with the positioning holes to fix the rotation of the turntable 204. The positioning assembly includes a first sleeve 301, a first pin 302, a first spring 303, a second sleeve 304, a second pin 305, a second spring 306, and a pressing handle 307. The first sleeve 301 is disposed on the outer wall of the turntable 204; The first pin 302 is slidably mounted on the first sleeve 301; The first spring 303 is disposed inside the first sleeve 301; The second sleeve 304 is disposed on the outer wall of the connecting sleeve 203; The second pin 305 is slidably mounted on the second sleeve 304; The second spring 306 is installed inside the second sleeve 304; The pressing handle 307 is located at the top of the second pin 305; It also includes magnet 401; Magnet 401 is disposed on the outer wall of turntable 204; It also includes anti-slip texture 501; Anti-slip texture 501 is applied to the outer wall of turntable 204; In this embodiment, a visible light camera, an infrared thermal imager, an ultraviolet imager, a lidar, and a millimeter-wave radar are integrated and housed inside an integrated housing 103. The integrated housing 103 is then mounted on a fixed device via a square insert 105 and a positioning slot 106. The unmanned aerial vehicle 101 moves the gimbal 102, which in turn moves the integrated housing 103 to adjust its angle, enabling multi-source data acquisition. When it is necessary to load or unload the integrated housing 103, the fixed device is controlled and adjusted. This allows for quick loading and unloading of the integrated housing 103, improving the efficiency of unit maintenance and operation. The rear of the integrated housing 103 is embedded between multiple sets of limiting members 202, and the square insert 105 passes through the square hole in the fixing plate 201. At this time, multiple sets of positioning grooves 106 move to the sides of multiple sets of locking blocks 205. Personnel rotate the turntable 204, which, after rotation, moves multiple sets of connecting posts 206 closer together via multiple sets of guide grooves 207, thereby causing the multiple sets of connecting posts 206 to drive multiple sets of locking blocks 205. After sliding, block 205 is inserted into multiple positioning slots 106, fixing the position of the positioning slots 106 and locking the integrated housing 103. When it is necessary to disassemble the integrated housing 103, the turntable 204 can be rotated in the opposite direction, improving the convenience of loading and unloading the integrated housing 103 and enhancing the stability of the installation and positioning of the integrated housing 103. After the turntable 204 rotates, it drives the first sleeve 301 to move circumferentially. The circumferential movement of the first sleeve 301 drives the first pin 302 to move. When the first pin 302 moves to the connection... After the cylinder 203 is positioned in the through hole, the first spring 303 pushes the first pin 302, causing the first pin 302 to be inserted into the through hole to achieve rotational locking of the turntable 204, thereby improving the fixing reliability of the integrated housing 103. When it is necessary to rotate the turntable 204, the pressing handle 307 is pressed down, which drives the second pin 305 to be inserted into the through hole and pushes the first pin 302 down, causing the first pin 302 to separate from the through hole, thus facilitating the rotation of the turntable 204 and improving the convenience of operation.

[0023] The main functions achieved by this invention are: 1. By using multimodal sensing and intelligent collaborative control technology, the efficiency of power grid inspection is improved, and accurate data collection and fusion in all scenarios are achieved. The multimodal imaging unit and collaborative logic support seamless switching between multiple tasks, and a single sortie can complete four types of detection: visible light, infrared, ultraviolet, and lidar. 2. Dynamically adjust flight attitude to reduce image shake; 3. Enables rapid loading and unloading of the integrated housing 103, improving the efficiency of unit maintenance and repair, and enhancing operational convenience.

[0024] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A multimodal unmanned aerial vehicle (UAV) integrated shooting system, characterized in that, It includes a multimodal perception module, an environmental perception and collaborative control unit, a data storage and transmission module, a dynamic path planning module, a central collaborative control unit, and a multimodal camera unit; The multimodal sensing module includes a sensor array, an inertial measurement unit, and a data fusion engine; Sensor group: Integrates visible light camera, infrared thermal imager, ultraviolet imager, lidar, and millimeter-wave radar, covering different frequency bands and environmental conditions. The visible light camera clearly captures the details of tower hardware, the infrared thermal imager detects overheating hazards in conductor joints and key parts of insulators, and the ultraviolet imager detects corona discharge phenomena, locates insulator contamination and equipment aging hazards, thereby generating multi-source data. Inertial Measurement Unit: Integrates a three-axis gyroscope and a three-axis accelerometer to measure the three-axis angular velocity and three-axis acceleration of the UAV in real time. It calculates the pitch angle, roll angle and yaw angle of the UAV through integral calculation, providing an attitude reference for the flight control of the UAV. Data fusion engine: It adopts a feature-level fusion algorithm, combines lidar point cloud and visible light image, automatically labels tower number, phase identifier, and equipment type, and fuses infrared thermal imaging and ultraviolet imaging data to generate a comprehensive temperature discharge defect map. Environmental perception and collaborative control unit: Based on synchronous positioning and mapping technology, combined with data from multimodal perception modules, a real-time environmental map is constructed, marking static and dynamic obstacles; Data storage and transmission module: Supports synchronous storage and compressed transmission of multimodal data; Dynamic path planning module: Based on the real-time environmental map and marking of static and dynamic obstacles constructed by the environmental perception and collaborative control unit, it continuously monitors the movement trajectory of obstacles through millimeter-wave radar, dynamically adjusts the path planning to avoid sudden obstacles, and automatically generates and dynamically optimizes the three-dimensional flight path of the UAV according to the input mission objectives and geographical constraints. Central Coordination and Control Unit: Connected to the multimodal perception module and dynamic path planning module, based on the three-dimensional flight path and mission objectives, it calculates and synchronously controls the UAV's flight attitude and the sensor group's shooting parameters, angles and triggering timings in real time, generates control commands and issues them; Multimodal imaging unit: It combines drones and gimbals to carry visible light cameras, infrared thermal imagers, ultraviolet imagers, lidar, and millimeter-wave radar.

2. The multimodal unmanned aerial vehicle integrated shooting system as described in claim 1, characterized in that, The dynamic path planning module includes a task parsing unit, a path generation unit, a dynamic optimization unit, and an energy consumption prediction submodule. Task parsing unit: used to decompose user-defined task objectives into specific waypoints, coverage areas and resolution requirements, cover all towers according to preset routes, focus on detecting joint temperature and insulator condition, and quickly locate fault points according to dispatch instructions; Path generation unit: A three-dimensional spatial path planning algorithm based on B-spline curves is adopted, taking into account the constraints of terrain undulation, obstacle avoidance, sensor field of view and UAV endurance. It prioritizes flying along the power transmission line corridor, reduces lateral deviation to reduce wind drift risk, and generates an initial smooth path. Dynamic optimization unit: Based on real-time acquired lidar point cloud data and visual information, it performs online local replanning of the flight path to avoid unforeseen dynamic or static obstacles; Energy consumption prediction submodule: Calculates remaining range in real time based on battery discharge curve and motor efficiency graph, and dynamically adjusts the number of waypoints when generating the route.

3. The multimodal unmanned aerial vehicle integrated shooting system as described in claim 1, characterized in that, The central coordination control unit executes the following coordination logic: S1. When flying to the preset area, the center point of the tower top is identified by the laser radar, and the flight attitude is automatically adjusted so that the camera is vertically aligned with the target. When shooting the insulator string, the gimbal is controlled to rotate horizontally to cover all the porcelain insulators. S2. When the visible light camera is shooting at high speed, the infrared thermal imager is simultaneously triggered to record temperature data. When the lidar scans the three-dimensional structure of the tower, the ultraviolet imager detects the corona discharge point. S3. When switching between tasks, adjust the gimbal angle in advance to smoothly transition to the next shooting posture.

4. The multimodal unmanned aerial vehicle integrated shooting system as described in claim 1, characterized in that, The central collaborative control unit is connected to the inertial measurement unit. By coordinating the wind speed monitored by the millimeter-wave radar with the data from the inertial measurement unit, the inertial measurement unit calculates the UAV's attitude angle in real time and compensates for high-frequency vibrations. This complements the low-frequency motion monitoring of the millimeter-wave radar, dynamically adjusting the flight attitude to reduce image jitter.

5. The multimodal unmanned aerial vehicle integrated shooting system as described in claim 1, characterized in that, The multimodal sensing module also includes a sensor spatiotemporal synchronization calibration module; Sensor spatiotemporal synchronization calibration module: It adopts the PTP protocol to achieve hardware-level time synchronization of visible light cameras, infrared thermal imagers, lidar and millimeter-wave radar, and calibrates the spatial coordinate system of each sensor through a laser interferometer to ensure spatiotemporal consistency before multi-source data fusion.

6. The multimodal unmanned aerial vehicle integrated shooting system as described in claim 1, characterized in that, The multimodal shooting unit includes a fixed device, an unmanned body (101), a gimbal (102), an integrated housing (103), a transparent window (104), a square plug (105), and a positioning groove (106). The gimbal (102) is mounted on the outer wall of the unmanned aerial vehicle (101); The visible light camera, infrared thermal imager, ultraviolet imager, lidar and millimeter-wave radar are installed inside the integrated housing (103); A transparent window (104) is provided on the outer wall of the integrated housing (103); A square insert (105) is located at the rear end of the integrated housing (103); Multiple positioning grooves (106) are all set on the outer wall of the square insert (105); The fixing device is set on the moving end of the gimbal (102). The fixing device is used to quickly fix the integrated housing (103) using a square plug (105) and multiple sets of positioning slots (106).

7. The multimodal unmanned aerial vehicle integrated shooting system as described in claim 6, characterized in that, The fixing device includes a positioning component, a fixing plate (201), a limiting component (202), a connecting cylinder (203), a turntable (204), a locking block (205), a connecting column (206), and a guide groove (207). The top of the fixed plate (201) is connected to the movable end of the gimbal (102), and a square hole is provided in the middle of the fixed plate (201), through which a square insert rod (105) passes; Multiple sets of limiting components (202) are all set on the front part of the outer side wall of the fixed plate (201), and the rear end of the integrated housing (103) is embedded between the multiple sets of limiting components (202); The connecting cylinder (203) is located on the rear part of the outer wall of the fixing plate (201), and the connecting cylinder (203) is provided with positioning holes; The turntable (204) is rotatably mounted on the connecting cylinder (203); Multiple sets of locking blocks (205) are slidably installed on the outer wall of the fixing plate (201); Multiple sets of connecting posts (206) are respectively set on the outer side wall of multiple sets of locking blocks (205); Multiple sets of guide grooves (207) are respectively set on the inner side wall of the turntable (204), and multiple sets of connecting columns (206) are respectively slidably set in the multiple sets of guide grooves (207); The positioning components are set on the connecting cylinder (203) and the turntable (204). The positioning components cooperate with the positioning holes to fix the rotation of the turntable (204).

8. The multimodal unmanned aerial vehicle integrated shooting system as described in claim 7, characterized in that, The positioning assembly includes a first sleeve (301), a first pin (302), a first spring (303), a second sleeve (304), a second pin (305), a second spring (306), and a pressing handle (307). The first sleeve (301) is set on the outer wall of the turntable (204); The first pin (302) is slidably mounted on the first sleeve (301); The first spring (303) is disposed inside the first sleeve (301); The second sleeve (304) is disposed on the outer wall of the connecting sleeve (203); The second pin (305) is slidably mounted on the second sleeve (304); The second spring (306) is installed inside the second sleeve (304); The pressing handle (307) is located at the top of the second pin (305).

9. A multimodal unmanned aerial vehicle integrated shooting system as described in claim 7, characterized in that, It also includes magnets (401); The magnet (401) is located on the outer wall of the turntable (204).

10. The multimodal unmanned aerial vehicle integrated shooting system as described in claim 7, characterized in that, It also includes anti-slip texture (501); Anti-slip texture (501) is provided on the outer wall of turntable (204).

Images