Unmanned aerial vehicle live broadcast image analysis processing method, system, device and medium

Abstract

The invention provides an unmanned aerial vehicle live broadcast image analysis processing method and device, equipment and a medium, and the method comprises the steps: dynamically generating an anti-interference inspection path based on real-time environment perception; synchronously acquiring multi-source heterogeneous data of the power station, and preprocessing and storing the multi-source heterogeneous data; constructing an intelligent diagnosis model, and performing material-level defect diagnosis on an image in the multi-source heterogeneous data through learning fusion of a numerical model driven by a physical mechanism and a data-driven machine; the adaptive optimization of the intelligent diagnosis model is realized through the cooperation of algorithm continuous evolution and hardware dynamic adaptation technologies, so as to solve the problems that the unmanned aerial vehicle inspection system is lack of multi-modal data fusion, insufficient in dynamic flight path planning capability and too high in cloud dependency.

Description

Technical Field

[0001] The present invention relates to the field of image analysis technology, and in particular to a method, device, equipment and medium for analyzing and processing live broadcast images of unmanned aerial vehicles. Background Art

[0002] With the large-scale deployment of photovoltaic power plants, traditional manual inspection methods can no longer meet the high-efficiency and high-precision operation and maintenance requirements. Existing drone inspection systems are mostly mounted with a single visible light or infrared imaging device, requiring manual remote control to fly and transmit massive amounts of raw data to the backend for analysis, resulting in three core defects:

[0003] First, due to the lack of deep integration of spectral sensing and AI real-time edge computing, online fault type identification and precise geographic location cannot be performed, requiring manual secondary screening, resulting in response delays.

[0004] Second, traditional trajectory planning relies on patrolling at preset coordinate points and is unable to adapt to changes in PV array layout and sudden occlusion scenarios, resulting in a high missed detection rate.

[0005] Third, video processing relies on cloud-based central servers, which can easily lead to limited transmission bandwidth and delayed analysis in areas with weak 5G coverage, making it difficult to meet the emergency response requirements of emergencies such as wildfires and foreign object intrusions within seconds.

[0006] Therefore, it is urgent to propose an analysis and processing method for drone live broadcast images to solve the technical problems of drone inspection systems, such as lack of multimodal data fusion, insufficient dynamic trajectory planning capabilities, and high dependence on the cloud. Summary of the Invention

[0007] In order to overcome the problems existing in the related art, the present disclosure provides a method, device, equipment and medium for analyzing and processing drone live broadcast images, so as to solve the technical problems of drone inspection systems in the related art, such as the lack of multimodal data fusion, insufficient dynamic trajectory planning capabilities and high cloud dependence.

[0008] One or more embodiments of this specification provide a method for analyzing and processing live broadcast images from a drone, including the following steps:

[0009] Dynamically generate interference-resistant inspection paths based on real-time environmental perception;

[0010] Synchronously collect, pre-process and store multi-source heterogeneous data from power plants;

[0011] Constructing an intelligent diagnostic model to diagnose material-level defects in images from the multi-source heterogeneous data by fusing a physical mechanism-driven numerical model with data-driven machine learning;

[0012] The adaptive optimization of the intelligent diagnosis model is achieved through the collaboration of continuous algorithm evolution and dynamic hardware adaptation.

[0013] Preferably, the synchronous collection of multi-source heterogeneous data of the power station and pre-processing and storage specifically includes the following steps:

[0014] A master-slave clock server is used to trigger sensor synchronization, and a 5±0.2ms level difference compensation mechanism is configured to align the lidar and electric field sensor data streams.

[0015] The 0.3T-400MHz electromagnetic harmonic characteristic matrix is ​​constructed using three-dimensional vector interpolation, and the power station coordinate system conversion model is established in combination with the MEMS quantum gyroscope;

[0016] The optical aperture is dynamically adjusted by a 200Hz variable aperture actuator, and the Sobel-Feldman algorithm is used to compensate for 3.5-6.5mm focal plane offset;

[0017] The 10-500 Hz vibration eigenmodes of photovoltaic panels are extracted and a natural frequency fingerprint library with a ±5% tolerance is established.

[0018] Preferably, the dynamic generation of an anti-interference inspection path based on real-time environmental perception specifically includes the following steps:

[0019] Fusion of 2cm precision LiDAR point cloud and FMEA database to generate risk probability density cloud map;

[0020] The improved Fast Marching algorithm is used to solve the three-dimensional risk gradient field, and the constraints are: flight attitude angle ≤ 25°, electromagnetic radiation exposure < 10V / m, and remaining power > 35%;

[0021] Deploy the SPB protocol to prioritize the transmission of 1080P video streams and 1kHz telemetry data;

[0022] Implement a federated learning framework that triggers inter-node weight synchronization when the gradient tensor compression rate is ≥78%.

[0023] Preferably, an intelligent diagnostic model is constructed to diagnose material-level defects in images from the multi-source heterogeneous data by fusing a physical mechanism-driven numerical model with data-driven machine learning, specifically comprising the following steps:

[0024] A FIB-SEM dual-beam system was used to acquire nanoscale lattice dislocation data, and EDS element distribution maps were aligned using a multi-scale registration algorithm;

[0025] Construct a 3D Maxwell equation solver with L2 regularization constraints to invert carrier migration trajectories;

[0026] The array-level entropy flux is calculated based on the Boltzmann transport equation, and the CUSUM algorithm is used to detect statistical mutations.

[0027] In 1024 3 Lattice Boltzmann microcrack growth simulation is performed on the mesh, and mesh adaptive refinement optimization is performed every 50 steps.

[0028] Preferably, the adaptive optimization of the intelligent diagnostic model is achieved by synergizing the continuous evolution of the algorithm with the dynamic adaptation of the hardware, specifically comprising the following steps:

[0029] Construct a teacher-student knowledge distillation framework and constrain the KL divergence threshold to ≤ 0.05;

[0030] Spectral normalized Wasserstein GAN is used to generate defect samples containing PID effects, and the gradient penalty coefficient λ is set to 10;

[0031] Design an eight-dimensional fitness function that includes path coverage and defect detection rate, and initiate an elite retention strategy;

[0032] ZigBee 3.0 and Modbus RTU protocols are uniformly managed through the hardware abstraction layer, maintaining a cross-platform operation error rate of less than 0.03%.

[0033] One or more embodiments of this specification provide a device for analyzing and processing live broadcast images of a drone, including a path generation module, a data acquisition module, a diagnosis module, and an optimization module;

[0034] The path generation module is used to dynamically generate an interference-resistant inspection path based on real-time environmental perception;

[0035] The data acquisition module is used to synchronously collect multi-source heterogeneous data of the power station and perform pre-processing and storage;

[0036] The diagnostic module is used to build an intelligent diagnostic model to diagnose material-level defects in images in the multi-source heterogeneous data by fusing a physical mechanism-driven numerical model with data-driven machine learning;

[0037] The optimization module is used to achieve adaptive optimization of the intelligent diagnosis model through the technology of continuous algorithm evolution and dynamic hardware adaptation.

[0038] Preferably, the data acquisition module includes a synchronization unit, a conversion unit, a compensation unit and a fingerprint library construction unit;

[0039] The synchronization unit is used to trigger sensor synchronization using a master-slave clock server and configure a 5±0.2ms level difference compensation mechanism to align the lidar and electric field sensor data streams;

[0040] The conversion unit is used to construct a 0.3T-400MHz electromagnetic harmonic characteristic matrix using three-dimensional vector interpolation and establish a power station coordinate system conversion model in combination with a MEMS quantum gyroscope;

[0041] The compensation unit is used to dynamically adjust the optical aperture through a 200Hz variable aperture actuator and compensate for the 3.5-6.5mm focal plane offset using the Sobel-Feldman algorithm;

[0042] The fingerprint library construction unit is used to extract the 10-500 Hz vibration eigenmode of the photovoltaic panel and establish a natural frequency fingerprint library with a tolerance of ±5%.

[0043] Preferably, the path generation module includes a risk probability density cloud map generation unit, a solution unit, a transmission unit and a synchronization unit;

[0044] The risk probability density cloud map generating unit is used to fuse the 2cm precision LiDAR point cloud and the FMEA database to generate a risk probability density cloud map;

[0045] The solving unit is used to solve the three-dimensional risk gradient field by applying the improved Fast Marching algorithm, and the constraints are: flight attitude angle ≤ 25°, electromagnetic radiation exposure < 10V / m, and remaining power > 35%;

[0046] The transmission unit is used to deploy the SPB protocol to preferentially transmit 1080P video streams and 1kHz telemetry data;

[0047] The synchronization unit is used to implement the federated learning framework and trigger inter-node weight synchronization when the gradient tensor compression rate is ≥78%.

[0048] One or more embodiments of this specification provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned method for analyzing and processing drone live broadcast images is implemented.

[0049] One or more embodiments of this specification provide a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the above-mentioned method for analyzing and processing drone live broadcast images.

[0050] The present disclosure provides a method, device, equipment and medium for analyzing and processing live drone images. The advantages are that by dynamically generating interference-resistant inspection paths based on real-time environmental perception, drones can achieve efficient and safe inspection operations in complex and changing environments, effectively avoid interference factors, significantly improve inspection efficiency and quality, greatly reduce the risk of missed inspections, and ensure the comprehensiveness and accuracy of inspection work; synchronously collect multi-source heterogeneous data from power stations, and pre-process and store them, which can effectively integrate multi-source data, provide comprehensive, accurate and orderly data support for subsequent analysis, ensure data consistency and availability, and lay a solid foundation for in-depth analysis; build an intelligent diagnosis model, through physical The fusion of the numerical model driven by the physical mechanism and the data-driven machine learning can diagnose material-level defects in the images in the multi-source heterogeneous data, and can deeply explore the material defect information from the two dimensions of microstructure and electromagnetic properties, which can significantly improve the accuracy and depth of defect diagnosis, and provide a strong basis for accurate positioning and problem solving; through the technical collaboration of continuous algorithm evolution and dynamic hardware adaptation, the adaptive optimization of the intelligent diagnostic model is achieved, and the diagnostic model is adjusted and improved in real time according to the constantly updated data and changing environmental conditions, which helps to continuously improve the analysis and processing capabilities of the model and the reliability of fault diagnosis, and ensure that the system always maintains efficient and accurate performance during long-term operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 A flowchart of a method for analyzing and processing live drone images provided in one or more embodiments of this specification;

[0053] Figure 2 A schematic diagram of the structure of a device for analyzing and processing live broadcast images of a drone provided in one or more embodiments of this specification;

[0054] Figure 3 A schematic diagram of the structure of a computer device provided in one or more embodiments of this specification. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this invention document.

[0056] The present invention will be described in detail below with reference to specific implementation methods and the accompanying drawings.

[0057] Method Example

[0058] According to an embodiment of the present invention, a method for analyzing and processing a live broadcast image of a drone is provided. Figure 1 FIG. 1 is a flow chart of a method for analyzing and processing a live broadcast image of a drone provided in this embodiment. The method for analyzing and processing a live broadcast image of a drone according to an embodiment of the present invention includes the following steps:

[0059] S110. Dynamically generate an interference-resistant inspection path based on real-time environmental perception.

[0060] S120 , synchronously collect multi-source heterogeneous data of the power station and perform pre-processing and storage.

[0061] S130. Construct an intelligent diagnostic model to diagnose material-level defects in images in the multi-source heterogeneous data by fusing a numerical model driven by physical mechanisms with data-driven machine learning.

[0062] S140. Implement adaptive optimization of the intelligent diagnosis model through the collaboration of continuous algorithm evolution and dynamic hardware adaptation technologies.

[0063] The method provided in this embodiment dynamically generates interference-resistant inspection paths based on real-time environmental perception, enabling drones to perform efficient and safe inspections in complex and changing environments. It can effectively avoid interference factors, significantly improve inspection efficiency and quality, greatly reduce the risk of missed inspections, and ensure the comprehensiveness and accuracy of inspection work. It simultaneously collects multi-source heterogeneous data from power stations and pre-processes and stores them, which can effectively integrate multi-source data and provide comprehensive, accurate and orderly data support for subsequent analysis, ensuring data consistency and availability, and laying a solid foundation for in-depth analysis. It builds an intelligent diagnostic model and uses physical mechanism-driven numerical models and data to Driven by machine learning fusion, it can diagnose material-level defects in images in the multi-source heterogeneous data, and deeply explore material defect information from the two dimensions of microstructure and electromagnetic properties, which can significantly improve the accuracy and depth of defect diagnosis, and provide a strong basis for accurate positioning and problem solving; through the technical collaboration of continuous algorithm evolution and dynamic hardware adaptation, the adaptive optimization of the intelligent diagnostic model is achieved, so that the diagnostic model can be adjusted and improved in real time according to the constantly updated data and changing environmental conditions, which helps to continuously improve the model's analysis and processing capabilities and the reliability of fault diagnosis, and ensure that the system always maintains efficient and accurate performance during long-term operation.

[0064] In one embodiment, step S110, dynamically generating an interference-resistant inspection path based on real-time environmental perception, specifically includes the following steps:

[0065] The PV array risk units are segmented using the Voronoi diagram method. A 2cm-precision point cloud map constructed using 2cm-precision Lidar SLAM (Light Detection and Ranging Simultaneous Localization and Mapping) is integrated with the FMEA (Failure Mode Database) failure mode database, and Monte Carlo simulation is used to generate a risk probability density cloud map.

[0066] The improved Fast Marching algorithm is applied to solve the optimal path of the three-dimensional risk gradient field, and a multi-objective optimization equation is configured. The constraints are as follows: flight attitude angle ≤ 25°, electromagnetic radiation exposure < 10V / m, and remaining power > 35%.

[0067] Deploy the SPB (Shortest Path Bridging) protocol, set a dynamic QoS policy to prioritize the transmission of 1080P video streams (<50ms latency) and 1kHz telemetry data, and develop an adaptive power adjustment algorithm based on Kalman prediction to maintain an RSRP value greater than -110dBm.

[0068] The detection model (ResNet-50 architecture) is encapsulated in Docker containers, a P2P transmission channel is established to implement the federated learning framework, and an incremental update protocol is designed to trigger inter-node weight synchronization when the gradient tensor compression rate is ≥78%.

[0069] The method provided in this embodiment generates a risk probability density cloud map by fusing a 2cm high-precision LiDAR point cloud with an FMEA database, providing a scientific basis for path planning. It uses an improved Fast Marching algorithm to solve the three-dimensional risk gradient field according to constraints such as flight attitude angle, electromagnetic radiation exposure, and remaining power, ensuring that the path is realistic and safe. It deploys the SPB protocol to prioritize the transmission of key data, helping operators obtain on-site real-time information. It implements a federated learning framework, triggering node weight synchronization with a gradient tensor compression rate of ≥78%, ensuring data security while continuously optimizing the model, improving inspection efficiency and reliability, and promoting intelligent and efficient power inspections.

[0070] In one embodiment, step S120, synchronously collecting, preprocessing, and storing multi-source heterogeneous data of a power station, specifically includes the following steps:

[0071] A master-slave clock server was used to trigger sensor synchronization, and a 5±0.2ms level difference compensation mechanism was configured to align the lidar (100Hz sampling rate) and electric field sensor data streams (2000S / s sampling rate). The SpikeRecorder XT toolkit was used to align the visible light YUV422 image stream (1920×1080 resolution, 30fps) and electromagnetic gradient tensor data and store them on a SanDisk FusionJet high-speed array.

[0072] Based on EagleEyes EM Pro 2.0 software, a 0.3T-400MHz electromagnetic harmonic characteristic matrix was constructed using three-dimensional vector interpolation. A power station coordinate system conversion model was established in combination with a MEMS quantum gyroscope (±0.005° accuracy). A CUDA-accelerated gradient descent algorithm was deployed on the Nvidia Jetson AGX to match physical coordinates in real time.

[0073] The turbulence intensity sensor is used to trigger the Hilbert-Huang transform to decompose the atmospheric disturbance mode. The optical aperture is dynamically adjusted by a 200 Hz variable aperture actuator (Physik Instrumente M-414.3PD). The Sobel-Feldman algorithm is used to compensate for the 3.5-6.5 mm focal plane offset to ensure that the spatial resolution of hot spot detection is maintained at 0.12 mrad.

[0074] The Polytec PDV-100 laser vibrometer system (1 nm displacement resolution) was started. The 10-500 Hz vibration eigenmodes of the photovoltaic panel were extracted using the Wigner-Ville time-frequency analysis method, and a natural frequency fingerprint library with a ±5% tolerance was established. The MTSLandmark data collector was configured to establish the component natural frequency fingerprint library, and the ±5% tolerance range was set as the threshold baseline for subsequent deformation monitoring.

[0075] The method provided in this embodiment uses a master-slave clock server and a differential compensation mechanism to precisely align the data streams of the lidar and electric field sensors, improving data synchronization accuracy. It constructs an electromagnetic harmonic feature matrix and uses a MEMS quantum gyroscope to establish a coordinate system conversion model. This allows for precise control of electromagnetic and coordinate information, utilizes a variable aperture actuator and algorithm to compensate for focal plane offset, optimizes optical imaging quality, extracts the eigenmodes of photovoltaic panel vibration, and establishes a natural frequency fingerprint library. This enhances the photovoltaic panel status monitoring capability and provides strong support for intelligent power station management.

[0076] In one embodiment, step S130, constructing an intelligent diagnostic model, by fusing a physical mechanism-driven numerical model with data-driven machine learning, to diagnose material-level defects in images in the multi-source heterogeneous data, specifically includes the following steps:

[0077] A FIB-SEM dual-beam system was used to acquire nanoscale lattice dislocation data (<5nm positioning accuracy), Hough transform was used to identify lattice dislocation points, and EDS element distribution maps were aligned using a multi-scale registration algorithm.

[0078] A 3D Maxwell equation with L2 regularization constraint is constructed, and the carrier migration trajectory is inverted by combining the measured potential distribution solver. The L2 regularization constraint is set to prevent the divergence of the charge conservation equation under pathological conditions.

[0079] A multidimensional phase space discrete grid (100×100×50) was defined, and the array-level entropy flux was calculated based on the Boltzmann transport equation. The CUSUM algorithm was used to detect statistical mutations in the non-steady-state process.

[0080] In 1024 3 Lattice Boltzmann microcrack propagation simulation is performed on the mesh, and the Johnson-Cook constitutive model is combined to set the polysilicon brittle fracture criterion. The mesh adaptive refinement optimization is performed every 50 steps.

[0081] The method provided in this embodiment utilizes a FIB-SEM dual-beam system and a multi-scale registration algorithm to acquire nanoscale lattice dislocation data and align it with EDS element distribution maps, revealing the causes of defects at the microstructural level. A 3D Maxwell equation solver with L2 regularization constraints is constructed to stably and accurately invert carrier migration trajectories, clarifying the relationship between electromagnetic properties and defects. Entropy flux is calculated based on the Boltzmann transport equation, and statistical mutations are detected using the CUSUM algorithm to promptly identify potential defects. Lattice Boltzmann microcrack growth simulations on a 10243 grid and adaptive mesh refinement every 50 steps accurately predict defect development trends. This method can significantly improve the accuracy of the analysis of the coupling relationship between material defect microstructure and electromagnetic properties, providing strong support for ensuring material and device performance.

[0082] In one embodiment, step S140, implementing the adaptive optimization of the intelligent diagnostic model through the collaboration of continuous algorithm evolution and dynamic hardware adaptation, specifically includes the following steps:

[0083] A teacher-student knowledge distillation framework (DenseNet-201→MobileNetV3) is constructed, an attention loss function is designed to extract key topological features, and the KL divergence threshold is constrained to ≤0.05 to ensure the effectiveness of knowledge compression.

[0084] The spectral normalized Wasserstein GAN is used to generate complex defect samples including PID effect, potential induced decay, etc. (generation resolution 2048×2048). The spectral normalization technique is used to ensure the Lipschitz continuity of the generator, and the gradient penalty coefficient λ is set to 10.

[0085] An elite retention strategy framework was developed, and an eight-dimensional fitness function was designed, including path coverage (≥85%), defect detection rate (≥98%), and diagnosis delay (≤200ms). The elite retention strategy was activated when the evolutionary generation reached 50 generations.

[0086] Through the hardware abstraction layer, ZigBee 3.0 and Modbus RTU protocols are uniformly managed, and ISAAC middleware is developed to implement instruction set translation between FPGA accelerator cards and GPU CUDA kernels, maintaining a cross-platform computing error rate of less than 0.03%.

[0087] The method provided in this embodiment constructs a teacher-student knowledge distillation framework, constrains the KL divergence threshold to ≤ 0.05, achieves efficient knowledge transfer and compression, and improves model operation efficiency. It uses spectrally normalized Wasserstein GAN to generate defect samples containing PID effects, sets the gradient penalty coefficient λ to 10, greatly enriches sample diversity, and enhances the model's ability to diagnose complex defects. It designs an eight-dimensional fitness function that includes path coverage and defect detection rate, and initiates NSGA-II multi-objective optimization to comprehensively optimize the model from multiple dimensions to ensure diagnostic accuracy and reliability. It uniformly manages the ZigBee 3.0 and Modbus RTU protocols through the hardware abstraction layer, maintains a cross-platform computing error rate of less than 0.03%, achieves seamless integration between devices and protocols, ensures model stability and versatility, and can significantly improve the model's adaptability, performance, and application scope, effectively promoting the intelligent development of related fields.

[0088] Device embodiment

[0089] According to an embodiment of the present invention, a device for analyzing and processing live broadcast images of a drone is provided. Figure 2 As shown, it is a structural diagram of the drone live broadcast image analysis and processing device provided in this embodiment. The drone live broadcast image analysis and processing device according to the embodiment of the present invention includes a path generation module 21, a data acquisition module 22, a diagnosis module 23 and an optimization module 24.

[0090] The path generation module 21 is used to dynamically generate an interference-resistant inspection path based on real-time environmental perception.

[0091] The data acquisition module 22 is used to synchronously acquire multi-source heterogeneous data of the power station and perform pre-processing and storage.

[0092] The diagnosis module 23 is used to build an intelligent diagnosis model to diagnose material-level defects in images in the multi-source heterogeneous data by fusing a numerical model driven by physical mechanisms with data-driven machine learning.

[0093] The optimization module 24 is used to realize the adaptive optimization of the intelligent diagnosis model by combining the continuous evolution of the algorithm with the dynamic adaptation of the hardware.

[0094] The device provided in this embodiment dynamically generates an anti-interference inspection path based on real-time environmental perception through the path generation module 21, so that the UAV can achieve efficient and safe inspection operations in complex and changing environments, effectively avoid interference factors, significantly improve inspection efficiency and quality, greatly reduce the risk of missed inspections, and ensure the comprehensiveness and accuracy of inspection work; the data acquisition module 22 synchronously collects multi-source heterogeneous data of the power station, and pre-processes and stores it, which can effectively integrate multi-source data, provide comprehensive, accurate and orderly data support for subsequent analysis, ensure data consistency and availability, and lay a solid foundation for in-depth analysis; the diagnosis module 23 constructs an intelligent diagnosis model, which is driven by physical mechanisms. The numerical model is integrated with data-driven machine learning to diagnose material-level defects in images from the multi-source heterogeneous data. It can deeply explore material defect information from the two dimensions of microstructure and electromagnetic properties, which can significantly improve the accuracy and depth of defect diagnosis, and provide a strong basis for accurate positioning and problem solving; the optimization module 24 realizes the adaptive optimization of the intelligent diagnostic model through the technology of continuous algorithm evolution and dynamic hardware adaptation, so that the diagnostic model can be adjusted and improved in real time according to the constantly updated data and changing environmental conditions, which helps to continuously improve the analysis and processing capabilities of the model and the reliability of fault diagnosis, and ensure that the system always maintains efficient and accurate performance during long-term operation.

[0095] In one embodiment, the path generation module 21 includes a risk probability density cloud map generation unit, a solution unit, a transmission unit, and a synchronization unit.

[0096] The risk probability density cloud map generating unit is used to fuse the 2cm precision LiDAR point cloud and the FMEA database to generate a risk probability density cloud map.

[0097] The solving unit is used to apply the improved Fast Marching algorithm to solve the three-dimensional risk gradient field, and the constraints are: flight attitude angle ≤ 25°, electromagnetic radiation exposure < 10V / m, and remaining power > 35%.

[0098] The transmission unit is used to deploy the SPB protocol to preferentially transmit 1080P video streams and 1kHz telemetry data.

[0099] The synchronization unit is used to implement the federated learning framework and trigger inter-node weight synchronization when the gradient tensor compression rate is ≥78%.

[0100] The device provided in this embodiment generates a risk probability density cloud map by fusing a 2cm high-precision LiDAR point cloud with an FMEA database, providing a scientific basis for path planning. It uses an improved Fast Marching algorithm to solve the three-dimensional risk gradient field in accordance with constraints such as flight attitude angle, electromagnetic radiation exposure, and remaining power, ensuring that the path is realistic and safe. It deploys the SPB protocol to prioritize the transmission of key data, helping operators obtain on-site real-time information. It implements a federated learning framework, triggering node weight synchronization with a gradient tensor compression rate of ≥78%, ensuring data security while continuously optimizing the model, improving inspection efficiency and reliability, and promoting intelligent and efficient power inspections.

[0101] In one embodiment, the data collection module 22 includes a synchronization unit, a conversion unit, a compensation unit, and a fingerprint library construction unit.

[0102] The synchronization unit is used to use a master-slave clock server to trigger sensor synchronization and configure a 5±0.2ms level difference compensation mechanism to align the laser radar and electric field sensor data streams.

[0103] The conversion unit is used to construct a 0.3T-400MHz electromagnetic harmonic characteristic matrix using three-dimensional vector interpolation, and to establish a power station coordinate system conversion model in combination with a MEMS quantum gyroscope.

[0104] The compensation unit is used to dynamically adjust the optical aperture through a 200 Hz variable aperture actuator and compensate for the 3.5-6.5 mm focal plane offset using the Sobel-Feldman algorithm.

[0105] The fingerprint library construction unit is used to extract the 10-500 Hz vibration eigenmode of the photovoltaic panel and establish a natural frequency fingerprint library with a tolerance of ±5%.

[0106] The device provided in this embodiment uses a master-slave clock server and a differential compensation mechanism to precisely align the data streams of the lidar and electric field sensors, improving data synchronization accuracy. It constructs an electromagnetic harmonic feature matrix and uses a MEMS quantum gyroscope to establish a coordinate system conversion model. This allows for precise control of electromagnetic and coordinate information, utilizes a variable aperture actuator and algorithm to compensate for focal plane offset, optimizes optical imaging quality, extracts the eigenmodes of photovoltaic panel vibration, and establishes a natural frequency fingerprint library. This enhances the photovoltaic panel status monitoring capability and provides strong support for intelligent power station management.

[0107] The embodiment of the present invention is an apparatus embodiment corresponding to the above-mentioned method embodiment. The specific operations of the processing steps of each module can be understood by referring to the description of the method embodiment, and will not be repeated here.

[0108] like Figure 3As shown, the present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for analyzing and processing the live broadcast image of the drone in the above-mentioned embodiment is implemented, or when the computer program is executed by a processor, the method for analyzing and processing the live broadcast image of the drone in the above-mentioned embodiment is implemented.

[0109] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0110] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art can understand and implement it without making any creative efforts.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and the contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.

Claims

1. A method for analyzing and processing live broadcast images of drones, characterized in that: The following steps are involved: Dynamically generate interference-resistant inspection paths based on real-time environmental perception; Synchronously collect, pre-process and store multi-source heterogeneous data from power plants; Constructing an intelligent diagnostic model to diagnose material-level defects in images from the multi-source heterogeneous data by fusing a physical mechanism-driven numerical model with data-driven machine learning; The adaptive optimization of the intelligent diagnosis model is achieved through the collaboration of continuous algorithm evolution and dynamic hardware adaptation.

2. The method for analyzing and processing live broadcast images of a drone according to claim 1, wherein: The synchronous collection of multi-source heterogeneous data of the power station and pre-processing and storage specifically includes the following steps: A master-slave clock server is used to trigger sensor synchronization, and a 5±0.2ms level difference compensation mechanism is configured to align the lidar and electric field sensor data streams. The 0.3T-400MHz electromagnetic harmonic characteristic matrix is ​​constructed using three-dimensional vector interpolation, and the power station coordinate system conversion model is established in combination with the MEMS quantum gyroscope; The optical aperture is dynamically adjusted by a 200Hz variable aperture actuator, and the Sobel-Feldman algorithm is used to compensate for 3.5-6.5mm focal plane offset; The 10-500 Hz vibration eigenmodes of photovoltaic panels are extracted and a natural frequency fingerprint library with a ±5% tolerance is established.

3. The method for analyzing and processing live broadcast images of a drone according to claim 1, wherein: The method of dynamically generating an anti-interference inspection path based on real-time environmental perception specifically includes the following steps: Fusion of 2cm precision LiDAR point cloud and FMEA database to generate risk probability density cloud map; The improved Fast Marching algorithm is used to solve the three-dimensional risk gradient field, and the constraints are: flight attitude angle ≤ 25°, electromagnetic radiation exposure < 10V / m, and remaining power > 35%; Deploy the SPB protocol to prioritize the transmission of 1080P video streams and 1kHz telemetry data; Implement a federated learning framework that triggers inter-node weight synchronization when the gradient tensor compression rate is ≥78%.

4. The method for analyzing and processing a live broadcast image of a drone according to claim 1, wherein: The intelligent diagnosis model is constructed to diagnose material-level defects in images from the multi-source heterogeneous data by fusing a physical mechanism-driven numerical model with data-driven machine learning. Specifically, the following steps are included: A FIB-SEM dual-beam system was used to acquire nanoscale lattice dislocation data, and EDS element distribution maps were aligned using a multi-scale registration algorithm; Construct a 3D Maxwell equation solver with L2 regularization constraints to invert carrier migration trajectories; The array-level entropy flux is calculated based on the Boltzmann transport equation, and the CUSUM algorithm is used to detect statistical mutations. In 1024 3 Lattice Boltzmann microcrack growth simulation is performed on the mesh, and mesh adaptive refinement optimization is performed every 50 steps.

5. The method for analyzing and processing live broadcast images of a drone according to claim 1, wherein: The adaptive optimization of the intelligent diagnostic model is achieved by synergizing the continuous evolution of the algorithm with the dynamic adaptation of the hardware, and specifically includes the following steps: Construct a teacher-student knowledge distillation framework and constrain the KL divergence threshold to ≤ 0.05; Spectral normalized Wasserstein GAN is used to generate defect samples containing PID effects, and the gradient penalty coefficient λ is set to 10; Design an eight-dimensional fitness function that includes path coverage and defect detection rate, and initiate an elite retention strategy; ZigBee 3.0 and Modbus RTU protocols are uniformly managed through the hardware abstraction layer, maintaining a cross-platform operation error rate of less than 0.03%.

6. An analysis and processing device for drone live broadcast images, characterized in that: It includes path generation module, data acquisition module, diagnosis module and optimization module; The path generation module is used to dynamically generate an interference-resistant inspection path based on real-time environmental perception; The data acquisition module is used to synchronously collect multi-source heterogeneous data of the power station and perform pre-processing and storage; The diagnostic module is used to build an intelligent diagnostic model to diagnose material-level defects in images in the multi-source heterogeneous data by fusing a physical mechanism-driven numerical model with data-driven machine learning; The optimization module is used to achieve adaptive optimization of the intelligent diagnosis model through the technology of continuous algorithm evolution and dynamic hardware adaptation.

7. The analysis and processing device for drone live broadcast images according to claim 6, characterized in that: The data acquisition module includes a synchronization unit, a conversion unit, a compensation unit and a fingerprint library construction unit; The synchronization unit is used to trigger sensor synchronization using a master-slave clock server and configure a 5±0.2ms level difference compensation mechanism to align the lidar and electric field sensor data streams; The conversion unit is used to construct a 0.3T-400MHz electromagnetic harmonic characteristic matrix using three-dimensional vector interpolation and establish a power station coordinate system conversion model in combination with a MEMS quantum gyroscope; The compensation unit is used to dynamically adjust the optical aperture through a 200Hz variable aperture actuator and compensate for the 3.5-6.5mm focal plane offset using the Sobel-Feldman algorithm; The fingerprint library construction unit is used to extract the 10-500 Hz vibration eigenmode of the photovoltaic panel and establish a natural frequency fingerprint library with a tolerance of ±5%.

8. The analysis and processing device for drone live broadcast images according to claim 6, characterized in that: The path generation module includes a risk probability density cloud map generation unit, a solution unit, a transmission unit and a synchronization unit; The risk probability density cloud map generating unit is used to fuse the 2cm precision LiDAR point cloud and the FMEA database to generate a risk probability density cloud map; The solving unit is used to solve the three-dimensional risk gradient field by applying the improved Fast Marching algorithm, and the constraints are: flight attitude angle ≤ 25°, electromagnetic radiation exposure < 10V / m, and remaining power > 35%; The transmission unit is used to deploy the SPB protocol to preferentially transmit 1080P video streams and 1kHz telemetry data; The synchronization unit is used to implement the federated learning framework and trigger inter-node weight synchronization when the gradient tensor compression rate is ≥78%.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for analyzing and processing the live broadcast image of the drone as described in any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for analyzing and processing the live broadcast image of the drone as described in any one of claims 1 to 4 are implemented.

Images