Real-time icing detection method based on unmanned aerial vehicle multi-mode image fusion

Abstract

The invention belongs to the technical field of unmanned aerial vehicles, and particularly relates to a real-time icing detection method based on unmanned aerial vehicle multi-mode image fusion, which comprises the steps of multi-mode image acquisition, image preprocessing, multi-mode image fusion, icing detection model construction, real-time icing detection and early warning and system integration. Unmanned aerial vehicle control, image acquisition, image fusion, data processing, model reasoning and early warning notification are integrated to an unmanned aerial vehicle edge computing platform, and early warning information is transmitted to a ground control center in real time through a 5G module. The power transmission line icing detection efficiency, accuracy and real-time performance are comprehensively improved, an advanced, reliable and efficient solution is provided for safe and stable operation of a power system, and the method has remarkable technical advantages and wide application prospects.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles (UAVs), and in particular to a real-time ice detection method based on UAV multi-mode image fusion. Background Art

[0002] With the rapid development of power systems, the safe and stable operation of transmission lines is crucial for ensuring energy supply and maintaining social stability. In harsh weather conditions such as low temperatures and freezing temperatures, transmission lines are prone to icing, which significantly increases line loads and can cause serious accidents such as line breakage and tower collapse, leading to large-scale power outages and significant economic losses and social impacts.

[0003] Traditional methods for detecting ice buildup on transmission lines rely primarily on manual inspections. However, manual inspections have numerous drawbacks. Firstly, they are inefficient and require significant human resources; secondly, conducting inspections in adverse weather conditions presents significant safety risks. Furthermore, the data obtained from manual inspections often lacks real-time and accuracy, making them difficult to meet the stringent requirements for transmission line status monitoring in modern power systems.

[0004] In recent years, drone technology has begun to be applied to power transmission line inspections. Drones offer advantages such as flexible operation, rapid response, and wide coverage. They can overcome the limitations of complex terrain and harsh weather, enabling rapid, large-scale inspections of power transmission lines. However, existing drone inspection technology still faces some pressing challenges in ice detection.

[0005] First, most drone inspection systems rely on a single image source, typically using only visible light cameras for image acquisition. This single image source makes it difficult to accurately identify ice in poor lighting conditions or when the ice is similar in color to the background, severely limiting the accuracy and reliability of detection. Second, existing drone inspection systems generally lack effective image processing and analysis methods, making it difficult to accurately identify ice-covered areas in complex background environments. They can also easily confuse ice with other similar surface phenomena (such as frost and dew), leading to misjudgments. Furthermore, existing drone inspection systems also have significant deficiencies in data processing and analysis. They often require the collected image data to be transmitted to a ground station for processing, which not only increases data processing latency but also significantly limits the ability to monitor and respond in real time.

[0006] In view of the various shortcomings of the above-mentioned existing technologies, the present invention aims to propose a new real-time icing detection method based on drone multi-mode image fusion, so as to effectively improve the efficiency and accuracy of transmission line icing detection and provide more reliable technical support for the safe operation of the power system. Summary of the Invention

[0007] (1) Technical problems solved

[0008] In response to the shortcomings of the existing technology, the present invention provides a real-time ice detection method based on multi-mode image fusion of drones. The present invention mainly addresses the shortcomings of existing transmission line ice detection technology and is committed to solving the following problems:

[0009] Traditional manual inspections of transmission line icing are inefficient and pose safety risks in severe weather conditions. The data obtained also lacks real-time and accuracy.

[0010] Existing drone inspection systems mostly use a single visible light camera to capture images. When lighting conditions are poor or the ice cover is similar in color to the background, it is difficult to accurately identify the ice cover, which limits the accuracy and reliability of detection.

[0011] The lack of effective image processing and analysis methods makes it difficult to accurately identify ice-covered areas from complex backgrounds, and it is easy to confuse ice with other surface phenomena, leading to misjudgment.

[0012] Most existing transmission line icing detection systems lack real-time monitoring and automatic early warning functions, and are unable to provide immediate feedback and processing of icing conditions, making it difficult to meet the real-time requirements of modern power systems for transmission line status monitoring.

[0013] (2) Technical solution

[0014] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0015] A real-time ice detection method based on multi-mode image fusion of UAVs includes the following steps:

[0016] S1: Multimodal image acquisition: Using the infrared camera and visible light camera carried by the drone, infrared and visible light images of the transmission line are simultaneously collected. The infrared image is used to identify temperature differences, and the visible light image is used to provide visual texture information.

[0017] S2: Image preprocessing, performing cropping, scaling, rotation, flipping and contrast enhancement operations on the acquired images;

[0018] Image preprocessing is the necessary preprocessing operations on the acquired images to improve the efficiency and accuracy of subsequent processing, including:

[0019] Crop: Remove unnecessary parts from the edges of an image.

[0020] Scaling: Adjust the image resolution to fit the model input requirements.

[0021] Rotation and flipping: image augmentation operations that improve the generalization ability of the model.

[0022] Contrast Enhancement: Improves the visual quality of images and makes ice features more distinct.

[0023] S3: Multimodal image fusion, which fuses the preprocessed infrared image and visible light image at the pixel level to generate a fused image;

[0024] S4: Ice detection model construction, using a lightweight neural network to process fused images and build a lightweight detection model for feature extraction;

[0025] S5: Real-time ice detection and warning: The fused image is fed into a trained lightweight neural network to identify ice-covered areas and generate ice location information. When the ice-covered area exceeds a preset threshold, an automatic warning mechanism is triggered.

[0026] S6: System integration: integrating drone control, image acquisition, image fusion, data processing, model reasoning, and warning notification into the drone edge computing platform, and transmitting warning information to the ground control center in real time through the 5G module.

[0027] Furthermore, in step S1: the infrared camera and the visible light camera achieve millisecond-level synchronous acquisition through a hardware synchronization device;

[0028] The drone's flight altitude is 1.2-1.8 times the vertical distance of the transmission line, and its flight speed is 3-8m / s.

[0029] Furthermore, the contrast enhancement in the image preprocessing step S2 adopts a histogram equalization method or a histogram equalization method with adaptive contrast limitation to highlight the ice-covered features and make the contrast between the ice-covered area and the background area in the image more obvious.

[0030] Furthermore, in step S3, a dataset is constructed and enhanced to construct a dataset containing infrared and visible light images. Annotation tools are used to annotate the images to clearly distinguish between ice-covered and non-ice-covered areas. The dataset is expanded through data enhancement technology to enhance the robustness and generalization ability of the model.

[0031] Furthermore, the pixel-level fusion in step S3 specifically includes:

[0032] Extract temperature features from infrared images and generate temperature gradient maps;

[0033] Extract texture features from visible light images and generate edge enhancement images;

[0034] The temperature gradient map and the edge enhancement map are fused by weighted superposition, and the weight coefficient is dynamically adjusted according to the ambient light intensity.

[0035] Furthermore, the weighted superposition formula is:

[0036] ;

[0037] in: ; β is the light attenuation coefficient, lux is the ambient illumination value.

[0038] Furthermore, the data enhancement technology includes random rotation, random scaling, random cropping, and color transformation operations to simulate different environmental conditions and shooting angles to expand the diversity of the data set.

[0039] Furthermore, in step S4, the neural network includes:

[0040] Pixel fusion: fusion of infrared and visible light images at the pixel level;

[0041] Backbone network: SqueezeNet is used to replace the original backbone network of YOLOv8;

[0042] Dynamic upsampling module: DySample dynamic upsampling algorithm is used in the Neck part.

[0043] Furthermore, in step S4:

[0044] The DySample dynamic upsampling algorithm adaptively adjusts the upsampling kernel through learnable parameters;

[0045] The output layer of the neural network uses the Sigmoid activation function to generate the ice cover probability heat map.

[0046] Furthermore, the early warning mechanism of step S5 includes:

[0047] Ice-covered area calculation formula: ; where P ice >0.85 is considered as ice-covered pixels;

[0048] When A ice / A line When it is >15%, a level 3 warning is triggered.

[0049] (3) Beneficial effects

[0050] Compared with the existing technology, the present invention provides a real-time ice detection method based on multi-mode image fusion of UAVs, which has the following beneficial effects:

[0051] Image fusion enhances detection accuracy: By effectively fusing infrared and visible light images, the model leverages the strengths of both, enabling it to more accurately distinguish between ice-covered and non-ice-covered areas. This image fusion technology significantly improves detection accuracy, effectively reducing misjudgments and missed detections, and providing a strong foundation for precise monitoring of transmission lines, particularly in conditions with poor lighting or when the ice and background are similar in color.

[0052] Efficient lightweight model operation: The lightweight SqueezeNet network is used as the backbone for feature extraction. The Neck component of YOLOv8 is optimized, and the DySample dynamic upsampling method is introduced. These optimizations significantly reduce the model's computational complexity and resource consumption, making it suitable for efficient operation on drone platforms with relatively limited computing resources. This not only ensures the model's real-time processing capabilities on drones, enabling rapid analysis and judgment of captured images, but also reduces the drone's energy requirements, extending its flight time and improving the system's practicality and cost-effectiveness.

[0053] Real-time monitoring and automatic warnings improve response speed: This invention deeply integrates an icing detection model with a drone platform to create a real-time monitoring and automatic warning system. This system instantly issues a warning signal upon detecting icing, promptly notifying maintenance personnel and enabling them to respond quickly, take effective preventive measures, or perform timely repairs. This feature significantly improves response speed to icing conditions, effectively reducing the risk of grid failures caused by icing, effectively ensuring reliable operation and stable power supply, and minimizing the economic losses and social impact of power outages. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 Schematic diagram of the system flow of the present invention;

[0055] Figure 2 This is a flow chart of the multimodal image fusion of the present invention;

[0056] Figure 3 Schematic diagram of the process steps of the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0058] like Figure 1 、 Figure 2 and Figure 3 As shown, an embodiment of the present invention proposes a real-time ice detection method based on multi-mode image fusion of drones, comprising the following steps:

[0059] S1: Multimodal image acquisition: Using the infrared camera and visible light camera carried by the drone, infrared and visible light images of the transmission line are simultaneously collected. The infrared image is used to identify temperature differences, and the visible light image is used to provide visual texture information.

[0060] S2: Image preprocessing, performing cropping, scaling, rotation, flipping, and contrast enhancement operations on the acquired images to improve the efficiency and accuracy of subsequent processing;

[0061] S3: Multimodal image fusion, which fuses the preprocessed infrared image and visible light image at the pixel level to generate a fused image;

[0062] S4: Ice detection model construction, using a lightweight neural network to process fused images and build a lightweight detection model for feature extraction;

[0063] S5: Real-time ice detection and warning: The fused image is input into a trained lightweight neural network to identify ice-covered areas and generate ice location information; when the ice-covered area exceeds a preset threshold, an automatic warning mechanism is triggered.

[0064] The trained model is used to analyze the real-time captured fusion images to determine whether there is ice covering the transmission lines. Once ice covering is detected, the system automatically issues an early warning and notifies maintenance personnel to take timely measures.

[0065] S6: System Integration: This integrates drone control, image acquisition, image fusion, data processing, model reasoning, and warning notifications into the drone edge computing platform. This system transmits warning information to the ground control center in real time via a 5G module. This integration ensures stable operation in various environments and enables real-time monitoring of transmission line ice coverage.

[0066] like Figure 2 As shown, in some embodiments, in step S1: the infrared camera and the visible light camera achieve millisecond-level synchronous acquisition through a hardware synchronization device;

[0067] The drone's flight altitude is 1.2-1.8 times the vertical distance of the transmission line, and its flight speed is 3-8m / s.

[0068] like Figure 2As shown, in some embodiments, the contrast enhancement in the image preprocessing step S2 adopts a histogram equalization method or a histogram equalization method with adaptive contrast limitation to highlight the ice-covered features and make the contrast between the ice-covered area and the background area in the image more obvious.

[0069] like Figure 2 and Figure 3 As shown, in some embodiments, the step S3 performs data set construction and enhancement, constructs a data set including infrared and visible light images, uses annotation tools (such as LabelImg) to annotate images, and clearly distinguishes ice-covered areas from non-ice-covered areas; expands the data set through data enhancement technology to enhance the robustness and generalization ability of the model.

[0070] like Figure 1 and Figure 3 As shown, in some embodiments, the pixel-level fusion in step S3 specifically includes:

[0071] Extract temperature features from infrared images and generate temperature gradient maps;

[0072] Extract texture features from visible light images and generate edge enhancement images;

[0073] The temperature gradient map and the edge enhancement map are fused by weighted superposition, and the weight coefficient is dynamically adjusted according to the ambient light intensity.

[0074] In some embodiments, the weighted superposition formula is:

[0075] ;

[0076] in: ; β is the light attenuation coefficient, lux is the ambient illumination value.

[0077] like Figure 1 and Figure 3 As shown, in some embodiments, the data enhancement technology includes random rotation, random scaling, random cropping, and color transformation operations to simulate different environmental conditions and shooting angles to expand the diversity of the data set.

[0078] like Figure 1 and Figure 3 As shown, in some embodiments, in step S4, the neural network includes:

[0079] Pixel fusion: Fusing infrared and visible light images at the pixel level to enhance ice features in the image;

[0080] Backbone network: SqueezeNet is used to replace the original backbone network of YOLOv8 to reduce computational complexity and increase the running speed of the model;

[0081] Dynamic upsampling module: The DySample dynamic upsampling algorithm is used in the Neck part to adapt to image inputs of different resolutions, improving the adaptability and detection accuracy of the model;

[0082] like Figure 1 and Figure 3 As shown, in some embodiments, in step S4:

[0083] The DySample dynamic upsampling algorithm adaptively adjusts the upsampling kernel through learnable parameters;

[0084] The output layer of the neural network uses the Sigmoid activation function to generate the ice cover probability heat map.

[0085] like Figure 1 and Figure 3 As shown, in some embodiments, in step S4: during the construction of the lightweight detection model, pixel fusion adopts weighted fusion or feature-level fusion to fuse the pixel values ​​of the infrared image and the visible light image according to certain weights to enhance the ice cover features in the image.

[0086] During the construction of the lightweight detection model, network architecture optimization also includes pruning and quantization operations on the model to further reduce the number of model parameters and computational complexity to adapt to the computing resource limitations of the UAV platform.

[0087] like Figure 1 、 Figure 2 and Figure 3 As shown, in some embodiments, the early warning mechanism of step S5 includes:

[0088] Ice-covered area calculation formula: ; where P ice >0.85 is considered as ice-covered pixels;

[0089] When A ice / A line When it is >15%, a level 3 warning is triggered.

[0090] like Figure 1 and Figure 3 As shown, in some embodiments, the warning of step S5, in the ice detection and warning system, the warning notification includes but is not limited to sending a text message to the maintenance personnel, pushing an application message or triggering an alarm sound to promptly remind the maintenance personnel to take measures.

[0091] Example: Figure 1 、 Figure 2 and Figure 3As shown, the real-time icing detection method based on UAV multimodal image fusion mainly includes a UAV platform, a multimodal camera system, an image preprocessing module, a data set construction and management module, a lightweight detection model, an icing detection and warning module, and a system integration and control module.

[0092] First, a drone equipped with a multimodal camera system flies near a power transmission line, capturing both infrared and visible light images of the line according to a pre-set flight path and attitude. The infrared and visible light cameras in the multimodal camera system are precisely synchronized and calibrated to ensure temporal and spatial consistency between the two captured images, providing reliable data for subsequent image fusion.

[0093] The captured infrared and visible light images are then transferred to the image preprocessing module. This module first crops the images, removing irrelevant image edges based on the transmission line's location and the pre-set monitoring range, while retaining the image content of the transmission line and its surrounding critical areas. The cropped images are then scaled to a resolution that meets the input requirements of the lightweight detection model, for example, by uniformly scaling the images to 416×416 pixels. This ensures that the model can process the images effectively, quickly, and accurately.

[0094] The scaled images are then subjected to image enhancement operations such as rotation and flipping. Specifically, the images are randomly rotated by 0°, 90°, 180°, and 270°, and flipped horizontally and vertically. This generates image samples at various angles and postures, expanding the dataset. Contrast enhancement is also performed on the images, using methods such as histogram equalization to increase the contrast between ice-covered and non-ice-covered areas. This makes ice features more prominent, facilitating subsequent model detection and recognition.

[0095] The preprocessed images are transferred to the dataset construction and management module. In this module, professionals use the professional labeling tool LabelImg to accurately label the ice-covered and non-ice-covered areas in the images. A corresponding labeling file is generated for each image sample, clearly indicating the location, shape, and range of the ice. These labeled image samples constitute the initial dataset for training the lightweight detection model. To further expand the dataset and enhance the model's robustness and generalization capabilities, data augmentation techniques such as random rotation, random scaling, and color jittering are applied to the initial dataset to generate a large number of enhanced image samples, constructing a rich, diverse, and representative training dataset.

[0096] In the lightweight detection model construction module, a lightweight neural network architecture is used to build the detection model. First, the preprocessed infrared image and visible light image are fused at the pixel level. Using a pixel fusion algorithm based on weighted averaging, the pixel values ​​of the two images are weighted and fused according to their importance in ice detection to generate fused image features. The weights can be determined through analysis and experimental verification of a large number of sample images. For example, experiments have found that under certain specific environmental conditions, the weight of the infrared image in ice detection can be set to 0.6, while the weight of the visible light image can be set to 0.4. This achieves a reasonable integration of the two image features and enhances the ice feature information in the image.

[0097] Next, the SqueezeNet network was selected as the backbone network for feature extraction, replacing the YOLOv8 backbone network. By employing unique designs such as the FireModule, the SqueezeNet network significantly reduces the number of network parameters and computational complexity while maintaining sufficient model performance, making it more suitable for resource-constrained drone platforms. Building on the established SqueezeNet backbone network, the Neck portion of YOLOv8 was optimized, and the DySample dynamic upsampling method was introduced to replace traditional upsampling methods. DySample dynamic upsampling adaptively adjusts the upsampling strategy by learning the mapping relationship between image features at different resolutions. Based on the input image features at different scales, it dynamically generates corresponding high-resolution feature maps, thereby better adapting to the multi-scale characteristics of transmission line ice-covered images and improving the model's detection accuracy.

[0098] The lightweight detection model constructed and optimized as described above was trained using the constructed training dataset. During training, optimization algorithms such as stochastic gradient descent (SGD) were employed, combined with loss functions such as the cross-entropy loss function, to continuously adjust and optimize the model parameters. This enabled the model to effectively represent ice features in both infrared and visible light images and accurately distinguish between ice-covered and non-ice-covered areas. During training, appropriate hyperparameters such as the learning rate and number of iterations, as well as strategies such as early stopping, were set to prevent overfitting and ensure good performance on both the training and validation sets.

[0099] The trained lightweight detection model is integrated into the icing detection and early warning module. During actual transmission line inspections, drones collect real-time infrared and visible light images of the transmission lines. After image preprocessing and pixel fusion, the fused images are input into the lightweight detection model. The model quickly analyzes and determines the icing conditions in the images. Once icing is detected, an early warning mechanism is triggered. This warning is transmitted via wireless communication modules (such as 4G / 5G networks) to the power system monitoring center and maintenance personnel's devices (such as mobile phones and computers). Detailed information, including the location and severity of icing, is displayed on the drone's onboard display, allowing maintenance personnel to promptly understand the situation and respond quickly.

[0100] like Figure 2 As shown in the figure, the multimodal image fusion process is as follows: the multimodal camera system carried by the UAV synchronously collects the infrared image (I) and visible light image (V) of the transmission line, and then performs preprocessing operations (including cropping, scaling, rotation, flipping and contrast enhancement) on the two images to obtain the preprocessed infrared image (I') and visible light image (V');

[0101] Next, a pixel fusion algorithm is used to fuse I' and V' at the pixel level to generate a fused image (F). This fused image F is input into the lightweight detection model, which performs a series of operations, including feature extraction and feature fusion, to ultimately output the detection results (including the presence, location, and extent of ice cover). If ice cover is detected, an early warning signal (A) is immediately issued. If no ice cover is detected, the next round of image acquisition and detection continues.

[0102] During system operation, the system integration and control module coordinates communication and data transmission between various functional modules, ensuring seamless and efficient operation of all aspects, including image acquisition, preprocessing, detection and analysis, and early warning notifications. This module also monitors and controls the drone's flight status, adjusting its flight attitude and path based on pre-set missions and real-time environmental information to ensure the drone can stably and safely complete its transmission line inspection mission.

[0103] The present invention's real-time icing detection method based on multimodal drone image fusion integrates a multimodal camera system and a lightweight detection model on a drone platform. Utilizing image fusion technology and advanced deep learning algorithms, it achieves rapid and accurate detection of icing on transmission lines, providing real-time monitoring and automatic early warning capabilities. Practical application testing has demonstrated that this method operates stably under various complex environmental conditions, significantly improving icing detection accuracy compared to traditional methods. This method effectively reduces the risks and costs of manual inspections, providing strong technical support and assurance for the safe and stable operation of power systems.

[0104] The innovative points of this application are summarized as follows:

[0105] An innovative multimodal image fusion technology is proposed to effectively fuse infrared images and visible light images, fully leveraging the advantages of the two images in temperature difference recognition and visual texture presentation. It significantly enhances the ability to recognize ice features and effectively solves the problem of limited accuracy of a single image source in ice detection. The detection performance advantage is obvious, especially in poor lighting conditions or when the ice is similar in color to the background.

[0106] A lightweight neural network architecture was constructed, using SqueezeNet as the backbone for feature extraction. The Neck component of YOLOv8 was optimized, and the DySample dynamic upsampling method was introduced. This innovative network architecture significantly reduced the model's computational complexity and resource consumption while maintaining detection accuracy, enabling efficient operation on resource-constrained drone platforms. This enabled real-time ice detection, reduced the drone's energy requirements, and improved the system's practicality and cost-effectiveness.

[0107] For the first time, an icing detection model has been deeply integrated with a drone platform to create a real-time monitoring and automatic warning system. This system provides continuous, real-time monitoring of transmission lines. Once icing is detected, it automatically issues a warning signal, notifying maintenance personnel to take timely action. This effectively improves response speed to icing conditions, significantly reduces the risk of grid failures caused by icing, and ensures reliable operation and stable power supply. The system has significant practical application value and broad prospects.

[0108] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A real-time ice detection method based on multi-mode image fusion from drones, characterized by: The following steps are involved: S1: Multimodal image acquisition: Using the infrared camera and visible light camera carried by the drone, infrared and visible light images of the transmission line are simultaneously collected. The infrared image is used to identify temperature differences, and the visible light image is used to provide visual texture information. S2: Image preprocessing, performing cropping, scaling, rotation, flipping and contrast enhancement operations on the acquired images; S3: Multimodal image fusion, which fuses the preprocessed infrared image and visible light image at the pixel level to generate a fused image; S4: Ice detection model construction, using a lightweight neural network to process fused images and build a lightweight detection model for feature extraction; S5: Real-time ice detection and warning: The fused image is fed into a trained lightweight neural network to identify ice-covered areas and generate ice location information. When the ice-covered area exceeds a preset threshold, an automatic warning mechanism is triggered. S6: System integration: integrating drone control, image acquisition, image fusion, data processing, model reasoning, and warning notification into the drone edge computing platform, and transmitting warning information to the ground control center in real time through the 5G module.

2. The real-time ice detection method based on multi-mode image fusion of UAV according to claim 1 is characterized by: In the step S1: the infrared camera and the visible light camera achieve millisecond-level synchronous acquisition through a hardware synchronization device; The drone's flight altitude is 1.2-1.8 times the vertical distance of the transmission line, and its flight speed is 3-8m / s.

3. The real-time ice detection method based on multi-mode image fusion from drones according to claim 1 is characterized by: The contrast enhancement in the image preprocessing step S2 adopts a histogram equalization method or a histogram equalization method with adaptive contrast limitation to highlight the ice-covered features and make the contrast between the ice-covered area and the background area in the image more obvious.

4. The real-time ice detection method based on multi-mode image fusion from drones according to claim 1 is characterized by: In step S3, a data set is constructed and enhanced. A data set including infrared and visible light images is constructed, and image annotation tools are used to clearly distinguish between ice-covered areas and non-ice-covered areas. The data set is expanded through data enhancement technology to enhance the robustness and generalization ability of the model.

5. The real-time ice detection method based on multi-mode image fusion of UAV according to claim 4 is characterized by: The pixel-level fusion in step S3 specifically includes: Extract temperature features from infrared images and generate temperature gradient maps; Extract texture features from visible light images and generate edge enhancement images; The temperature gradient map and the edge enhancement map are fused by weighted superposition, and the weight coefficient is dynamically adjusted according to the ambient light intensity.

6. The real-time ice detection method based on multi-mode image fusion of UAV according to claim 5 is characterized by: The weighted superposition formula is: ； in: ; β is the light attenuation coefficient, lux is the ambient illumination value.

7. The real-time ice detection method based on multi-mode image fusion of UAV according to claim 4 is characterized by: The data enhancement technology includes random rotation, random scaling, random cropping, and color transformation operations to simulate different environmental conditions and shooting angles and expand the diversity of the data set.

8. The real-time ice detection method based on multi-mode image fusion of UAV according to claim 1 is characterized by: In step S4, the neural network includes: Pixel fusion: fusion of infrared and visible light images at the pixel level; Backbone network: SqueezeNet is used to replace the original backbone network of YOLOv8; Dynamic upsampling module: DySample dynamic upsampling algorithm is used in the Neck part.

9. The real-time ice detection method based on multi-mode image fusion of UAV according to claim 6 is characterized by: In the step S4: The DySample dynamic upsampling algorithm adaptively adjusts the upsampling kernel through learnable parameters; The output layer of the neural network uses the Sigmoid activation function to generate the ice cover probability heat map.

10. The real-time ice detection method based on multi-mode image fusion of UAV according to claim 1 is characterized by: The early warning mechanism of step S5 includes: Ice-covered area calculation formula: ; where P ice >0.85 is considered as ice-covered pixels; When A ice / A line When it is >15%, a level 3 warning is triggered.

Images