Cable hidden danger identification method, system and device based on multi-modal difference fusion and medium

Abstract

The invention discloses a cable hidden danger identification method, system and device based on multi-modal difference fusion and a medium, and the method comprises the steps: collecting an initial infrared image and an initial visible light image of a cable, and carrying out the preprocessing of the initial infrared image and the initial visible light image, and obtaining a standard infrared image and a standard visible light image; performing modal feature extraction on the standard infrared image and the standard visible light image through an efficient channel attention module to obtain an infrared modal feature and a visible light modal feature; differential operation is carried out on the infrared modal features and the visible light modal features to obtain modal difference features, and reweighted fusion is carried out on the modal difference features to obtain a fusion feature map; and inputting the feature fusion image into a target detection network to obtain cable hidden danger information, and generating a cable hidden danger identification result according to the cable hidden danger information. The thermal information of the infrared image and the structure detail information of the visible light image are fused, and the limitation of existing single-mode detection is broken through.

Description

Technical Field

[0001] This invention relates to the field of power equipment hazard identification technology, and in particular to a cable hazard identification method, system, equipment and medium based on multimodal difference fusion. Background Technology

[0002] Cable lines are a critical infrastructure of modern urban power grids, and their operational stability directly affects power supply reliability and public safety. With the acceleration of urbanization, the scale of cable laying is becoming increasingly large and the structure increasingly complex, and many are distributed in hidden spaces such as underground utility tunnels and tunnels. This places extremely high demands on the comprehensiveness, accuracy, and timeliness of operation and maintenance inspections.

[0003] Currently, cable hazard detection mainly relies on single-type image sensors. Infrared imaging-based detection methods can effectively capture abnormal temperature rises in cables, but they are insensitive to structural information such as texture and morphology, and are prone to false alarms under complex thermal background interference. While visible light imaging-based methods can clearly present the physical details of the cable surface, they are significantly affected by ambient lighting, with performance deteriorating sharply at night, in strong backlight, or in rainy or foggy weather. Both of these single-modal methods have inherent limitations, making it difficult to achieve all-weather, accurate cable hazard identification in real, complex operation and maintenance scenarios, thus hindering further improvements in intelligent operation and maintenance. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a cable hazard identification method, system, device, and medium based on multimodal difference fusion to solve the problem that existing single-modal detection methods have inherent limitations and cannot reliably perceive and identify multiple cable hazards under all-weather operating conditions.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a cable hazard identification method based on multimodal difference fusion, comprising: acquiring an initial infrared image and an initial visible light image of a cable, and preprocessing the initial infrared image and the initial visible light image respectively to obtain a standard infrared image and a standard visible light image; extracting modal features from the standard infrared image and the standard visible light image respectively through an efficient channel attention module to obtain infrared modal features and visible light modal features; performing differential operations on the infrared modal features and the visible light modal features to obtain modal difference features, and performing reweighted fusion on the modal difference features to obtain a fused feature map; inputting the feature fusion map into a target detection network to obtain cable hazard information, generating a cable hazard identification result based on the cable hazard information, and visually displaying the cable hazard identification result.

[0008] As a preferred embodiment of the cable hazard identification method based on multimodal difference fusion described in this invention, the steps of obtaining a standard infrared image and a standard visible light image include: performing image registration on the initial infrared image and the initial visible light image respectively to obtain a registered infrared image and a registered visible light image; performing image enhancement and radiometric correction on the registered infrared image to obtain the standard infrared image; and performing image enhancement on the registered visible light image to obtain the standard visible light image.

[0009] The beneficial effects of this preferred technical solution are as follows: image registration aligns the initial infrared image and the initial visible light image in spatial coordinates, ensuring consistency of image data at the pixel level across different modalities; image enhancement processing of the registered infrared image improves the contrast between hot spot areas and the background and suppresses noise interference, while radiometric correction converts image grayscale values ​​into absolute temperature information, giving the standard infrared image clear physical dimensions; image enhancement processing of the registered visible light image improves image quality and enhances texture details; after preprocessing, the standard infrared image and the standard visible light image achieve a unified standard in spatial location and feature quality, forming standardized multimodal input data. This standardization effectively eliminates the impact of sensor differences and environmental interference on subsequent analysis, ensuring the accurate execution of multimodal feature extraction and fusion operations.

[0010] As a preferred embodiment of the cable hazard identification method based on multimodal difference fusion described in this invention, the steps of obtaining infrared modal features and visible light modal features include: performing depthwise separable convolution and channel shuffling operations on both the standard infrared image and the standard visible light image to obtain preliminary infrared features and preliminary visible light features; applying a channel attention mechanism to the preliminary infrared features and the preliminary visible light features respectively to obtain infrared attention-weighted features and visible light attention-weighted features; performing a residual concatenation between the infrared attention-weighted features and the standard infrared image to obtain the infrared modal features; and performing a residual concatenation between the visible light attention-weighted features and the standard visible light image to obtain the visible light modal features.

[0011] The beneficial effects of this preferred technical solution are as follows: By performing depthwise separable convolution operations on standard infrared and standard visible light images, the computational complexity is reduced while maintaining the modal feature extraction capability; the subsequent channel shuffling operation promotes the interaction of information between different channel groups and enhances the expressive power of modal features; channel attention mechanisms are applied to the obtained preliminary infrared and visible light features respectively, enabling the efficient channel attention module to focus on key channel features related to cable hazards and suppress interference from irrelevant information; residual connection is performed between the generated infrared attention-weighted features and the standard infrared image, which not only preserves the radiation characteristic information of the original image but also avoids the gradient vanishing problem in deep network training; similarly, residual connection is performed between the visible light attention-weighted features and the standard visible light image, maintaining the integrity of texture details.

[0012] As a preferred embodiment of the cable hazard identification method based on multimodal difference fusion described in this invention, the step of performing a difference operation on the infrared modal features and the visible light modal features to obtain modal difference features includes: aligning the infrared modal features and the visible light modal features in the channel dimension; and subtracting the infrared modal features and the visible light modal features element-wise after channel dimension alignment to obtain the modal difference features.

[0013] The beneficial effects of this preferred technical solution are as follows: By aligning the infrared modal features and visible light modal features along the channel dimension, the calculation deviation that may be caused by the inconsistent number of channels in different modal feature maps is eliminated; by performing element-wise subtraction of the two modal features after channel dimension alignment, the difference regions between infrared thermal radiation information and visible light texture information can be directly highlighted. This difference feature can effectively capture potential hidden dangers that are not obvious in a single modality, such as thermal anomalies that only appear in infrared images or structural defects that appear in visible light images; the obtained modal difference features explicitly encode the complementarity and contradiction between multimodal data through mathematical operations, providing a more discriminative feature representation for the recognition process than a single modality.

[0014] As a preferred embodiment of the cable hazard identification method based on multimodal difference fusion described in this invention, the step of reweighting and fusing the modal difference features to obtain a fused feature map includes: performing global average pooling on the modal difference features in the horizontal and vertical directions respectively to obtain horizontal global features and vertical global features; concatenating the horizontal global features and vertical global features, and performing feature transformation through a shared convolutional layer to obtain difference context features; multiplying the difference context features and the modal difference features element-wise to obtain weighted difference features; and applying a channel attention mechanism to the weighted difference features to obtain the fused feature map.

[0015] The beneficial effects of this preferred technical solution are as follows: By performing global average pooling operations on the modal difference features in both the horizontal and vertical directions, the long-range dependencies of the difference features in the spatial dimension can be captured, resulting in direction-aware global contextual information; by concatenating the horizontal and vertical global features and performing feature transformation through a shared convolutional layer, the fusion and dimensionality reduction of contextual information in different directions are achieved, resulting in compact difference contextual features; by multiplying the difference contextual features element-wise with the original modal difference features, the model can adjust the local difference features according to the global contextual information, enhancing the feature response of salient regions; finally, a channel attention mechanism is applied to the weighted difference features, and by recalibrating the weight relationship of each channel, the contribution of important channels is highlighted. The resulting fused feature map retains detailed difference information and incorporates global contextual understanding, forming a more discriminative feature representation.

[0016] As a preferred embodiment of the cable hazard identification method based on multimodal difference fusion described in this invention, the step of inputting the fused feature map into the target detection network to obtain cable hazard information includes: inputting the fused feature map into the backbone network of the target detection network for multi-scale feature extraction to obtain multi-scale backbone features; fusing the multi-scale backbone features with multi-scale context features to obtain enhanced context features; inputting the enhanced context features into the multi-scale detection head of the target detection network for prediction to obtain the corresponding bounding box information of targets at different scales; decoding the bounding box information to obtain multiple bounding boxes, parsing each bounding box as a cable hazard target, and outputting the location coordinates, confidence level, and category of each cable hazard target as the cable hazard information.

[0017] The beneficial effects of this preferred technical solution are as follows: By inputting the fused feature map into the backbone network of the target detection network for multi-scale feature extraction, it is possible to simultaneously capture the local details and global semantic information of cable hazards, forming multi-scale backbone features; by fusing multi-scale contextual features into these multi-scale backbone features, it is possible to integrate feature information from different receptive fields, giving the enhanced contextual features richer spatial hierarchical relationships; by inputting the enhanced contextual features into the multi-scale detection head for prediction, the network can process target detection tasks of different scales in parallel, effectively covering various hazard forms from small cable surface defects to large overheated areas; by decoding the bounding box information generated by the prediction, the bounding box information is transformed into structured cable hazard target data, and the final output location coordinates, confidence level, and category information constitute a complete cable hazard identification result, providing a directly operable target location and classification basis for cable operation and maintenance.

[0018] As a preferred embodiment of the cable hazard identification method based on multimodal difference fusion described in this invention, the step of generating a cable hazard identification result based on the cable hazard information includes: traversing the cable hazard targets in the cable hazard information; querying the corresponding alarm threshold in a preset database according to the category of the current cable hazard target; comparing the alarm threshold with the confidence level of the current cable hazard target; when the confidence level is not lower than the alarm threshold, determining the current cable hazard target as a target to be alarmed; mapping the position coordinates of the target to be alarmed to a physical space coordinate system to obtain physical location information; and fusing the physical location information with the category in the current target to be alarmed to generate the cable hazard identification result.

[0019] The beneficial effects of this preferred technical solution are as follows: By traversing the cable hazard targets in the cable hazard information, each detected potential hazard is ensured to be independently evaluated; based on the category of the current cable hazard target, the corresponding alarm threshold is queried in the preset database, realizing a differentiated judgment strategy based on hazard type; the obtained alarm threshold is compared with the confidence level of the current cable hazard target, and when the confidence level is not lower than the alarm threshold, the target is judged as a target to be alarmed, establishing a screening mechanism based on statistical confidence level; the location coordinates of the target to be alarmed are mapped to the physical space coordinate system, transforming the two-dimensional image coordinates into physical location information in the actual three-dimensional space; finally, the physical location information and the corresponding hazard category are integrated to generate cable hazard identification results, forming structured output data that includes spatial positioning and type determination, providing a decision-making basis for cable operation and maintenance management that can be directly located and handled.

[0020] Secondly, the present invention provides a cable hazard identification system based on multimodal difference fusion, comprising:

[0021] Multimodal image acquisition module: used to simultaneously acquire the initial infrared and initial visible light images of the cable;

[0022] Image preprocessing module: used to preprocess the initial infrared image and the initial visible light image to obtain a standard infrared image and a standard visible light image;

[0023] Multimodal feature fusion module: used to extract modal features from the standard infrared image and the standard visible light image respectively through the efficient channel attention module, to obtain infrared modal features and visible light modal features, and to perform differential operation and reweighted fusion of the infrared modal features and visible light modal features to obtain a fused feature map;

[0024] Target detection module: used to input the fused feature map into the target detection network to obtain cable hazard information;

[0025] Result generation and alarm module: used to generate cable hazard identification results based on the cable hazard information, and to display them visually or trigger alarms.

[0026] Thirdly, the present invention provides an electronic device, comprising:

[0027] Memory and processor;

[0028] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the cable hazard identification method based on multimodal difference fusion.

[0029] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the cable hazard identification method based on multimodal difference fusion.

[0030] Compared with existing technologies, the beneficial effects of this invention are as follows: By fusing thermal information from infrared images with structural detail information from visible light images, it overcomes the limitations of existing single-modal detection, solves the problems of single infrared imaging being insensitive to texture and morphological information and prone to false alarms, and single visible light imaging being greatly affected by lighting conditions and experiencing performance degradation at night or in severe weather. It can reliably perceive and identify cable hazards in complex environments such as nighttime, strong backlight, rain, and fog, meeting all-weather maintenance needs. The efficient channel attention module enhances the feature extraction capability for hot spots, damage, and obstructed areas, and the cross-modal difference enhancement module highlights the differences between infrared and visible light features, improving the identification accuracy of various hazards such as overheated cable areas, sheath cracks, exposed metal, and loose joints. This effectively reduces false alarm and false negative rates and demonstrates stronger robustness under complex background interference. During feature extraction and fusion, designs such as depthwise separable convolution, channel shuffling, and residual connection enhance the perception of fine structures such as cable textures and slit morphology. At the same time, it can locate potential hazards in cables of different sizes and types, solving the problem of insufficient identification of small targets and fine structural hazards in cables by existing technologies. Attached Figure Description

[0031] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram of the overall process of a cable hazard identification method based on multimodal difference fusion according to an embodiment of the present invention. Detailed Implementation

[0033] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0034] Example 1, referring to Figure 1As an embodiment of the present invention, a cable hazard identification method based on multimodal difference fusion is provided, including steps S100 to S400:

[0035] S100: Acquire the initial infrared image and initial visible light image of the cable, and preprocess the initial infrared image and initial visible light image respectively to obtain the standard infrared image and standard visible light image.

[0036] S200: The high-efficiency channel attention module extracts modal features from standard infrared and standard visible light images respectively, and obtains infrared modal features and visible light modal features.

[0037] S300. Perform differential operations on the infrared modal features and visible light modal features to obtain modal difference features, and then perform reweighted fusion on the modal difference features to obtain a fused feature map.

[0038] S400: Input the feature fusion map into the target detection network to obtain cable hazard information, generate cable hazard identification results based on the cable hazard information, and visualize the cable hazard identification results.

[0039] It should be noted that cables are widely distributed in power systems and are prone to hidden dangers during operation due to insulation aging, overload, and external damage. Infrared images can reflect thermal anomalies in cables, while visible light images can reveal surface details. However, the two have significant modal differences, and relying solely on a single modality or using an inappropriate fusion method can lead to insufficient extraction of hazard features. Furthermore, external environmental factors such as temperature and lighting conditions significantly interfere with image quality, and inadequate preprocessing can further affect subsequent identification results. Traditional hazard identification methods lack the specificity to fuse multimodal differences, easily resulting in missed or false detections of hazards. Moreover, the visualization of the identification results is low, making it difficult to quickly and accurately locate hazards and assess their severity. Therefore, the need for accurate identification and visualization of cable hazards is extremely urgent.

[0040] Therefore, to address the aforementioned problems of difficulty in multimodal difference fusion, significant environmental interference, and insufficient recognition accuracy and visualization, steps S100~S400 are used to preprocess the infrared and visible light images of the cable, extract and fuse modal difference features, and utilize a target detection network to obtain cable hazard information and visualize it. This achieves effective fusion of multimodal differences, improves the sufficiency and accuracy of cable hazard feature extraction, resolves the impact of environmental interference on image quality, and reduces the rate of missed and false detections of hazards. Simultaneously, the visualization of hazards provides a reliable basis for rapid location, severity assessment, and handling of cable hazards, ensuring the safe and stable operation of the cable.

[0041] Example 2, refer to Figure 1As an embodiment of the present invention, based on the above embodiment, a cable hazard identification method based on multimodal difference fusion is provided.

[0042] In this embodiment of the application, the steps of obtaining the standard infrared image and the standard visible light image in step S100 include A1~A3:

[0043] A1. Perform image registration on the initial infrared image and the initial visible light image respectively to obtain the registered infrared image and the registered visible light image;

[0044] In this embodiment, a synchronously triggered dual-modal imaging system is used to acquire initial infrared and initial visible light images. The infrared camera has a resolution of 512×512, and the visible light camera has a resolution of 1200×1024. Feature points are extracted and matched from the initial infrared and initial visible light images using a keypoint matching method based on the SIFT algorithm. An affine transformation matrix is ​​then calculated, and the initial visible light image is spatially transformed using the affine transformation matrix to align it with the initial infrared image in space. Finally, a registered infrared image and a registered visible light image are output.

[0045] A2. Perform image enhancement and radiometric correction on the registered infrared image to obtain a standard infrared image;

[0046] First, the CLAHE algorithm is used to enhance the registered infrared image output in step A1, improving the contrast between the hotspot area and the background. Next, bilateral filtering is used to suppress noise in the enhanced registered infrared image while preserving edge information of the temperature distribution. Finally, based on the radiometric correction formula, the 16-bit digital quantization values ​​acquired by the infrared camera are converted into a temperature matrix, completing the radiometric correction process and outputting a standard infrared image containing absolute temperature information. The radiometric correction formula is expressed as:

[0047] ;

[0048] In the formula, T is the output temperature value (unit: °C), and G is the original 16-bit grayscale value acquired by the infrared camera (value range: 0~65535). This is the digital quantization value corresponding to the maximum effective grayscale range. To preset the temperature measurement range, This serves as the starting reference value for the temperature measurement range. The infrared camera used in this embodiment outputs a 16-bit digital signal, therefore... Set to 16383 (corresponding to the effective range of 0~16383). Set to 40℃ Set to 10℃, and the temperature measurement range will be 10℃~50℃.

[0049] A3. Perform image enhancement on the registered visible light image to obtain a standard visible light image;

[0050] First, histogram equalization is performed on the registered visible light image output in step A1 to improve the overall image contrast. Then, Gaussian filtering is used to smooth and denoise the registered visible light image, suppressing noise while maintaining the integrity of the cable surface texture features. The processed registered visible light image is then integrated with the temperature matrix data in the standard infrared image output in step A2 by channel dimension. At the same time, the timestamp, ambient temperature and humidity, and wind speed metadata recorded during the acquisition process are also included to jointly construct a four-channel tensor containing multimodal information. Finally, a standard visible light image that can be used as network input is output.

[0051] In an optional implementation, the standard infrared image and standard visible light image obtained in step S100 can also adopt a multimodal image standardization process: The ORB feature detector is used to extract and match feature points in the initial infrared image and initial visible light image. After eliminating mismatches using the RANSAC algorithm, the homography transformation matrix is ​​calculated to achieve sub-pixel-level image registration, resulting in a registered dual-modal image. Guided filtering is applied to the registered infrared image for detail enhancement, and piecewise linear transformation is used to adjust the grayscale distribution. Subsequently, a grayscale-temperature mapping relationship is established using the double blackbody calibration method, and radiometric correction is completed using quadratic polynomial fitting to obtain the standard infrared image. The registered visible light image is processed using contrast-limited adaptive histogram equalization, combined with a nonlocal mean denoising algorithm to preserve texture features. Finally, the enhanced visible light image and the corrected infrared temperature information are stitched together in the channel dimension, incorporating GPS positioning data and environmental parameters collected by meteorological sensors to construct a standard infrared image and a standard visible light image with spatiotemporal consistency.

[0052] In another optional implementation, the standard infrared image and standard visible light image obtained in step S100 can also be rigidly registered using the phase correlation method. By calculating the cross power spectrum of the initial infrared image and the initial visible light image and locating the peak position, translation and rotation parameters are obtained to generate a registered image pair. Nonlinear contrast stretching and guided filtering are sequentially performed on the registered infrared image to enhance the sense of layering of thermal distribution features and maintain edge integrity. Then, the pixel values ​​are converted into a physical temperature field through a pre-established emissivity-temperature lookup table. The registered visible light image is subjected to illumination compensation using the Retinex algorithm and combined with nonlocal mean filtering to eliminate noise interference. Finally, the processed visible light image and the infrared temperature field are stitched together, and environmental parameters such as GPS positioning information and atmospheric transmittance are fused to form a standardized standard infrared image and standard visible light image output.

[0053] In this embodiment of the application, the step of obtaining infrared mode features and visible light mode features in step S200 includes B1 to B4:

[0054] B1. Perform depth-separable convolution and channel shuffling operations on both standard infrared and standard visible light images to obtain preliminary infrared and visible light features.

[0055] In this embodiment, spatial feature extraction is first performed using alternating 5×1 and 1×5 depthwise separable convolutional kernels. Standard infrared and standard visible light images use independent convolutional weights, and the output channel number of the depthwise separable convolution is set to 64. Subsequently, a 1×1 convolution is used to increase the channel dimension to 128. A channel shuffling operation is then performed on the obtained 128-dimensional features, uniformly dividing the feature channels into two groups and performing cross-recombination to promote information exchange between groups. Finally, preliminary infrared and visible light features with 128 channels are output respectively.

[0056] B2. Apply channel attention mechanisms to the preliminary infrared and visible light features respectively to obtain infrared attention-weighted features and visible light attention-weighted features;

[0057] First, the preliminary infrared and visible light features output in step B1 are used as feature parameters. Global average pooling is performed on each feature parameter, compressing the features of each channel in both height and width dimensions into a single scalar value, forming a 128-dimensional channel description vector. This channel description vector is then input into an attention mechanism consisting of two fully connected layers. The first fully connected layer reduces the dimension to 128 / r, where r is the compression ratio, set to 4 in this embodiment. This dimensionality reduction operation reduces the number of parameters in the first fully connected layer and enhances its nonlinear representation capability. The second fully connected layer restores the dimension to 128. Finally, a 128-dimensional channel attention weight vector is generated using the Sigmoid activation function. The channel attention weight vector is then multiplied channel-by-channel with the corresponding feature parameters to obtain the infrared attention-weighted features and the visible light attention-weighted features, respectively.

[0058] B3. Perform residual connection between the infrared attention-weighted features and the standard infrared image to obtain the infrared modal features;

[0059] First, a 1×1 convolution operation is performed on the standard infrared image to adjust the channel dimension of the standard infrared image to 128 dimensions, consistent with the infrared attention-weighted features. Then, the adjusted standard infrared image and the infrared attention-weighted features are added element-wise to complete the residual connection operation. The final output infrared modal features retain both the underlying temperature information of the standard infrared image and the semantic features of the deep network.

[0060] B4. Perform residual connection between the visible light attention-weighted features and the standard visible light image to obtain the visible light modal features;

[0061] Using the visible light attention-weighted features output in step B2 and the standard visible light image obtained in step A3 as input, and employing the same residual connection mechanism as in step B3, a 1×1 convolution operation is first performed on the standard visible light image to expand the three-channel RGB data of the standard visible light image to a 128-dimensional feature space. Then, the expanded visible light image features are added element-wise with the visible light attention-weighted features to complete residual fusion. The final output visible light modal features contain both the texture details of the standard visible light image and the semantic information extracted by the deep network.

[0062] In an optional implementation, obtaining infrared modal features and visible light modal features in step S200 can also be achieved by constructing independent infrared feature extraction paths and visible light feature extraction paths. In the infrared path, a combination of 7×1 and 1×7 asymmetric convolution kernels is used to extract temperature distribution features in the horizontal and vertical directions. In the visible light path, a combination of 5×5 and 3×3 symmetric convolution kernels is used to capture multi-scale texture features. The extracted features are then fed into a spatial-channel collaborative attention module, which simultaneously calculates the spatial attention map and channel attention weights. A joint attention map is generated through matrix outer product operations to weight the respective modal features. Then, a restricted cross-modal channel exchange mechanism is used to exchange information in the channel dimension between the weighted infrared features and the visible light features at a ratio of 15%. Finally, the original standard infrared image and the enhanced infrared features are fused through gated residual connections, and the original standard visible light image and the enhanced visible light features are also fused, forming independent infrared modal features and visible light modal features output respectively.

[0063] In this embodiment of the application, step S300, which involves performing a difference operation on the infrared modal features and the visible light modal features to obtain the modal difference features, includes steps C1 to C2:

[0064] C1. Align the infrared modal features with the visible light modal features in the channel dimension;

[0065] First, the spatial resolution of the infrared and visible light modal features is unified. The spatial size of the infrared modal features is adjusted to be consistent with that of the visible light modal features through bilinear interpolation. Then, a shared 1×1 convolutional layer is used to map the channel dimensions of the infrared and visible light modal features to ensure that the number of output channels for both is 128. Finally, the output channels are aligned for the infrared and visible light modal features.

[0066] C2. Subtract the infrared modal features after aligning the channel dimensions from the visible light modal features element by element to obtain the modal difference features;

[0067] The infrared modal features and visible light modal features are subtracted element-wise at their corresponding spatial locations and channel dimensions to obtain initial difference features. To further enhance the significance of the difference features, the absolute value of the initial difference features is processed, and the final output is a modal difference feature that can effectively characterize the complementary differences between the infrared modal features and the visible light modal features.

[0068] In this embodiment of the application, the step S300 of reweighting and fusing the modal difference features to obtain the fused feature map includes D1~D3:

[0069] D1. Perform global average pooling in the horizontal and vertical directions on the modal difference features to obtain horizontal global features and vertical global features.

[0070] First, a global average pooling operation is performed on the modal difference features obtained in step C2 along the horizontal direction, compressing the modal difference features into a single-column feature vector in the width dimension, thus obtaining the horizontal global features. At the same time, a global average pooling operation is performed on the modal difference features along the vertical direction, compressing the modal difference features into a single-row feature vector in the height dimension, thus obtaining the vertical global features. The horizontal global features and the vertical global features capture the global distribution characteristics of the difference features in the horizontal and vertical directions, respectively.

[0071] D2. Concatenate the horizontal global features and the vertical global features, and perform feature transformation through a shared convolutional layer to obtain differential context features;

[0072] First, the horizontal and vertical global features obtained in step D1 are concatenated along the channel dimension to form a joint feature containing bidirectional contextual information. Then, the joint feature is transformed through a shared 1×1 convolutional layer. In this embodiment, the number of input channels of the shared convolutional layer is 256 and the number of output channels is 128. The ReLU activation function is used to enhance the nonlinear expressive power, and the final output is a differential contextual feature that integrates bidirectional global context.

[0073] D3. Multiply the differential context features and modal differential features element-wise to obtain weighted differential features. Apply channel attention mechanism to the weighted differential features to obtain the fused feature map.

[0074] First, the differential context features obtained in step D2 are multiplied element-wise with the modal differential features obtained in step C2 to achieve context-aware weighting of the differential context features, resulting in weighted differential features. Then, a channel attention mechanism is applied to the weighted differential features. By recalibrating the weights between channels, the contribution of important channels is highlighted and the influence of redundant channels is suppressed, ultimately outputting a fusion feature map with higher discriminative power.

[0075] In an optional implementation, the modal difference features reweighted and fused to obtain the fused feature map in step S300 can also be processed using a parallel multi-scale convolutional group, which includes three different scales of depth separable convolutional kernels: 1×3, 3×1, and 3×3, to extract the difference features of orientation-sensitive and local regions, respectively. After concatenating the multi-scale output features along the channel dimension, a gated recurrent unit mechanism is used to control the fusion weights of the features at each scale, where the gate signal is generated by the global context features through a fully connected layer. Then, the fused features are spatially recalibrated by a spatial weighting module, which generates a two-dimensional spatial weight map by calculating the saliency score of each spatial location. Finally, a lightweight self-attention mechanism is used to optimize the channel dimension of the reweighted features, and feature recalibration is achieved by calculating the correlation matrix between channels, outputting an enhanced fused feature map.

[0076] In this embodiment of the application, the step S400 of inputting the feature fusion map into the target detection network to obtain cable hazard information includes E1~E4:

[0077] E1. Input the fused feature map into the backbone network of the target detection network to perform multi-scale feature extraction and obtain multi-scale backbone features;

[0078] The fused feature map output in step D3 is input into the backbone network of the object detection network. In this embodiment, the backbone network adopts a cross-stage partial structure. First, preliminary feature extraction is performed through convolutional layers with a kernel size of 3×3 and a stride of 2, outputting a feature map with 64 channels. Then, it passes through a two-stage feature transformation module. Each unit of the two-stage feature transformation module is composed of a 1×1 convolution, a 3×3 convolution, and a 1×1 convolution connected in series. The first 1×1 convolution is used to reduce the channel dimension, and the second 1×1 convolution is used to restore the channel dimension. By fusing shallow detail features and deep semantic features through cross-layer connections, the final output is a backbone feature map with three scales and different spatial resolutions, which are the original... Figure 1 Feature maps at scales of 8, 1 / 16, and 1 / 32 together constitute the multi-scale backbone features.

[0079] E2. Multi-scale backbone features are fused with multi-scale contextual features to obtain enhanced contextual features;

[0080] First, the backbone features of the three scales output in step E1 are input into the spatial pyramid pooling module. At each scale, four different sizes of max pooling kernels (1×1, 3×3, 5×5, and 7×7) are used in parallel to extract features. The features obtained from different receptive fields are concatenated in the channel dimension. Then, a 1×1 convolutional layer is used to compress the number of channels of the concatenated features to 128 dimensions, thereby refining and fusing cross-scale features. Finally, the enhanced contextual features containing contextual information are output.

[0081] E3. The enhanced contextual features are input into the multi-scale detection head of the target detection network for prediction to obtain the corresponding bounding box information of targets at different scales;

[0082] In this embodiment, the multi-scale detection head includes three output branches at different scales, corresponding to spatial resolutions of 128×128, 64×64, and 32×32, respectively. Each branch undergoes feature transformation through a 3×3 convolutional layer, and then outputs bounding box information through a 1×1 convolutional layer, including the bounding box center coordinates, width and height, confidence score, and class probability. The outputs at the three scales are responsible for detecting cable hazard targets of small, medium, and large sizes, respectively, and together they constitute a complete set of bounding box information.

[0083] E4. Decode the bounding box information to obtain multiple bounding boxes, parse each bounding box into a cable hazard target, and output the location coordinates, confidence level and category of each cable hazard target as cable hazard information;

[0084] First, the coordinate offset of the bounding box is decoded according to the preset anchor box size, and the relative coordinates are converted into absolute coordinates. The Sigmoid function is used to process the predicted absolute coordinates, and the processed predicted absolute coordinates are mapped to the original image space. Then, the Softmax function is applied to determine the hazard category to which the category probability of each bounding box belongs. Finally, the non-maximum suppression algorithm is used to remove redundant bounding boxes with high overlap, and each retained bounding box is parsed as a cable hazard target, outputting structured cable hazard information containing location coordinates, confidence level, and category.

[0085] In an optional implementation, step S400, where the feature fusion map is input into the target detection network to obtain cable hazard information, can also construct a feature extraction structure containing three parallel branches. These branches use dilated convolution, depthwise separable convolution, and standard convolution to process the input feature fusion map, respectively. The dilation rate of the dilated convolution is set to two modes: 2 and 4, to capture multi-scale contextual information. The outputs of the three branches are then fed into a feature selection module, which dynamically adjusts the fusion weights by calculating the saliency scores of each branch's features. Subsequently, a path aggregation network architecture is used to bidirectionally fuse deep semantic features with shallow detail features, achieving feature enhancement through bidirectional paths from top to bottom and bottom to top. Finally, a decoupled detection head is used to process classification and regression tasks, where the classification branch uses a focus loss function to optimize the sample imbalance problem, and the regression branch uses a GIoU loss function to improve the bounding box localization accuracy.

[0086] In this embodiment of the application, step S400, which generates cable hazard identification results based on cable hazard information, includes F1 to F4:

[0087] F1. Iterate through the cable hazard targets in the cable hazard information and query the corresponding alarm threshold in the preset database according to the current category of the cable hazard target.

[0088] Based on the category of the currently traversed cable hazard targets (including overheating, damage, foreign object attachment, etc.), the alarm threshold is queried in the preset database. The data in the preset database is stored in the form of key-value pairs, where the key is the category of the cable hazard target, and the value is the alarm threshold corresponding to the category after experimental calibration. For example, the threshold for the overheating category is 0.85, and the threshold for the damage category is 0.75.

[0089] F2. Compare the alarm threshold with the confidence level of the current cable hazard target. When the confidence level is not lower than the alarm threshold, the current cable hazard target is identified as a target to be alarmed.

[0090] The confidence level of the currently traversed cable hazard target is compared with the alarm threshold obtained in step F1. When the confidence level is greater than or equal to the alarm threshold, the current cable hazard target is determined to be a valid hazard and marked as a target to be alarmed. All targets to be alarmed that pass the judgment will be collected to form a set of targets to be alarmed.

[0091] F3. Map the location coordinates of the target to be alarmed to the physical space coordinate system to obtain the physical location information;

[0092] Using the inverse transformation of the affine transformation matrix obtained in the image registration stage in step A1, the position coordinates of each target to be alerted in the registered visible light image are mapped back to the coordinate system of the initial visible light image.

[0093] F4. Integrate physical location information with the category of the current target to be alarmed to generate cable hazard identification results;

[0094] The physical location information and category of each target to be alarmed are structured and integrated, and the original confidence level of the target to be alarmed is also included to form a complete identification record. Multiple identification records together constitute the final cable hazard identification result. The cable hazard identification result can be directly output to the cable operation and maintenance management system to trigger audible and visual alarms, generate maintenance work orders, or update the equipment status database.

[0095] In an optional implementation, the generation of cable hazard identification results based on cable hazard information in step S400 can also employ a spatiotemporal correlation decision fusion method: First, a spatiotemporal correlation model of the cable hazard target is established. The Kalman filter algorithm is used to track the trajectory of the cable hazard target in consecutive frames, and the detection results of the same cable hazard target at different times are correlated to form a spatiotemporal trajectory. Then, a multi-factor decision model is adopted, which, in addition to the confidence level of the cable hazard target, introduces three auxiliary decision factors: the size change rate, spatial distribution density, and duration of the cable hazard target. The size change rate is calculated by the area change of the target bounding box in adjacent frames, and the spatial distribution density is calculated by the DBSCAN clustering algorithm for multiple targets within the same frame. The distance relationships of individual cable hazard targets are analyzed, and the duration is obtained by statistically analyzing the time span of the cable hazard target trajectory. Next, a two-layer threshold decision mechanism is constructed. The first layer uses a fixed threshold for initial screening of confidence levels, and the second layer uses a regularized threshold to make a final decision based on the weighted scores of multiple factors. The regularized threshold is determined based on a mapping table established between environmental visibility and equipment operating load. Finally, a temporal fusion algorithm is used to integrate multiple detection results of the same hazard. The accuracy of the location coordinates is optimized through weighted averaging, and the final hazard category is determined through a majority voting mechanism. The output is a cable hazard identification result containing spatiotemporal consistency information, effectively solving the problems of instantaneous false detection and missed detection, and improving the stability and reliability of the identification results.

[0096] In summary, by fusing thermal information from infrared images with structural details from visible light images, this method overcomes the limitations of existing single-modal detection. It addresses the issues of single infrared imaging's insensitivity to texture and morphological information, leading to false alarms, and single visible light imaging's susceptibility to lighting conditions and performance degradation at night or in adverse weather. This enables reliable perception and identification of cable hazards in complex environments such as nighttime, strong backlight, rain, and fog, meeting all-weather maintenance requirements. The efficient channel attention module enhances feature extraction capabilities for hot spots, damage, and obstructed areas, while the cross-modal difference enhancement module highlights the differences between infrared and visible light features. This improves the accuracy of identifying various hazards such as overheated areas, sheath cracks, exposed metal, and loose joints, effectively reducing false alarm and false negative rates and demonstrating stronger robustness under complex background interference. During feature extraction and fusion, designs such as depthwise separable convolution, channel shuffling, and residual connections enhance the perception of fine structures such as slender cable textures and narrow seam morphologies. Simultaneously, it enables the localization of cable hazard targets of different sizes and types, solving the problem of insufficient identification capabilities for small targets and fine-structure hazards in existing technologies.

[0097] Example 3 illustrates a schematic scheme for a cable hazard identification method based on multimodal difference fusion. It should be noted that the technical solution of this cable hazard identification system based on multimodal difference fusion is based on the same concept as the technical solution of the cable hazard identification method based on multimodal difference fusion described above. Details not described in detail in the technical solution of the cable hazard identification system based on multimodal difference fusion in this embodiment can be found in the description of the technical solution of the cable hazard identification method based on multimodal difference fusion described above.

[0098] This embodiment also provides a cable hazard identification system based on multimodal difference fusion, including:

[0099] Multimodal image acquisition module: used to simultaneously acquire the initial infrared and initial visible light images of the cable;

[0100] Image preprocessing module: used to preprocess the initial infrared image and initial visible light image to obtain standard infrared image and standard visible light image;

[0101] Multimodal feature fusion module: It is used to extract modal features from standard infrared images and standard visible light images respectively through an efficient channel attention module to obtain infrared modal features and visible light modal features, and then perform differential operations and reweighted fusion on the infrared modal features and visible light modal features to obtain a fused feature map;

[0102] Target detection module: used to input the fused feature map into the target detection network to obtain cable hazard information;

[0103] The result generation and alarm module is used to generate cable hazard identification results based on cable hazard information, and to display them visually or trigger alarms.

[0104] This embodiment also provides an electronic device applicable to cable hazard identification based on multimodal difference fusion, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the cable hazard identification method based on multimodal difference fusion as proposed in the above embodiment.

[0105] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the cable hazard identification method based on multimodal difference fusion as proposed in the above embodiments.

[0106] The storage medium proposed in this embodiment and the cable hazard identification method based on multimodal difference fusion proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0107] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0108] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for identifying cable hazards based on multimodal difference fusion, characterized in that, include: Acquire initial infrared and initial visible light images of the cable, and preprocess the initial infrared and initial visible light images respectively to obtain standard infrared and standard visible light images; The standard infrared image and the standard visible light image are extracted using an efficient channel attention module to obtain infrared modal features and visible light modal features respectively. The infrared modal features and the visible light modal features are subjected to a difference operation to obtain modal difference features, and the modal difference features are then reweighted and fused to obtain a fused feature map. The feature fusion map is input into the target detection network to obtain cable hazard information, and a cable hazard identification result is generated based on the cable hazard information. The cable hazard identification result is then visualized.

2. The cable hazard identification method based on multimodal difference fusion as described in claim 1, characterized in that, The steps to obtain standard infrared and standard visible light images include: The initial infrared image and the initial visible light image are respectively image registered to obtain a registered infrared image and a registered visible light image; The registered infrared image is enhanced and radiometrically corrected to obtain the standard infrared image; The registered visible light image is enhanced to obtain the standard visible light image.

3. The cable hazard identification method based on multimodal difference fusion as described in claim 2, characterized in that, The steps for obtaining infrared modal characteristics and visible light modal characteristics include: Both the standard infrared image and the standard visible light image are subjected to depth-separable convolution and channel shuffling operations to obtain preliminary infrared features and preliminary visible light features; Channel attention mechanisms are applied to the infrared preliminary features and visible light preliminary features respectively to obtain infrared attention-weighted features and visible light attention-weighted features; The infrared attention-weighted features are residually concatenated with the standard infrared image to obtain the infrared modal features; The visible light attention-weighted features are residually concatenated with the standard visible light image to obtain the visible light modal features.

4. The cable hazard identification method based on multimodal difference fusion as described in claim 3, characterized in that, The steps of performing a difference operation on the infrared modal features and the visible light modal features to obtain modal difference features include: Align the infrared modal features with the visible light modal features in the channel dimension; The modal difference feature is obtained by subtracting the infrared modal feature from the visible light modal feature element by element after aligning the channel dimensions.

5. The cable hazard identification method based on multimodal difference fusion as described in claim 4, characterized in that, The steps of reweighting and fusing the modal difference features to obtain the fused feature map include: The modal difference features are subjected to global average pooling in the horizontal and vertical directions respectively to obtain horizontal global features and vertical global features; The horizontal global features and vertical global features are concatenated, and feature transformation is performed through a shared convolutional layer to obtain differential context features; The difference context features and the modal difference features are multiplied element-wise to obtain weighted difference features. A channel attention mechanism is then applied to the weighted difference features to obtain the fused feature map.

6. The cable hazard identification method based on multimodal difference fusion as described in claim 5, characterized in that, The steps of inputting the feature fusion map into the target detection network to obtain cable hazard information include: The fused feature map is input into the backbone network of the target detection network for multi-scale feature extraction to obtain multi-scale backbone features; The multi-scale backbone features are fused with multi-scale contextual features to obtain enhanced contextual features; The enhanced contextual features are input into the multi-scale detection head of the target detection network for prediction to obtain the corresponding bounding box information of targets at different scales; The bounding box information is decoded to obtain multiple bounding boxes. Each bounding box is parsed as a cable hazard target, and the location coordinates, confidence level, and category of each cable hazard target are output as the cable hazard information.

7. The cable hazard identification method based on multimodal difference fusion as described in claim 6, characterized in that, The steps for generating cable hazard identification results based on the cable hazard information include: The cable hazard targets in the cable hazard information are traversed, and the corresponding alarm threshold is queried in the preset database according to the category of the current cable hazard target; The alarm threshold is compared with the confidence level of the current cable hazard target. When the confidence level is not lower than the alarm threshold, the current cable hazard target is determined as a target to be alarmed. The location coordinates of the target to be alarmed are mapped to the physical space coordinate system to obtain the physical location information; The cable hazard identification result is generated by combining the physical location information with the category of the target to be alarmed.

8. A cable hazard identification system based on multimodal difference fusion, employing the method described in any one of claims 1-7, characterized in that, include: Multimodal image acquisition module: used to simultaneously acquire the initial infrared and initial visible light images of the cable; Image preprocessing module: used to preprocess the initial infrared image and the initial visible light image to obtain a standard infrared image and a standard visible light image; Multimodal feature fusion module: used to extract modal features from the standard infrared image and the standard visible light image respectively through the efficient channel attention module, to obtain infrared modal features and visible light modal features, and to perform differential operation and reweighted fusion of the infrared modal features and visible light modal features to obtain a fused feature map; Target detection module: used to input the fused feature map into the target detection network to obtain cable hazard information; Result generation and alarm module: used to generate cable hazard identification results based on the cable hazard information, and to display them visually or trigger alarms.

9. An electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the cable hazard identification method based on multimodal difference fusion as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the cable hazard identification method based on multimodal difference fusion as described in any one of claims 1 to 7.

Images