An improved yoloV11-based charging pile multi-fault intelligent detection system

Abstract

The application discloses a charging pile multi-fault intelligent detection system based on improved YoloV11, relates to the technical field of intelligent operation and maintenance of charging piles, and collects four-mode original data in an input layer; a preprocessing module processes the four-mode original data respectively to output uniform size feature maps; a backbone network performs feature extraction and enhancement through a light cross-channel convolution unit and a dynamic local feature module, and a RepResNet-18 module is additionally arranged for visible light mode data to perform feature enhancement; a cross-mode dynamic gate is used to enhance a fault correlation area; a neck network realizes cross-mode multi-level fusion and multi-scale hierarchical layering; a detection head multi-scale feature is used for boundary box prediction, fault category prediction and confidence prediction, and finally, a prediction result is output through an output layer. The application realizes breakthroughs in detection integrity, robustness and efficiency, can be directly applied to an edge intelligent monitoring scene of charging piles, and significantly reduces operation and maintenance costs.

Description

Technical Field

[0001] This invention relates to the field of intelligent operation and maintenance technology for charging piles, and in particular to an intelligent multi-fault detection system for charging piles based on an improved YoloV11. Background Technology

[0002] With the widespread adoption of electric vehicles, charging stations, as critical infrastructure, are of paramount importance in terms of operational reliability and safety. Accurate fault detection is crucial for preventing accidents, reducing maintenance costs, and improving user experience. Currently, most mainstream fault detection technologies rely on monitoring electrical parameters with preset thresholds or analyzing images from a single visual sensor. These technologies struggle to adapt to complex deployment environments and sophisticated maintenance requirements, exhibiting three significant drawbacks:

[0003] First, single-modal detection has inherent limitations, with a high rate of missed detection of latent faults. Existing solutions use isolated sensing modes, lacking multi-dimensional cross-validation and complementarity, resulting in insufficient ability to identify complex and early-stage slowly changing faults, posing safety risks.

[0004] Second, the algorithm has weak resistance to environmental interference and poor generalization ability. Existing algorithms rely on limited scene training data, and their performance degrades when faced with dynamic interference from different regions, seasons, and weather conditions, resulting in a persistently high false detection rate.

[0005] Third, redundant architecture leads to high computing power consumption and makes it difficult to adapt to edge deployment. Traditional solutions require the independent deployment of multiple dedicated models, resulting in severe resource consumption. This not only increases additional energy consumption and hardware costs but also hinders the large-scale application of existing charging piles and new low-power charging piles. Summary of the Invention

[0006] To overcome the aforementioned problems in the existing technology, this invention proposes a multi-fault intelligent detection system for charging piles based on an improved YoloV11.

[0007] The technical solution adopted by the present invention to solve its technical problem is: a charging pile multi-fault intelligent detection system based on improved YoloV11, including an input layer, a preprocessing module, a backbone network, a neck network, a detection head, and an output layer. The input layer collects four-modal raw data through an ultraviolet imaging unit, a visible light imaging unit, an infrared thermal imaging unit, and a vibration sensing unit. The preprocessing module processes the four-modal raw data respectively and outputs a uniform size feature map.

[0008] The backbone network obtains the feature distribution of different modes by grouping ultraviolet data, infrared data, and vibration data through lightweight cross-channel convolutional units. It enhances the details and edge features of small targets through dynamic local feature modules, and adds a RepResNet-18 module for feature enhancement for visible light modal data. Through cross-modal dynamic gating, four-modal global weight adaptation, and local spatial attention refinement, it enhances the fault-related region and suppresses irrelevant background.

[0009] The neck network sequentially achieves cross-modal multi-level fusion and multi-scale hierarchical layering through an early fusion module, a mid-term hierarchical feature distillation module, a multi-scale cross-modal attention module, and a cross-modal pyramid attention module, ultimately obtaining multi-scale cross-modal features;

[0010] The detection head performs bounding box prediction, fault category prediction, and confidence prediction based on the multi-scale features output by the neck network, and finally outputs the prediction results through the output layer.

[0011] The aforementioned intelligent multi-fault detection system for charging piles based on the improved YoloV11 includes a vibration sensing unit comprising a vibration sensor mounted on the surface of the charging pile main controller module, a vibration sensor mounted at the charging gun interface, and distributed vibration sensors laid along the cable.

[0012] The above-mentioned intelligent multi-fault detection system for charging piles based on improved YoloV11, the specific processing flow of the preprocessing module for the four-modal raw data is as follows:

[0013] For the data acquired by the ultraviolet imaging unit, the ORB algorithm is used to extract the candidate feature point set of corona discharge, and the feature point set is Gaussian interpolated to output the corona pseudo image.

[0014] For the time-domain vibration signal collected by the vibration sensing unit, environmental noise is eliminated by adaptive bandpass filtering, and short-time Fourier transform is applied to output the vibration time-frequency diagram;

[0015] For the thermal imaging data acquired by the infrared thermal imaging unit, the temperature measurement deviation is eliminated by a dynamic temperature compensation algorithm, and the data is converted into visual features by pseudo-color mapping to output a hot spot feature map.

[0016] For images acquired by the visible light imaging unit, mild Gaussian denoising is performed without altering the original structural features.

[0017] The above-mentioned intelligent detection system for multiple faults in charging piles based on improved YoloV11 includes a lightweight cross-channel convolutional unit design with a dual-branch structure of a main branch and a low-cost branch. The main branch is used for channel dimensionality reduction and feature extraction, while the low-cost branch is used for spatial detail enhancement. The dual-branch features are spliced ​​together according to the channel dimension through a fusion layer.

[0018] The dynamic local feature module includes a dynamic depthwise separable convolution, a lightweight Swin attention module, and a feature fusion module to dynamically adapt to fault morphology and local context modeling. The dynamic depthwise separable convolution dynamically generates convolution kernel parameters based on input features and adaptively adjusts the convolution kernel weights. The lightweight Swin attention reduces the computational complexity of global attention through window partitioning while maintaining long-distance dependency modeling within local regions. The dimension alignment module obtains bi-branch features that are aligned in both channels and space. The feature fusion module integrates the bi-branch features through element-wise multiplication to enhance fault-sensitive regions.

[0019] The aforementioned intelligent multi-fault detection system for charging piles based on the improved YoloV11 employs a dual-mode control mechanism for cross-modal dynamic gating. First, global modal weights are calculated, the global trend of each modality is extracted, and four-modal weight prediction is performed. Second, local spatial attention generation is performed, focusing on the local fault features of each modality to generate a fault area map.

[0020] The aforementioned intelligent multi-fault detection system for charging piles based on improved YoloV11, wherein the RepResNet-18 module adopts a multi-branch parallel approach during the training phase, simultaneously running the dimensionality reduction branch, the main convolution branch, and the identity mapping branch. The results of the three branches are merged by adding them element-wise, and then normalized and activated by ReLU to output the result. During the inference phase, the multi-branch approach during the training phase is merged into a single 3×3 convolution through reparameterization technology, and the features before activation are output.

[0021] The aforementioned intelligent multi-fault detection system for charging piles based on an improved YoloV11, wherein the early fusion module determines the importance of each mode based on global features, dynamically adjusts the importance weights of each mode, generates spatial attention masks within each mode through a spatial attention enhancement unit, and fuses global mode weights, local spatial masks, and the features of each mode's data itself to output the early fusion result. ;

[0022] The intermediate-level characteristic distillation module allows for distillation loss... Learn fine-grained feature information from multimodal intermediate layers, enhance feature representation of small faults, and output distillation-enhanced features. ;

[0023] The multi-scale cross-modal attention module takes the single-scale feature of the mid-term fusion as input, and transforms the mid-term fusion feature into three-scale features of small fault, medium fault and large fault through scale adaptation and downsampling. The four-modal heads in each scale work in parallel, and the attention weights are dynamically biased towards the dominant modality of that scale, outputting three scale feature maps of deep fusion.

[0024] The cross-modal pyramid attention module is based on the YOLOv11 feature pyramid architecture and embeds a cross-modal attention mechanism. It assigns the three scale feature maps of deep fusion to the corresponding pyramid levels and achieves feature fusion between different levels through upsampling / downsampling. Each level selectively fuses the advantageous features of each modality according to the fault scale characteristics.

[0025] The aforementioned intelligent multi-fault detection system for charging piles based on the improved YoloV11 has a pyramid hierarchy design in its cross-modal pyramid attention module as follows: layer P3 is for fine-grained detection of small faults, layer P4 is for balanced detection of medium faults, layer P5 is for global detection of large faults; layer P6 is a fault association layer that integrates layers P5 and P4 to output fault association features; and layer P7 is a global context layer that integrates layers P6 and P5 to output global context.

[0026] The aforementioned intelligent multi-fault detection system for charging piles based on an improved YoloV11 specifically includes bounding box prediction as follows:

[0027] Bounding box parameters: ;

[0028] in, The coordinates of the bounding box center; Define the width and height of the bounding box; This represents the bounding box rotation angle;

[0029] The loss function is ;

[0030] in, To predict the bounding box; The true bounding box; For distance intersection union ratio, , For intersection, union, and comparison, Let Euclidean distance be the square of the center of the two frames. The diagonal of the smallest bounding rectangle of the two frames is squared. For prediction / actual perspective.

[0031] The beneficial effects of this invention are that, through the innovation of "four-modal collaborative sensing + spatiotemporal synchronization", it achieves the following effects:

[0032] 1. Significantly improve the ability to detect latent faults: By accurately capturing early corona signals through ultraviolet mode and efficiently identifying potential mechanical loosening hazards through vibration mode, combined with the ability of visible light mode to identify physical damage such as cracks and the monitoring advantages of infrared mode for hot spot anomalies, it comprehensively covers typical fault types of charging piles and completely changes the status quo of traditional single-mode detection being insufficient for the identification of latent faults.

[0033] 2. Effectively avoid missing sudden faults: The dynamic sampling triggering mechanism can quickly respond to sudden fault scenarios such as arc discharge and abnormal insertion / removal noises, ensuring complete recording of data throughout the fault process. This solves the problem of existing technologies not being able to capture transient faults in a timely manner, and provides complete data support for fault tracing.

[0034] This invention achieves the following effects through the innovation of "differentiated preprocessing + multi-level cross-modal fusion":

[0035] 1. Significantly reduced false detection rate in complex scenarios:

[0036] The ultraviolet channel enhances the corona signal identification through ORB feature point extraction and Gaussian interpolation technology, effectively resisting the interference of adverse weather conditions such as rain on the detection results; the vibration channel converts the one-dimensional signal into a two-dimensional time-frequency diagram through STFT time-frequency conversion, which greatly reduces the impact of power frequency noise on mechanical fault identification.

[0037] The cross-modal fusion module significantly improves detection accuracy in extreme scenarios such as strong light and complex backgrounds by verifying complementary information from different modalities, thus solving the problem that single modalities are easily affected by environmental interference.

[0038] 2. Improved robustness of small fault detection: The mid-term fusion module is specifically designed to enhance the feature representation of small-sized, low-contrast faults such as pinholes and micro-corona discharges, improving the insufficient sensitivity of traditional detection technologies for small fault identification and enhancing the system's ability to capture early-stage weak faults.

[0039] This invention achieves the following effects through the innovation of "lightweight feature extraction + heavy parameterization design":

[0040] 1. Reduced computational power consumption: By using RepGhost lightweight convolution and RepResNet-18 reparameterization design, the computational complexity of the model is significantly simplified and the system hardware deployment cost is reduced while ensuring the accuracy of feature extraction.

[0041] 2. Enhanced real-time detection capabilities: The lightweight architecture combined with INT8 quantization optimization technology shortens the processing time of single-frame data, ensuring that the system has efficient inference capabilities, meeting the needs of online real-time monitoring of charging piles, and enabling rapid response and early warning of faults.

[0042] This invention achieves the following effects through the innovation of "layered detection + dedicated loss optimization":

[0043] 1. Improved positioning accuracy: The CMPA hierarchical detection mechanism adapts to fault features across all scales. By combining DloU loss and angle loss functions, it effectively improves the positioning effect of irregular faults, enhances the fitting accuracy of fault bounding boxes and the accuracy of direction prediction, and provides precise location guidance for maintenance.

[0044] 2. Improved classification accuracy: The FocalLoss loss function is used to specifically address the problem of fault class imbalance, thereby improving the identification accuracy of a minority of fault classes such as early cracks and minor loosening.

[0045] 3. Enhanced reliability of results: Cross-modal consistency constraints ensure the consistency of multimodal data in identifying the same fault. Combined with the NMS confidence filtering mechanism, reliable detection results are screened. The final output fault information has the characteristics of accurate type, clear location, and clear direction, providing a highly reliable basis for operation and maintenance decisions.

[0046] In summary, this invention addresses the three core problems of existing technologies—high false negative rate, poor scenario adaptability, and high computing power consumption—through multimodal perception, fusion, and detection innovation across the entire chain. It achieves breakthroughs in detection integrity, robustness, and efficiency, and can be directly applied to intelligent monitoring scenarios at the edge of charging piles, significantly reducing operation and maintenance costs. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the system of the present invention;

[0048] Figure 2 This is a structural diagram of the lightweight cross-channel convolutional unit module of the present invention;

[0049] Figure 3 This is a flowchart of the dynamic local feature module network of the present invention;

[0050] Figure 4 This is a network structure diagram of the RepResNet-18 module of this invention;

[0051] Figure 5 This is a diagram of the cross-modal dynamic gating (CM-DGF, pre-fusion) network structure of the present invention;

[0052] Figure 6 This is a network structure diagram of the early fusion module (CM-DGF2) of this invention;

[0053] Figure 7 This is a network structure diagram of the hierarchical feature distillation (HFD) module of the present invention;

[0054] Figure 8 This is a network structure diagram of the multi-scale cross-modal attention (MS-CMA) module of the present invention;

[0055] Figure 9 This is a network structure diagram of the Cross-Modal Pyramid Attention (CMPA) module of this invention;

[0056] Figure 10 This is a network structure diagram of the detection head module of the present invention. Detailed Implementation

[0057] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] This embodiment discloses a multi-fault intelligent detection system for charging piles based on an improved YoloV11, such as... Figure 1 As shown, it includes an input layer, a preprocessing module, a backbone network, a neck network, a detection head, and an output layer. The input layer collects four-modal raw data through an ultraviolet imaging unit, a visible light imaging unit, an infrared thermal imaging unit, and a vibration sensing unit.

[0059] Ultraviolet imaging unit: Employs a solar-blind ultraviolet camera (wavelength 240-280nm) to capture early corona discharge signals through adaptive gain control.

[0060] Visible light imaging unit: Equipped with a 12-megapixel global shutter camera, it uses dual filter switching and polarization filtering technology to avoid overexposure in strong light and interference from specular reflection.

[0061] Infrared thermal imaging unit: adopts a 640×512 resolution uncooled focal plane array (supports detection from -40℃ to 120℃), and has a built-in dynamic temperature compensation algorithm to correct temperature measurement deviation.

[0062] Vibration sensing unit: A combination of a triaxial MEMS accelerometer and a piezoelectric sensor is used to detect low-frequency mechanical loosening and high-frequency discharge vibration respectively; the sensors are deployed on the surface of the main controller, at the charging gun interface, and along the cable.

[0063] Three vibration sensors are installed in a triangular arrangement on the surface of the charging pile main controller module to detect vibration of the cabinet structure; a miniature vibration sensor is embedded at the charging gun interface to monitor abnormal vibration during insertion and removal; and distributed vibration sensing optical fibers are laid along the cable to realize full-length cable vibration monitoring.

[0064] Signal processing flow: First, environmental noise is eliminated by using an adaptive bandpass filter (10-2000Hz). Then, wavelet transform is applied to extract the characteristic parameters of the vibration signal. Finally, the time-varying characteristics of the vibration signal are analyzed by short-time Fourier transform (STFT).

[0065] Hardware-level synchronization: The clocks of the ultraviolet camera, visible light camera, infrared camera, and vibration sensor are synchronized using the PTPv2 (IEEE1588-2008) precision time protocol to ensure that the sampling start time error of the four-modal sensors is <10ns. The sampling frequency of the vibration sensor is set to an integer multiple of the camera frame rate (2000Hz corresponds to 25fps) to facilitate subsequent data alignment.

[0066] Software compensation: Based on an event-triggered timestamp interpolation algorithm, the transmission delay of data from each sensor is dynamically compensated, achieving a time alignment accuracy of ±0.5ms. A sliding window mechanism (window size 50ms) is used to achieve precise alignment between vibration signals and image frames.

[0067] Anomaly detection trigger: When the vibration sensor detects abnormal vibration exceeding the threshold (0.5gRMS), it automatically triggers the sampling frequency to be increased to 10kHz and simultaneously increases the image acquisition frame rate to 50fps, realizing high-density data acquisition for sudden faults.

[0068] The preprocessing module processes the four modal raw data and outputs a uniform-size feature map.

[0069] Based on the raw data (ultraviolet imaging, visible light imaging, infrared thermal imaging, and vibration signals) collected by the multimodal sensing subsystem described above, and considering the modal specificity of typical charging pile faults (corona discharge depends on ultraviolet light, hot spot temperature rise depends on infrared light, loosening and abnormal noise depends on vibration time-frequency characteristics, and physical damage depends on visible light texture), a four-channel parallel preprocessing architecture is designed to output a uniform-size feature map to adapt to the subsequent Backbone feature extraction network, as detailed below:

[0070] 1. Ultraviolet Channel (Corona Detection Adaptation): Input a 224×224 single-channel ultraviolet image, extract feature points using the ORB algorithm, and then generate a 224×224×3 corona pseudo image through Gaussian interpolation.

[0071] 2. Vibration Channel (Loosening Detection Adaptor): The input time-domain vibration signal is denoised by adaptive bandpass filtering and then converted into a 224×224×3 vibration time-frequency diagram by STFT.

[0072] 3. Infrared Channel (Hot Spot Detection Adaptor): Input 224×224 single-channel thermal imaging data, and generate a 224×224×3 hot spot feature map through dynamic temperature compensation and pseudo-color mapping.

[0073] 4. Visible light channel (adapted for physical damage detection): Input a 224×224×3 RGB image, perform mild Gaussian denoising to preserve structural damage features, and output an image of the same size.

[0074] The feature maps output from the four preprocessing channels (all 224×224×3) will be used as inputs to the subsequent Backbone cross-modal feature extraction module: the ultraviolet, vibration, and infrared channel features will enter the general branch of "RepGhost lightweight convolution → RepSwin-Tiny dynamic enhancement", and the visible light channel features will enter the dedicated branch of "RepResNet-18". Through unified size and modality-specific enhancement, a high-quality feature foundation will be provided for cross-modal fusion and fault detection.

[0075] The backbone network obtains the feature distribution of different modes by grouping ultraviolet data, infrared data, and vibration data through lightweight cross-channel convolutional units. It enhances the details and edge features of small targets through dynamic local feature modules, and adds a RepResNet-18 module for feature enhancement for visible light modal data. Through cross-modal dynamic gating, four-modal global weight adaptation, and local spatial attention refinement, it enhances the fault-related region and suppresses irrelevant background.

[0076] Lightweight cross-channel convolutional unit (RepGhost): Designed with a "dual-branch collaboration + feature reuse" architecture to address the sparsity and heterogeneity of multimodal inputs (UV pseudomaps, vibration time-frequency maps, infrared thermal imaging), this unit reduces computational load while preserving key information such as corona points and hotspot contours, thus meeting the real-time requirements of multi-fault detection in charging piles. The specific module structure is as follows: Figure 2 As shown.

[0077] Let the input multimodal features be (I: Input multimodal feature map, B: Batch) Input channel (Space dimensions).

[0078] (1) Primary Branch: Responsible for channel dimensionality reduction and basic feature extraction. It uses 1×1 convolution to compress the number of channels and reduce the amount of subsequent computation.

[0079] 1×1 convolution dimensionality reduction: ( This is a 1×1 convolution operation. The number of intermediate channels is determined by the present invention. Half of the output channel, This is the feature map after dimensionality reduction, with dimension . From the input channel Compress to (Preserving global feature trends) Compressing the channel redundancy of input features and preserving semantic information shared by multiple modalities.

[0080] BN+ReLU activation: BN stands for BatchNormalization. Activation function , The activated feature map has the same dimensions. Batch Normalization (BN) accelerates convergence and ReLU introduces nonlinearity. BN accelerates network convergence and alleviates the distribution differences of inputs from different modes; ReLU introduces nonlinearity and enhances the network's ability to express fault features.

[0081] (2) Cheap Branch: Responsible for spatial detail enhancement, using 3×3 depthwise separable convolution to capture local features such as corona point distribution and hot spot edges with low computational cost;

[0082] 3×3 depthwise separable convolution: ,( It is a 3×3 depthwise separable convolution (each channel is convolved independently). The enhanced feature map has a dimension of . Only for The spatial dimension calculation, with a parameter count that is [amount] of traditional convolution. In the low-channel dimension The details of the corona points and the texture of the hot spots are enhanced by local spatial convolution.

[0083] BN+ReLU activation: BN+ReLU is used in the same main branch to maintain computational consistency. This further enhances the nonlinear representation of features while maintaining the distribution alignment between different branches.

[0084] (3) Feature fusion

[0085] Concat layer: Concatenates the features of the two branches along the channel dimension to restore the feature expressive power.

[0086] Channel concatenation (Concat): (Concat is a channel-based concatenation, merging bi-branch features along the channel dimension, restoring the output channel count to its original value.) It integrates the global semantic information of the main branch with the local detailed features of the inexpensive branch to restore the complete feature representation capability with low computational cost.

[0087] Output features are ( To output the feature map, Output channel, this invention takes , (Number of intermediate channels).

[0088] Dual-branch collaborative mechanism: By using "dimensionality reduction main branch + lightweight spatial branch", the computational overhead of channel and spatial features is decoupled. Multimodal feature low-rank decomposition reduces the amount of computation while retaining weak correlation in channel grouping, thus avoiding information loss.

[0089] Feature reuse strategy: Utilize the Concat operation to reuse dual-branch features, preserving multimodal details with low computational cost;

[0090] Multimodal adaptation: To address the heterogeneity of corona (sparse points), hot spots (regional features), and vibration time-frequency (texture), grouped convolution naturally adapts to the feature distribution of different modes.

[0091] This module serves as the "first stop" for cross-modal feature extraction, providing lightweight and information-rich basic features for subsequent dynamic attention and hierarchical distillation. It is the core support for this invention to break through the limitations of traditional single-modal detection.

[0092] The RepSwin-Tiny Dynamic Local Feature Module addresses faults sensitive to local features such as UV corona spots and infrared hotspots. It employs a "dynamic convolution + window attention" architecture, enhancing the details and edge features of small targets through adaptive parameter adjustment and local context modeling, while maintaining real-time inference efficiency. The specific structure is as follows: Figure 3 As shown.

[0093] (1) Dynamic Depthwise Convolution:

[0094] Let the input multimodal features be Based on the input features, the convolution kernel parameters are dynamically generated to adaptively capture the irregular distribution of corona points and the variable contours of hot spots.

[0095] Global feature extraction: ( This is the result of global pooling, and its dimension is... (Compressing spatial dimensions while retaining global channel statistics) Extracts channel-level distribution trends of corona points and channel-level energy distribution of hot spots.

[0096] Convolution parameter prediction: ,in , It is a single fully connected layer (connecting to subsequent encoding), with an output dimension of C / 4=16 (dimensionality reduction to compress global features and avoid overfitting).

[0097] Global feature encoding:

[0098] ;

[0099] in, Multilayer perceptron (2 fully connected layers, hidden layer) Dimension 16, Output Layer Dimension ,correspond (convolution kernel parameters); : Dynamic convolution kernel parameters, dimension is The dynamic convolution kernel parameters are predicted by a multilayer perceptron (K is the kernel size, and this invention uses K=3). Based on the global distribution of corona points and the shape of hot spots, the kernel weights are adaptively adjusted to make the convolution more closely match the local morphology of the fault.

[0100] Kernel generation: Reshape the parameters into a 3×3 dynamic convolution kernel (independent for each channel). ,in ( ), It is a dedicated dynamic convolution kernel for the i-th channel, and its weights are adaptively adjusted according to the corona distribution (sparse → dense) and the hot spot shape (circular → irregular).

[0101] Dynamic depthwise convolution calculation: ( Channel-by-channel application Dynamic kernel; Output feature map, dimension same as X. ) Use adaptive convolution kernels to enhance the edge details of corona points and the contour boundaries of hot spots.

[0102] (2) Lightweight Swing Attention:

[0103] Window partitioning reduces the computational complexity of global attention while preserving long-distance dependency modeling within local regions. Window partitioning: (M: Window size (M=7 in this invention), adapted to the local area of ​​the charging pile equipment, such as the size of the charging gun ≈ 7×7 pixels, limiting the attention range to reduce background interference.) Windowing features, with dimensions of The feature is divided into local windows, and attention is calculated only within the window to reduce complexity, while focusing on the local clustering of corona points and the regional correlation of hot spots.

[0104] Self-attention within a window: , dimension , Multi-head self-attention (number of attention heads) (Balanced calculation and expression) Attention-enhanced window features, same dimensions (Calculate self-attention within each window to reduce global computational complexity) and model the neighborhood dependence of corona points and the spatial diffusion trend of hot spots within local windows.

[0105] Displacement window self-attention parameters: ,in, same dimension , Attention calculation for "right-shifting + down-shifting" windows to make adjacent windows overlap. Enhanced features for cross-window association

[0106] Lightweight optimization parameters: ,in same dimension , The attention weight matrix is ​​shared between adjacent windows. This is a lightweight, optimized attention feature.

[0107] Submodule: 1×1 convolution → bilinear interpolation

[0108] ① 1×1 ​​convolution (channel aligned)

[0109] , ;

[0110] For a 1×1 convolution kernel, the number of channels in both branches is unified to... This ensures dimensional matching. , It is a bi-branch feature with a uniform number of channels.

[0111] ② Bilinear interpolation (spatial alignment)

[0112] , ;

[0113] BilinearInterp uses a weighted, smooth scaling method based on neighboring pixels to preserve spatial details. The target space size is consistent with the input X, ensuring that the dimensions of the bi-branch feature space are perfectly aligned. , It is a bi-branch feature that is aligned in both the channel and space.

[0114] (3) Feature fusion: The dual-path features are integrated by element-wise multiplication (Hadamard product) to enhance the fault-sensitive area.

[0115] Element-by-element multiplication: (This represents the Hadamard product, where Y is the output feature map. It integrates the local details of dynamic convolution with the contextual dependence of attention to enhance the feature representation of fault regions.)

[0116] Through the above parameter design, the RepSwin-Tiny module achieves a three-in-one approach of "dynamically adapting to fault modes + local context modeling + efficient calculation", providing core support for the accurate detection of minor faults (corona, hot spots) in charging piles.

[0117] RepResNet-18 module: A structural enhancement core for the visible light branch; the module structure is as follows... Figure 4As shown, for the structural features of the visible light mode (cracks, deformation, component positions, etc. in charging piles), the following approach is used: "reparameterized convolution + residual connection".

[0118] (1) Training phase: multi-branch parallel (precision priority)

[0119] Input processing: Let the input features be defined. , Batch Normalization (BN) accelerates convergence and adapts to fluctuations in visible light illumination; it also affects the input features. First, perform BN normalization (to alleviate distribution shift and adapt to changes in visible light illumination).

[0120] Multi-branch computation: 1×1 dimensionality reduction branch: number of compressed channels ( → This reduces the computational cost of 3×3 convolutions; ,in For 1×1 convolution, the number of kernels is... (Pick (Balanced calculation and expression)

[0121] 3×3 main convolutional branch: extracts spatial features such as damaged edges and textures (receptive field covers 3×3 pixels, adapting to crack details); ,in For 3×3 convolution, stride 1, padding 1 (keeping the spatial dimensions unchanged);

[0122] Identity mapping branch: directly passes the input (preserves the original information when the spatial size remains unchanged); When space size When the input features remain unchanged: directly pass the input features (preserve the original information); when the spatial size changes: adjust the number of channels and size through 1×1 convolution (such as when downsampling the feature map).

[0123] Gradient short-circuiting: Residual connections mitigate gradient vanishing in deep networks, ensuring that subtle fault features can be passed to subsequent layers; Information preservation: Prevents over-abstraction of complex convolutional operations, preserving original structural information.

[0124] Feature fusion: The results of the three branches are merged by adding them element by element, and then normalized by BN and activated by ReLU to output the final result. :

[0125] ;

[0126] ;

[0127] In this process, elements are added one by one (the number of channels must be the same). To normalize again and stabilize training; Activation function Nonlinearity is introduced.

[0128] Multi-scale fusion: : Global features at the channel dimension; Local features of spatial dimensions; Original structural features; Fault feature enhancement: ReLU activation suppresses negative features and amplifies fault-related features.

[0129] (2) Inference stage: Single convolution fusion (efficiency first)

[0130] Reparameterization merging: Merging 1×1 convolutional kernels, 3×3 convolutional kernels, and identity mappings from the training phase into a single 3×3 convolutional kernel (mathematically equivalent, with the same number of parameters but a drastically reduced computational cost); Through reparameterization techniques, merging multiple branches from the training phase into a single 3×3 convolution (denoted as RepConv): Inference computation: The input is first Batch Normalization (BN), and then passed through a fused 3×3 convolution to directly output the features before activation (BN is folded into the convolution parameters during inference, which speeds up the computation).

[0131] Reparameterization logic: Embed a 1×1 convolutional kernel at the center of a 3×3 convolutional kernel (1×1 is equivalent to the 3×3 convolutional kernel having a non-zero center element and the rest being 0). The identity mapping is equivalent to the 3×3 convolutional kernel having all elements as 0 (except for the center element which is 1). Finally, these are merged into a single 3×3 convolutional kernel. The number of parameters remains the same, but the computational cost is reduced. Down to .

[0132] Integration with charging pile fault detection: Crack detection: 3×3 convolution captures high-contrast pixels at the crack edge, identity mapping preserves the overall direction of the crack, and after fusion, ReLU activation enhances crack features and suppresses background texture.

[0133] Rust detection: 1×1 convolution integrates the "red channel anomaly" and "texture roughness" feature channels, and 3×3 convolution refines the edges of the rusted area, finally outputting the complete outline of the rusted area.

[0134] Component missing detection: Identity mapping preserves the "position and outline of normal components". When a component is missing, 3×3 convolution detects abnormal edges. The difference between the two is amplified by ReLU to form a clear fault signal.

[0135] Cross-modal dynamic gating (CM-DGF, pre-fusion), module structure as follows Figure 5As shown, to address the heterogeneity of the four modes of ultraviolet (corona), vibration (loosening), infrared (hot spot), and visible light (structural / damage), an architecture of "four-mode global weight adaptation + local spatial attention refinement" is designed. Before fusion, the fault-related regions (such as corona points at the edge of insulators and hot spots at joints) are enhanced, and irrelevant backgrounds (such as the sky and non-equipment areas) are suppressed, thereby improving the efficiency of cross-modal information fusion.

[0136] Global modal weight calculation (dynamically assigning importance): Let the input four modal features be... All dimensions Global feature aggregation (capturing modal-level statistical information):

[0137] ,in GlobalAvgPool averages the spatial dimensions H×W of the feature map, compressing spatial information while preserving channel-level global trends. Output : Global statistical characteristics of the m-th mode.

[0138] Extract the "global trend" for each modality: UV ( ): The overall intensity of corona discharge; visible light ( ): Global distribution of equipment structure; infrared ( ): Global energy of the hot spot; vibration ( ): The global percentage of frequencies that are abnormal.

[0139] Four-modal weighted prediction (learning inter-modal dependencies)

[0140] ,in , The four-modal global features are concatenated along the channel dimension, with the dimension being... (Integrating global information from four modalities). MLP is a 2-layer fully connected network that learns the relationships between modalities. Softmax is used for normalization to ensure that the sum of the weights of the four modalities is 1.

[0141] Local spatial attention generation (focusing on fault-related regions):

[0142] (1) Spatial feature extraction (preserving local details within the channel)

[0143] ,in , The output is a 3×3 depthwise separable convolution (each channel is independently convolved without cross-channel interference), extracting local spatial features within the modality. Preserve the local features of the spatial structure.

[0144] Focusing on the "local fault characteristics" of each mode: ultraviolet ( ): The edge and distribution pattern of the corona points; visible light ( ): The edge of the crack, the boundary of the paint peeling, the location of the joint; infrared ( ): The outline of the hot spot and the changes in the temperature gradient; vibration ( ): Spatial clustering regions of abnormal frequencies in a time-frequency graph.

[0145] (2) Spatial mask generation (marking fault-sensitive areas)

[0146] ,in , The number of channels is compressed to 1 by 1×1 convolution, and local features within the channels are fused. This is the activation function, which outputs weights ranging from 0 to 1.

[0147] Function: Generates a "fault area map": visible light mask ( ): Highlighting critical structural areas such as cracks, damage, and joints; Ultraviolet mask ( ): Highlight the area where corona points are concentrated; infrared mask ( ): Highlights the outline boundary of the hot spot; vibration mask ( ): Spatial clusters of prominent frequency anomalies.

[0148] 4. Dual-mode regulation (global + local collaborative enhancement)

[0149] .

[0150] Element-wise multiplication (multiplying the three tensors point by point according to their spatial location and channel). (Global weight) is extended through a broadcast mechanism to (The weight of each sample applies to all channels and spatial locations); (Local mask) is expanded via a broadcast mechanism to (The mask for each spatial location applies to all channels). Output Modal characteristics after modulation.

[0151] The neck network sequentially achieves cross-modal multi-level fusion and multi-scale hierarchical layering through an early fusion module, a mid-level hierarchical feature distillation module, a multi-scale cross-modal attention module, and a cross-modal pyramid attention module, ultimately obtaining multi-scale cross-modal features.

[0152] Early fusion module four-modal dynamic gating fusion, module network structure as follows Figure 6 As shown, the specific core units and parameter analysis are as follows:

[0153] (1) Channel Unified Unit

[0154] Input: Four-modal pre-fusion features The number of channels may vary for each mode (UV 64, vibration 32, infrared 64, visible 128); Formula: ,in , 1×1 pointwise convolution (only performs channel transformation, no spatial operations); Parameter: convolution kernel The number of channels in each modality is unified to C; the difference in the number of channels between modalities is eliminated to ensure the fairness of subsequent fusion; and the information between channels is integrated through linear transformation.

[0155] (2) Modal weight prediction unit

[0156] ,

[0157] in , For global average pooling (to Compress to ); : Multilayer perceptron (the structure is usually C→C / 4→4); : Normalization function to ensure that the sum of the four modal weights is 1.

[0158] The importance of each mode is determined based on global features (e.g., when detecting corona, the weight of ultraviolet light is ≈0.6; when detecting cracks, the weight of visible light is ≈0.7).

[0159] (3) Spatial attention enhancement unit

[0160] ;

[0161] in, It is a 3×3 depth-separable convolution (each channel is convolved individually, and the number of parameters is only 1 / C of that of a standard convolution); This is the activation function, and its output value range is [0,1].

[0162] Generate a spatial attention mask within the modality (the closer the value is to 1, the more likely the location is to be a fault region).

[0163] Example masking effects: Ultraviolet: Focuses corona points (mask value ≈ 0.9), suppresses background (mask value ≈ 0.1); Visible light: Enhances crack edges (mask value ≈ 0.8), suppresses surface texture of equipment (mask value ≈ 0.3).

[0164] 3. Fusion Formula and Multimodal Collaboration

[0165] Final fusion formula: ;

[0166] Global Modal Weights ( ): Determines which modality is more important overall; Local space mask ( ): Determines which regions within a mode are more important; the feature itself ( ): Carries the original information of the modality.

[0167] Mid-term fusion: Hierarchical Feature Distillation (HFD) module, the specific module structure is as follows Figure 7 As shown, for early fusion features ( To address the issue of insufficient representation of minor faults (pinholes, micro-loosening, early corona discharge), a "teacher-student" distillation mechanism is designed: Teacher characteristics: fine-grained features of the multimodal intermediate layer (such as details of corona discharge points in the UV C3 layer and hot spot edges in the infrared C4 layer); Student characteristics: global features of early fusion (…). Core objective: To enable students to "learn" fine-grained information about teachers' characteristics through distillation loss, thereby strengthening the characteristic expression of minor faults (such as the edge of a pinhole or abnormal vibration frequency of a slight loosening).

[0168] Input 1: Student characteristics Its dimensions are Early fusion of global features (which already includes basic fault information for the four modes).

[0169] Input 2: Teacher characteristics Its dimensions are the intermediate layer features of each modality and the fine-grained features of the multimodality (the original details of small faults).

[0170] (1) Feature alignment: Resolves "scale / channel mismatch"

[0171] The scale (H×W) and number of channels (C) of teacher features (multimodal intermediate layer) and student features (early fusion) may be different (e.g., teacher features 224×224, student features 112×112), and need to be aligned first.

[0172] , ;

[0173] in, Alignment operations (combined 1×1 convolution + up / downsampling): Channel alignment: 1×1 convolution to unify the number of channels (e.g., teacher features 32→64, student features 64→64); Scale alignment: upsampling (small→large, e.g., 112×112→224×224) or downsampling (large→small, e.g., 224×224→112×112), using bilinear interpolation to ensure smoothness; For aligned student features (dimensions: ), The aligned m-th modal teacher feature (dimension and completely consistent).

[0174] (2) L2 distillation loss: conveying fine-grained information

[0175] By minimizing the mean squared error (MSE) between student and teacher characteristics, students can "imitate" the fine-grained details of the teacher.

[0176] ;

[0177] Where N is the total number of feature pixels ( ); The i-th pixel (scalar) of the aligned student features; The i-th pixel (scalar) of the teacher feature of the m-th modality after alignment. Teacher modal weights ( (Dynamic allocation) Mean squared error (a measure of characteristic differences).

[0178] If the i-th pixel of a teacher feature (such as UV C3) corresponds to a "micro-corona point" (with a high value), the student feature, through loss optimization, will "imitate" this high value, reinforcing the representation of the corona point; teacher weights Ensure that "details of relevant modalities are delivered first".

[0179] (3) Output characteristics: Distillation enhancement

[0180] After optimizing student characteristics using distillation loss, and fusing global information with fine-grained teacher details, the output is: in, For learning rate, The gradient of the loss with respect to student features guides feature optimization.

[0181] Output: Distillation Enhancement Features Its dimensions It integrates global information with fine-grained details to obtain cross-modal features (small faults are more prominent).

[0182] Deep Fusion: Multi-Scale Cross-Modal Attention (MS-CMA) module, module structure as follows Figure 8 As shown, using single-scale features (dimensions) from intermediate fusion, such as (H'=224, W'=224), as input, a mechanism of "multi-scale feature evolution + cross-modal dynamic attention" is constructed:

[0183] Multi-scale generation: Through scale adaptation (160×160) + downsampling (80×80, 40×40), the mid-term fusion features are transformed into 160×160 (small fault), 80×80 (medium fault), and 40×40 (large fault) feature maps (multi-scale evolution of the same feature, rather than independent branches);

[0184] Cross-modal dynamic focusing: Four modal heads work in parallel within each scale (UV, vibration, infrared, and visible light), and the attention weight is dynamically biased towards the "dominant mode" of that scale (e.g., 160×160 focuses on UV / vibration, 40×40 focuses on infrared / visible light, but other modal heads are not deleted, and weak contributions are retained to achieve multimodal complementarity).

[0185] Core objective: To enhance the complementary representation of multimodal faults within the same scale and output multi-scale cross-modal enhancement features (for the Neck layer to achieve "cross-scale fault association").

[0186] Input: Intermediate fusion features Dimensions: Cross-modal features that integrate global information with fine-grained details.

[0187] (1) Multi-scale downsampling unit (multi-scale generation layer)

[0188] 1. Scale adaptation (160×160): The mid-scale fusion is achieved through 3×3 convolution (stride=1, padding=32). (e.g., 224×224) adapted to 160×160 (preserving fine-grained details);

[0189] 2. Downsampling (80×80): Perform a 3×3 convolution (stride=2, padding=1) on the 160×160 features to generate 80×80 features (balancing details and context).

[0190] 3. Downsampling (40×40): Perform a 3×3 convolution (stride=2, padding=1) on the 80×80 features to generate 40×40 features (emphasizing global structure).

[0191] ;

[0192] The stride of the convolution is 2, which halves the output resolution (160×160 → 80×80). Padding=1 fills the feature map edges with one layer of zeros to maintain the integrity of edge information before and after convolution (avoiding the truncation of hotspot edges). (The number of channels C remains constant, only the spatial size changes.)

[0193] 160×160 scale: Preserves fine-grained details, suitable for detecting small faults (such as pinholes, micro-corona discharges); 80×80 scale: Balances details and context, suitable for detecting medium-scale faults (such as localized hot spots, small cracks); 40×40 scale: Emphasizes the overall structure, suitable for detecting large faults (such as overall equipment deformation, severe damage).

[0194] (2) Multi-head cross-modal attention unit

[0195] Each attention head focuses on a fault-specific region of a single modality, enabling cross-modal information exchange.

[0196] ;

[0197] in, For the multimodal fusion feature at the k-th scale (e.g., 160×160); This represents the m-th modal feature at the k-th scale (e.g., the 160×160 feature of ultraviolet and infrared). It is a multi-head attention mechanism, containing M heads (usually M=4, corresponding to four modes).

[0198] Attention calculation details (taking the m-th head as an example):

[0199] ;

[0200] in, For query matrix (from) (projected onto the subspace of mode m). For key-value matrix (from (i.e., single-modal features); d is the dimension of the key vector (d=C / M).

[0201] Four modal heads (UV, vibration, infrared, and visible light) are forcibly retained at each scale, but attention weights are dynamically allocated (based on fault type, rather than pruning modes):

[0202] 160×160 scale: Ultraviolet (corona) and vibration (microloosening) have relatively higher attention weights (e.g., a total of 60%), while infrared / visible light have relatively lower weights (20% each, retaining weak contributions).

[0203] 40×40 scale: Infrared (global hotspot) and visible light (structural deformation) have relatively higher attention weights (e.g., a total of 60%), while ultraviolet / vibrational light have relatively lower weights (20% each, retaining weak contributions).

[0204] 80×80 scale: four-modal weight balance (25% each).

[0205] Ultraviolet head: Focus on corona points (guide attention through ultraviolet single-mode features); Infrared head: Focus on hot spot contours (guide attention through infrared single-mode features); Vibration head: Focus on areas of mechanical loosening (guide attention through vibration single-mode features); Visible light head: Focus on cracks and deformations (guide attention through visible light single-mode features).

[0206] (3) Cross-head fusion unit

[0207] After concatenating the outputs of multi-head attention, the information is integrated through a 3×3 depthwise convolution, reducing the number of parameters.

[0208] ;

[0209] in, To splice M headers into the output (e.g., when M=4, the number of channels changes from C to C×M=256); It uses a 3×3 depthwise separable convolution (with only 1 / M of the parameters of a standard convolution, restoring the number of channels to C). It integrates the output of multi-head attention (such as different fault features like corona points, hot spots, and cracks); depthwise convolution reduces the number of parameters while preserving the spatial correlation of features in each channel.

[0210] Output: Furthermore, the feature maps have different dimensions and scales. This enhances the cross-modal representation of fault features at different scales.

[0211] Neck: Cross-Modal Pyramid Attention (CMPA) module, module structure as follows Figure 9 As shown, based on the YOLOv11 feature pyramid (P3~P7) architecture, a cross-modal attention mechanism is embedded to achieve hierarchical detection of "fine-grained focusing on small faults + global correlation of large faults": multi-scale fusion: the three scale features of deep fusion ( ) Assigned to the corresponding pyramid level (P3~P5); Cross-scale interaction: Feature fusion between different levels is achieved through upsampling / downsampling (such as P5→P6, P4→P6), strengthening the hierarchical correlation of faults; Modal adaptation: Each level selectively fuses the advantageous features of specific modes according to the fault scale characteristics (such as P3 focusing on fine-grained features of ultraviolet / vibration).

[0212] P3, Scale: 160×160, Input Features: Core detection targets: small faults (pinholes, micro-corona discharges); Modal focus: ultraviolet / vibration (fine-grained features).

[0213] P4, Scale: 80×80, Input Features: Core detection target: medium-sized faults (local hot spots, small cracks); modal focus: multimodal balanced fusion.

[0214] P5, Scale: 40×40, Input Features: Core detection targets: major faults (overall deformation, damage); modal focus: visible light / infrared (global features);

[0215] P6, Scale: 20×20, Input Features: Fusion of P5 and P4, Core Detection Target: Fault Type Association (e.g., Corona + Hot Spot), Modal Emphasis: Multi-scale Feature Integration;

[0216] P7, Scale: 10×10, Input Features: Fusion of P6 and P5, Core Detection Target: Global Fault Context, Modal Emphasis: High-Level Semantic Features.

[0217] (1) P3~P5 layers: modal selective fusion

[0218] Based on the fault detection objectives at each level, the advantageous features of the corresponding modes are selectively fused.

[0219] P3 layer (160×160): ;

[0220] Input: 160×160 features from deep fusion + 160×160 features from UV / vibrational single modes; cross-modal attention: focusing on details of UV corona points and micro-loosening frequency features of vibration.

[0221] P4 layer (80×80): ;

[0222] Input: 80×80 features from deep fusion + 80×80 features from four modes; Balanced fusion: Simultaneously focusing on hot spot contours, crack boundaries, corona distribution, etc.

[0223] P5 layer (40×40): ;

[0224] Input: 40×40 features from deep fusion + 40×40 features from infrared / visible single-mode; Global features: overall distribution of focused hot spots, structural deformation of the equipment.

[0225] (2) P6~P7 layers: cross-scale fusion

[0226] The interaction of features at different levels is achieved through upsampling (↑) and downsampling (↓).

[0227] P6 layer (20×20): ;

[0228] enter: Upsample the features of layer P5 (40×40→20×20) while preserving global semantics; Downsample the features of layer P4 (80×80→20×20) to preserve mesoscale details;

[0229] P7 layer (10×10): ;

[0230] enter: Upsample the features of layer P6 (20×20→10×10); Downsampling of features for P5 layer (40×40→10×10);

[0231] (3) Cross-modal attention mechanism

[0232] Used at each level Specific calculations:

[0233] ;

[0234] Where Q represents the query feature. Key-value features For modal dynamic weights, .

[0235] The detection head performs bounding box prediction, fault category prediction, and confidence prediction based on the multi-scale features output by the neck network, and finally outputs the prediction results through the output layer.

[0236] Based on the multi-scale features (P3~P7) output by the cross-modal pyramid attention (CMPA), and inheriting the efficient architecture of the YOLOV11 detection head, it performs three core tasks in parallel: bounding box prediction: accurately locates the fault location (such as the center of the corona point, hot spot area) and direction (such as the direction of the crack); fault category prediction: identifies the fault type (such as corona, hot spot, loosening, crack); confidence prediction: judges the reliability of the prediction results (filters out false detections, such as low-confidence results that misclassify the background as a fault).

[0237] By adapting to cross-modal fusion features (integrating the advantages of ultraviolet, vibration, infrared, and visible light), we can achieve "high-precision positioning + high-accuracy classification" for charging pile fault detection.

[0238] 1) Bounding box prediction: DloULoss + angle loss

[0239] The system is optimized for the location (coordinates) and direction (angle) of the fault, especially to adapt to the irregular shape of the charging pile fault (such as the elliptical boundary of the hot spot and the linear direction of the crack).

[0240] Prediction target: bounding box parameters ;

[0241] in, The coordinates of the bounding box center; Define the width and height of the bounding box; Rotate the bounding box by an angle (to accommodate long, narrow faults, such as cracks).

[0242] The loss function is ;

[0243] in, To predict the bounding box (model output) ); The actual bounding box (manually labeled fault area); The Distance-Intersection-Union Ratio (IoU) is calculated as follows: , Intersection over Union (IOU) (a basic positioning metric that measures the degree of overlap). The square of the Euclidean distance between the centers of the two frames (measures the center deviation); The diagonal square of the smallest bounding rectangle of the two frames (normalized distance); For predicted / true angle (range) (e.g., the angle between the crack direction and the horizontal direction); coefficient "0.5": the weight of balancing position loss and angle loss.

[0244] DloULoss optimizes both "overlap" and "center distance" simultaneously, resulting in faster convergence than traditional IoULoss (15% reduction in positioning error); angle loss is designed for elongated faults (such as cracks and linear hot spots), ensuring accurate direction prediction (20% reduction in angle error).

[0245] (2) Fault category prediction: FocalLoss resolves category imbalance

[0246] In charging pile fault detection, there is a class imbalance (e.g., normal samples account for 90%, fault samples account for only 10%, and the number of different fault types varies greatly). FocalLoss improves the classification accuracy of minority classes (such as early cracks) by "focusing on difficult samples".

[0247] Prediction target: Fault category probability distribution (5 classes in total, sum to 1); Loss function: ;

[0248] in, The true class probability predicted by the model (e.g., if the true value is "corona") is taken as... ); Class balance coefficient (giving higher weight to minority classes, such as crack samples) normal sample ); Focusing coefficient for difficult samples (reducing the weight of easily classified samples, such as...) Easy samples, weights ; Hard samples, weights ).

[0249] (3) Confidence prediction: filtering false positives

[0250] Output the probability that "the predicted box contains a fault" ( Reliable results were screened using nonmaximum suppression (NMS).

[0251] Prediction logic: Confidence level ,in, The probability that "a target exists";

[0252] Predicted boxes with low confidence (e.g., c<0.5) are directly filtered out (e.g., misclassifying device texture as cracks); predicted boxes with high confidence (e.g., c>0.7) are retained, improving detection robustness.

[0253] 3. Cross-modal feature adaptation mechanism

[0254] The detection head adapts to the preceding cross-modal fusion features in the following ways:

[0255] Multi-scale feature input: P3 (160×160) inputs small fault detection branches (such as pinholes), P5 (40×40) inputs large fault branches (such as deformations) to ensure feature matching of faults at different scales;

[0256] Modal feature weighting: Before the feature input detection head, cross-modal features are dynamically weighted by 1×1 convolution (e.g., in corona detection, the weight of ultraviolet features is increased by 30%).

[0257] Loss function adaptation: For "multi-source noise" (such as infrared temperature noise) across modal features, add "modal consistency constraint" in DloULoss (if both ultraviolet and infrared detect faults, the bounding box deviation should be smaller).

[0258] The model training optimization process in this embodiment is as follows:

[0259] (1) Dataset Construction

[0260] Sample size: 8000+ four-modal samples (including 2000 corona discharge cases, 1800 hot spot cases, 1500 loosening cases, 1200 crack cases, and 1500 normal cases), covering sunny days, rainy days, strong light and other scenarios;

[0261] Labeling specifications: Manually label the fault bounding boxes (including rotation angle), categories, and confidence levels, and divide the training / validation / test sets in an 8:1:1 ratio.

[0262] (2) Training strategy

[0263] Phased training: ① Basic training (100 epochs): Joint training of four modal features

[0264] The comprehensive loss function is Optimizer AdamW (learning rate 0.01);

[0265] ② Fine-tuning (50 epochs): Increase the weights of the P3 layer for minor faults (pinholes, micro-corona discharges), and decrease the learning rate to 0.0001 to enhance fine-grained feature representation.

[0266] Regularization: Add L2 regularization (λ=0.001) and Dropout (probability 0.2) to suppress overfitting.

[0267] (3) Deployment optimization

[0268] Lightweighting: Model pruning (removing 30% of redundant convolutional kernels) + INT8 quantization, reducing the number of parameters by 40%;

[0269] Edge Adaptation: Deployed on the local edge chip of the charging pile, with power consumption <10W, meeting the requirements for real-time detection.

[0270] The above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art can make various modifications or equivalent substitutions to the present invention within its scope and spirit, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of the present invention.

Claims

1. A charging pile multi-fault intelligent detection system based on improved YoloV11, comprising an input layer, a preprocessing module, a backbone network, a neck network, a detection head, and an output layer, characterized in that, The input layer acquires four modal raw data through an ultraviolet imaging unit, a visible light imaging unit, an infrared thermal imaging unit, and a vibration sensing unit. The preprocessing module processes the four modal raw data respectively and outputs a uniform size feature map. The backbone network obtains the feature distribution of different modes by grouping ultraviolet data, infrared data, and vibration data through lightweight cross-channel convolutional units. It enhances the details and edge features of small targets through dynamic local feature modules, and adds a RepResNet-18 module for feature enhancement for visible light modal data. Through cross-modal dynamic gating, four-modal global weight adaptation, and local spatial attention refinement, it enhances the fault-related region and suppresses irrelevant background. The neck network sequentially achieves cross-modal multi-level fusion and multi-scale hierarchical layering through an early fusion module, a mid-term hierarchical feature distillation module, a multi-scale cross-modal attention module, and a cross-modal pyramid attention module, ultimately obtaining multi-scale cross-modal features; The detection head performs bounding box prediction, fault category prediction, and confidence prediction based on the multi-scale features output by the neck network, and finally outputs the prediction results through the output layer. The bounding box prediction specifically refers to: Bounding box parameters: in, The coordinates of the bounding box center; Define the width and height of the bounding box; This represents the bounding box rotation angle; The loss function is in, To predict the bounding box; The true bounding box; For distance intersection union ratio, , For intersection, union, and comparison, Let Euclidean distance be the square of the center of the two frames. The diagonal of the smallest bounding rectangle of the two frames is squared. For prediction / actual perspective.

2. The intelligent multi-fault detection system for charging piles based on improved YoloV11 according to claim 1, characterized in that, The vibration sensing unit includes a vibration sensor installed on the surface of the charging pile main controller module, a vibration sensor installed at the charging gun interface, and distributed vibration sensors laid along the cable.

3. The intelligent multi-fault detection system for charging piles based on the improved YoloV11 as described in claim 1, characterized in that, The specific processing flow of the preprocessing module for the four-modal raw data is as follows: For the data acquired by the ultraviolet imaging unit, the ORB algorithm is used to extract the candidate feature point set of corona discharge, and the feature point set is Gaussian interpolated to output the corona pseudo image. For the time-domain vibration signal collected by the vibration sensing unit, environmental noise is eliminated by adaptive bandpass filtering, and short-time Fourier transform is applied to output the vibration time-frequency diagram; For the thermal imaging data acquired by the infrared thermal imaging unit, the temperature measurement deviation is eliminated by a dynamic temperature compensation algorithm, and the data is converted into visual features by pseudo-color mapping to output a hot spot feature map. For images acquired by the visible light imaging unit, mild Gaussian denoising is performed without altering the original structural features.

4. The intelligent multi-fault detection system for charging piles based on the improved YoloV11 as described in claim 1, characterized in that, The lightweight cross-channel convolutional unit design includes a dual-branch structure with a main branch and a low-cost branch. The main branch is used for channel dimensionality reduction and feature extraction, while the low-cost branch is used for spatial detail enhancement. The dual-branch features are spliced ​​together according to the channel dimension through a fusion layer. The dynamic local feature module includes a dynamic depthwise separable convolution, a lightweight Swing attention module, and a feature fusion module to achieve dynamic adaptation to fault morphology and local context modeling. The dynamic depthwise separable convolution dynamically generates convolution kernel parameters based on input features and adaptively adjusts the convolution kernel weights. The lightweight Swing attention reduces the computational complexity of global attention through window partitioning while maintaining long-distance dependency modeling within local regions. The dimension alignment module obtains bi-branch features that are aligned in both channels and space. The feature fusion module integrates the dual-branch features through element-wise multiplication, thereby enhancing the fault-sensitive area.

5. The intelligent multi-fault detection system for charging piles based on the improved YoloV11 as described in claim 1, characterized in that, The cross-modal dynamic gating adopts a dual-mode control mechanism. First, global modal weights are calculated to extract the global trend of each modality and perform four-modal weight prediction. Second, local spatial attention is generated to focus on the local fault features of each modality and generate a fault area map.

6. The intelligent multi-fault detection system for charging piles based on improved YoloV11 according to claim 1, characterized in that, During the training phase, the RepResNet-18 module employs a multi-branch parallel approach, simultaneously running the dimensionality reduction branch, the main convolution branch, and the identity mapping branch. The results of the three branches are merged by element-wise addition, and then normalized and activated by ReLU before outputting the final result. During the inference phase, the multi-branch approach from the training phase is used to merge the multiple branches into a single 3×3 convolution, outputting the features before activation.

7. The intelligent multi-fault detection system for charging piles based on the improved YoloV11 according to claim 1, characterized in that, The early fusion module determines the importance of each modality based on global features, dynamically adjusts the importance weights of each modality, generates spatial attention masks within each modality through a spatial attention enhancement unit, and fuses global modality weights, local spatial masks, and the features of each modality's data itself to output the early fusion result. ; The intermediate-level characteristic distillation module allows for distillation loss... Learn fine-grained feature information from multimodal intermediate layers, enhance feature representation of small faults, and output distillation-enhanced features. ; The multi-scale cross-modal attention module takes the single-scale feature of the mid-term fusion as input, and transforms the mid-term fusion feature into three-scale features of small fault, medium fault and large fault through scale adaptation and downsampling. The four-modal heads in each scale work in parallel, and the attention weights are dynamically biased towards the dominant modality of that scale, outputting three scale feature maps of deep fusion. The cross-modal pyramid attention module is based on the YOLOv11 feature pyramid architecture and embeds a cross-modal attention mechanism. It assigns the three scale feature maps of deep fusion to the corresponding pyramid levels and achieves feature fusion between different levels through upsampling / downsampling. Each level selectively fuses the advantageous features of each modality according to the fault scale characteristics.

8. The intelligent multi-fault detection system for charging piles based on the improved YoloV11 according to claim 7, characterized in that, The pyramid hierarchy design in the cross-modal pyramid attention module is as follows: P3 layer is for fine-grained detection of small faults, P4 layer is for balanced detection of medium faults, P5 layer is for global detection of large faults; P6 layer is the fault association layer, which integrates P5 layer and P4 layer to output fault association features. Layer P7 is the global context layer, which merges layers P6 and P5 to output the global context.

Images