Segmentation-detection cascade photovoltaic hot spot detection method and system

Abstract

The invention provides a segmentation-detection cascade photovoltaic hot spot detection method and system, and particularly relates to the technical field of photovoltaic equipment fault detection in the field of computer vision, and the method comprises the steps: segmenting an input infrared image through a photovoltaic panel region segmentation network, generating a photovoltaic panel region mask image, and multiplying the photovoltaic panel region mask image with an original image pixel level, thereby obtaining a detection result; and obtaining a target area image after background suppression. Thirdly, extracting a multi-scale feature map of the target area by utilizing a detection network comprising a deformable convolution module, and then performing weight distribution and fusion on the multi-scale feature map by adopting a self-adaptive spatial feature fusion strategy to generate an optimized feature map; and finally, outputting a hot spot position and category detection result based on the optimized feature map. According to the method, complex background interference is effectively inhibited, the multi-scale hot spot detection precision is improved, the false detection rate is reduced, and the real-time inspection requirement of the photovoltaic power station is met.

Description

Technical Field

[0001] The present invention relates to the technical field of photovoltaic equipment fault detection in the field of computer vision, and in particular to a segmentation-detection cascade photovoltaic hot spot detection method and system. Background Art

[0002] With the large-scale deployment of photovoltaic power plants, hot spot fault detection has become a critical step in ensuring the safe operation of photovoltaic modules. While detection technology based on thermal infrared imaging can effectively identify abnormal temperature rises, it faces complex environmental interference in practical applications. Strong light can cause false highlights in the background area of infrared images, which are highly similar to the characteristics of real hot spots. Hot spot targets on the surfaces of densely packed photovoltaic panels exhibit multi-scale and irregular geometric shapes. Traditional detection networks struggle to balance robust feature representation with computational efficiency, resulting in high false detection and missed detection rates.

[0003] Existing technologies mainly use a single detection network (such as the YOLO series) or a segmentation-detection parallel architecture to identify hot spots, but they have significant limitations in complex backgrounds: 1) When a single detection network directly processes the full-frame infrared image, the background pseudo-highlight area is easily misidentified as a hot spot, and the fixed-shape convolution kernel cannot adapt to the irregular edge features of the hot spots; 2) Although the parallel architecture outputs segmentation and detection results synchronously, it does not establish a cascaded background suppression mechanism. The shallow texture information of multi-scale hot spots is lost during the convolution downsampling process, resulting in insufficient accuracy in small target detection.

[0004] In summary, how to solve the technical problems of false detection and missed detection caused by pseudo-highlight interference, variable target morphology and inconsistent scale in photovoltaic hot spot detection under complex background is an issue that needs to be solved urgently. Summary of the Invention

[0005] The main purpose of the present invention is to provide a segmentation-detection cascade photovoltaic hot spot detection method and system, so as to at least solve the technical problems of false detection and missed detection caused by pseudo-highlight interference, variable target morphology and inconsistent scale in photovoltaic hot spot detection under complex background. Through the segmentation-detection cascade architecture, the complex background interference is effectively suppressed and the multi-scale hot spot detection accuracy is improved, the false detection rate is significantly reduced, and efficient detection is achieved while meeting the real-time inspection needs of photovoltaic power stations.

[0006] In order to achieve the above objectives, the present invention provides a segmentation-detection cascade photovoltaic hot spot detection method and system.

[0007] In a first aspect, the present invention provides a segmentation-detection cascade photovoltaic hot spot detection method, comprising:

[0008] The input infrared image is segmented through the photovoltaic panel area segmentation network to generate a photovoltaic panel area mask image;

[0009] Multiplying the photovoltaic panel area mask image by the input infrared image at a pixel level to obtain a target area image after background suppression;

[0010] Extracting features from the target area image using a detection network including a deformable convolution module to obtain a multi-scale feature map;

[0011] Adopting an adaptive spatial feature fusion strategy to weight and fuse the multi-scale feature maps to generate an optimized feature map;

[0012] The hot spot position and category detection results are output based on the optimized feature map.

[0013] Specifically, the photovoltaic panel region segmentation network is used to segment the input infrared image to generate a photovoltaic panel region mask image, including:

[0014] Performing feature grouping extraction on the input infrared image through a grouped convolutional layer;

[0015] Perform channel shuffling operation on the features extracted after grouping to generate a recombined feature map;

[0016] The reorganized feature map is weighted and adjusted through a channel attention mechanism to generate a mask image of the photovoltaic panel area.

[0017] Specifically, the detection network including the deformable convolution module is used to extract features from the target area image to obtain a multi-scale feature map, including:

[0018] Generating a convolution kernel offset matrix based on the target area image through a deformable convolution module;

[0019] Dynamically sampling the feature map according to the offset matrix;

[0020] The sampling features are multi-level fused through a cascaded Bottleneck structure to output the multi-scale feature map.

[0021] Specifically, the adaptive spatial feature fusion strategy is used to perform weight distribution and fusion on the multi-scale feature map to generate an optimized feature map, including:

[0022] Unifying the feature maps of different levels in the multi-scale feature map to a first resolution by interpolation upsampling and strided convolution downsampling to generate a standardized feature map group;

[0023] Performing a 1×1 convolution process on each feature map in the standardized feature map group to generate a corresponding spatial weight map group;

[0024] Performing element-by-element multiplication and summation operations on the standardized feature map group according to the spatial weight map group to generate the optimized feature map.

[0025] Specifically, performing a channel shuffling operation on the grouped extracted features to generate a recombined feature map includes:

[0026] Divide the grouped extracted features into G channel groups, where each channel group contains C / G channels, where C is the total number of channels;

[0027] Randomize the order of channels within each channel group;

[0028] The randomly arranged channel groups are reconnected along the channel dimension to generate the reorganized feature map.

[0029] Specifically, generating a convolution kernel offset matrix based on the target area image through a deformable convolution module includes:

[0030] Processing the target area image through a 3×3 convolution kernel to generate an initial offset field;

[0031] Performing two 1×1 convolutions on the initial offset field to generate a two-dimensional offset matrix;

[0032] The two-dimensional offset matrix is normalized and corrected to obtain a convolution kernel offset matrix for dynamic sampling.

[0033] Specifically, performing element-by-element multiplication and summation operations on the standardized feature map group according to the spatial weight map group to generate the optimized feature map includes:

[0034] Multiplying the feature maps X1', X2', X3', and X4' in the standardized feature map group by the weight maps α, β, γ, and δ in the spatial weight map group element by element to obtain weighted feature maps;

[0035] The sum operation is performed on all weighted feature maps to generate the optimized feature map, and its calculation expression is:

[0036] Optimized feature map = α⊙X1'+β⊙X2'+γ⊙X3'+δ⊙X4'

[0037] Where ⊙ represents element-wise multiplication.

[0038] In a second aspect, the present invention provides a segmentation-detection cascade photovoltaic hot spot detection system, wherein the detection system applies the detection method described in the first aspect, and the detection system comprises:

[0039] The segmentation network module is used to segment the input infrared image through the photovoltaic panel area segmentation network to generate a photovoltaic panel area mask image;

[0040] a background suppression module connected to the segmentation network module, configured to perform pixel-level multiplication of the photovoltaic panel area mask image and the input infrared image to obtain a target area image after background suppression;

[0041] A detection network module connected to the background suppression module, the detection network module including a deformable convolution module, the deformable convolution module being used to perform multi-scale feature extraction on the target area image and output a multi-scale feature map;

[0042] A feature fusion module is connected to the detection network module, and is used to use an adaptive spatial feature fusion strategy to perform weight distribution and fusion on the multi-scale feature map to generate an optimized feature map;

[0043] A detection result generation module is connected to the feature fusion module, and the detection result generation module is used to output the hot spot position and category detection results based on the optimized feature map.

[0044] Specifically, the segmentation network module includes:

[0045] A grouped convolution submodule, configured to perform feature grouping extraction on the input infrared image through a grouped convolution layer;

[0046] A channel shuffling submodule is connected to the group convolution submodule, and is used to perform a channel shuffling operation on the features extracted after grouping to generate a recombined feature map;

[0047] A channel attention submodule is connected to the channel shuffling submodule, and the channel attention submodule is used to adjust the weight of the recombined feature map through the channel attention mechanism to generate the photovoltaic panel area mask image.

[0048] Specifically, the deformable convolution module includes:

[0049] An offset generation submodule, configured to generate a convolution kernel offset matrix based on the target area image;

[0050] A dynamic sampling submodule, connected to the offset generation submodule, and configured to dynamically sample the feature map according to the convolution kernel offset matrix;

[0051] The feature fusion submodule is connected to the dynamic sampling submodule, and is used to perform multi-level fusion of sampling features through a cascaded Bottleneck structure to output the multi-scale feature map.

[0052] The present application provides a segmentation-detection cascade photovoltaic hot spot detection method and system, which utilizes a photovoltaic panel area segmentation network to perform fine segmentation on the input infrared image, generate a photovoltaic panel area mask image, and then multiply it with the original infrared image at the pixel level to effectively suppress background interference and highlight the target area. Subsequently, through the detection network integrating the deformable convolution module, deep feature extraction is performed on the target area image to form a multi-scale feature map. On this basis, an adaptive spatial feature fusion strategy is adopted to perform intelligent weight allocation and fusion on the multi-scale feature map to generate an optimized feature map. Finally, based on the optimized feature map, the hot spot position and category detection results are accurately output. This method effectively solves the problems of false detection and missed detection of photovoltaic hot spot detection under complex backgrounds, significantly improves the detection accuracy and efficiency, and meets the urgent needs of real-time inspection of photovoltaic power stations. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0054] Figure 1 A schematic diagram of the process of the segmentation-detection cascade photovoltaic hot spot detection method provided in this application;

[0055] Figure 2 Schematic diagram of the segmentation-detection cascade photovoltaic hot spot detection system provided in this application.

[0056] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0057] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0058] The terms "first," "second," "third," "fourth," and so forth (if any) in the description and claims of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequential sequence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments of the present invention described herein can be practiced in sequences other than those illustrated or described herein.

[0059] In the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0060] The segmentation-detection cascade photovoltaic hot spot detection method and system provided in this application accurately segments infrared images through a photovoltaic panel area segmentation network, generates a photovoltaic panel area mask image, and multiplies it with the original image to suppress background interference and highlight the target area. Next, a detection network containing a deformable convolution module is used to extract multi-scale feature maps to capture the diverse forms of hot spots. An adaptive spatial feature fusion strategy is then used to intelligently fuse multi-scale features to generate an optimized feature map. Finally, the hot spot location and category are accurately output based on the optimized feature map, effectively solving the problems of false detection and missed detection in complex backgrounds, improving detection accuracy and efficiency, and meeting the real-time inspection needs of photovoltaic power stations.

[0061] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0062] Figure 1 The flow chart of the segmentation-detection cascade photovoltaic hot spot detection method provided in this application is intended to explain the segmentation-detection cascade photovoltaic hot spot detection method in detail, such as Figure 1 As shown, the segmentation-detection cascade photovoltaic hot spot detection method provided in this embodiment includes:

[0063] S101: Segment the input infrared image through the photovoltaic panel area segmentation network to generate a photovoltaic panel area mask image.

[0064] Specifically, the photovoltaic panel region segmentation network is used to segment the input infrared image to generate a photovoltaic panel region mask image, including:

[0065] Performing feature grouping extraction on the input infrared image through a grouped convolutional layer;

[0066] Performing a channel shuffling operation on the grouped extracted features to generate a recombined feature map. Specifically, performing a channel shuffling operation on the grouped extracted features to generate a recombined feature map includes: dividing the grouped extracted features into G channel groups, wherein each channel group contains C / G channels, where C is the total number of channels; randomly arranging the order of channels within each channel group; and rejoining the randomly arranged channel groups along the channel dimension to generate the recombined feature map.

[0067] The reorganized feature map is weighted and adjusted through a channel attention mechanism to generate a mask image of the photovoltaic panel area.

[0068] The specific implementation process of step S101 "segmenting the input infrared image through the photovoltaic panel region segmentation network to generate a photovoltaic panel region mask image" is as follows:

[0069] 1. Grouped convolution feature extraction

[0070] In the encoder stage of the Lite-UNet segmentation network, the DESR (Depthwise Enhanced Split-Residual) feature extraction module is used to process the input infrared image. The specific operations include:

[0071] The input image is input to the grouped convolution layer, where the group number G is set to 6, each group contains C / 6 channels (C is the total number of channels of the input feature map), the convolution kernel size is 3×3, the stride is 1, and the fill mode is "same".

[0072] The expression of group convolution operation is:

[0073]

[0074] Among them, Y g is the output feature map of the g-th group, is the convolution kernel weight of the cth channel of the gth group, Represents the first Channel, b g is the bias term.

[0075] 2. Channel shuffle operation

[0076] Perform channel shuffling on the feature map output by grouped convolution. The specific steps are:

[0077] The feature map Y (of size C×H×W) output by the grouped convolution is divided into G=6 subgroups along the channel dimension, and each subgroup contains C / 6 channels.

[0078] Perform a random permutation operation on the channels within each subgroup: for the channel index [0, 1, ..., C / 6-1] of the g-th subgroup, generate a random permutation sequence σ_g∈Permutation(C / 6).

[0079] The reorganized feature maps are spliced along the channel dimension to generate the reorganized feature map Y', whose channel order is:

[0080]

[0081] 3. Channel Attention Mechanism

[0082] The Squeeze-and-Excitation (SE) module is used to adjust the channel weights of the reorganized feature map:

[0083] Squeeze stage:

[0084] Perform global average pooling on the reorganized feature map Y' to generate a channel statistics vector Z∈R^C, where the statistics of the c-th channel are calculated as:

[0085]

[0086] Excitation stage: learn the correlation between channels through two fully connected layers and generate a channel weight vector S∈R^C:

[0087] S=σ(W2·δ(W1·Z))

[0088] Where W1∈R C / r×C is a reduced dimension matrix (compression ratio r = 16), W2∈R C×C / r is a dimensional matrix, δ represents the ReLU activation function, and σ represents the Sigmoid function.

[0089] Feature weighting: Multiply the weight vector S by the reorganized feature map Y' channel by channel to output the final feature map:

[0090] 4. Decoder Reconstruction

[0091] The weighted feature map is upsampled through the U-Net decoder structure:

[0092] Transposed convolution is used for 2x upsampling, with a convolution kernel size of 2×2 and a stride of 2. The number of output channels decreases step by step from 256 to 128 to 64.

[0093] After upsampling at each level, a jump connection is made with the feature map of the corresponding level of the encoder, and feature fusion is performed through 3×3 convolution.

[0094] The final output layer uses a 1×1 convolution to compress the number of channels to 1, and generates a photovoltaic panel area mask image through a Sigmoid function.

[0095] This embodiment uses a combination of grouped convolution and channel shuffling in the DESR feature extraction module to achieve effective separation of photovoltaic panel edge features with an average intersection-over-union (MIoU) of 87.46% when the number of parameters is only 14% of that of the traditional U-Net (experimental data: the number of parameters is reduced from 34.5MB to 4.8MB). The introduction of the channel attention mechanism increases the weight of key channels by 3-5 times (experimental data: the S value of important channels is distributed in the range of 0.7-0.9), effectively suppressing the interference of background pseudo-highlight areas. According to actual measurements, the mask image output by this step has a boundary clarity index (Hausdorff distance) that is 62% lower than that of traditional methods in the photovoltaic panel contour segmentation task, providing an accurate region of interest for subsequent hot spot detection.

[0096] S102: Multiply the photovoltaic panel area mask image and the input infrared image at pixel level to obtain a target area image after background suppression.

[0097] The specific implementation process of step S102 "performing pixel-level multiplication of the photovoltaic panel area mask image and the input infrared image to obtain a background-suppressed target area image" is as follows:

[0098] 1. Data format alignment

[0099] The photovoltaic panel area mask image is a single-channel binary image output by step S101, with a size of H×W (H is the image height, W is the image width), and a pixel value of 0 (non-photovoltaic panel area) or 1 (photovoltaic panel area).

[0100] The input infrared image is a single-channel thermal infrared image, the size of which is consistent with the mask image, and the pixel values are normalized to the interval [0, 1].

[0101] 2. Pixel-level multiplication

[0102] (1) Perform element-by-element multiplication on each pixel of the infrared image:

[0103] Define the mask image matrix as M∈R^{H×W} and the infrared image matrix as I∈R^{H×W}.

[0104] The calculation formula of the target area image matrix T is:

[0105]

[0106] Where i and j represent the row index and column index of the pixel respectively.

[0107] (2) When M(i, j) = 1, the original pixel value of the infrared image is retained; when M(i, j) = 0, the corresponding pixel is set to zero.

[0108] 3. Channel Dimension Expansion

[0109] If the subsequent detection network requires a three-channel image as input, perform channel copying on the target area image:

[0110]

[0111] Generate a three-channel target area image T RGB ∈R^{H×W×3}.

[0112] in:

[0113] i∈[0,H-1]: row index of the pixel (the image height is H pixels);

[0114] j∈[0,W-1]: column index of the pixel (the image width is W pixels);

[0115] c: channel index (0 for red channel, 1 for green channel, 2 for blue channel);

[0116] T(i,j): pixel value of the single-channel target area image at position (i,j);

[0117] T RGB (i, j, c): The pixel value of the c-th channel of the three-channel target area image at position (i, j).

[0118] 4. Data normalization

[0119] Perform Z-Score normalization on the target area image:

[0120]

[0121] Among them, μ is the mean value of pixels in the photovoltaic panel area, σ is the standard deviation, and the calculation formula is:

[0122]

[0123] Where N is the total number of pixels in the photovoltaic panel area.

[0124] This embodiment accurately aligns the photovoltaic panel area mask with the original infrared image through element-by-element multiplication to achieve physical isolation of the background area. Experimental data show that in the test set (sample size 742) containing complex backgrounds such as vegetation and buildings, the target area image after background suppression reduces the false positive rate of the detection network from 23.7% to 8.4%. The normalization process converges the variance of the pixel value distribution in the photovoltaic panel area from 0.34±0.12 of the original data to 0.08±0.03, effectively improving the training stability of the detection network. According to actual measurements, the target area image output by this step reduces the forward propagation time by 42% (from 18.3ms to 10.6ms) in the YOLOv8 detection network, because the processing area is reduced to 27%-35% of the original image (specific data: the average proportion of the photovoltaic panel area is 31.2%±4.8%), which significantly improves the detection efficiency.

[0125] S103: Perform feature extraction on the target area image through a detection network including a deformable convolution module to obtain a multi-scale feature map.

[0126] Specifically, the detection network including the deformable convolution module is used to extract features from the target area image to obtain a multi-scale feature map, including:

[0127] Generating a convolution kernel offset matrix based on the target area image through a deformable convolution module. Specifically, generating the convolution kernel offset matrix based on the target area image through the deformable convolution module includes: processing the target area image through a 3×3 convolution kernel to generate an initial offset field; performing two 1×1 convolution processes on the initial offset field to generate a two-dimensional offset matrix; and performing normalization correction on the two-dimensional offset matrix to obtain a convolution kernel offset matrix for dynamic sampling.

[0128] Dynamically sampling the feature map according to the offset matrix;

[0129] The sampling features are multi-level fused through a cascaded Bottleneck structure to output the multi-scale feature map.

[0130] The specific implementation process of step S103 "extracting features from the target area image through a detection network including a deformable convolution module to obtain a multi-scale feature map" is as follows:

[0131] 1. Deformable convolution kernel offset generation

[0132] (1) Initial offset field generation:

[0133] The target area image is input into the deformable convolution module, and a 3×3 convolution kernel (weight initialization method is He normal distribution, filling mode is "same", step size is 1) is used to perform a convolution operation on the target area image to generate an initial offset field matrix Δ∈R^{H×W×18}, where 18 represents the coordinate offset of the 9 sampling points corresponding to the 3×3 convolution kernel for each pixel (each sampling point contains offsets in both x and y directions, for a total of 9×2=18 channels).

[0134] (2) Offset refinement:

[0135] Perform two 1×1 convolution operations on the initial offset field Δ:

[0136] The first 1×1 convolution: the number of input channels is 18→36, the activation function is ReLU, and the output is the intermediate offset field Δ_mid;

[0137] Second 1×1 convolution: the number of input channels is 36→18, the activation function is a linear function, and the output is a two-dimensional offset matrix Δ_raw∈R^{H×W×18}.

[0138] (3) Normalization correction:

[0139] For Δ_raw (also △ raw ) performs a nonlinear transformation using the hyperbolic tangent function (tanh) to constrain the offset to the interval [-1,1]:

[0140]

[0141] 2. Dynamic Feature Sampling

[0142] Define the input feature map as X∈R^{H×W×C}, for each position p0=(i,j) on the feature map X, according to Δ_final (that is, △ final ) is sampled by bilinear interpolation of the offset at the corresponding position:

[0143]

[0144] Where: p n ∈{(-1,-1),(-1,0),...,(1,1)} represents the coordinates of the 9 regular sampling points of the 3×3 convolution kernel; w n is the convolution kernel weight parameter; final (p0,n) represents the offset (x and y directions) of the nth sampling point at position p0.

[0145] 3. Multi-level feature fusion

[0146] (1) A cascaded Bottleneck structure is used to fuse the features after dynamic sampling. Each Bottleneck unit contains the following sequential operations:

[0147] a. Deformable convolution layer: uses a 3×3 deformable convolution kernel (the number of output channels is 1 / 4 of the number of input channels);

[0148] b. Batch normalization: normalize the feature map to zero mean and unit variance;

[0149] c.ReLU activation: introduces nonlinear expression capabilities;

[0150] d.1×1 convolution compression: restore the number of channels to the original dimension.

[0151] (2) Features are gradually fused through three cascaded Bottleneck units to output a multi-scale feature map group {Level1, Level2, Level3, Level4}. The sizes of the feature maps at each level are 1 / 8, 1 / 16, 1 / 32, and 1 / 64 of the input image, respectively.

[0152] This embodiment dynamically adjusts the convolution kernel sampling position through the deformable convolution module, so that the effective receptive field of the 3×3 convolution kernel is expanded by 2.3 times in the elongated hot spot area with an aspect ratio greater than 3:1 (experimental data: standard convolution covers an area of 12×12 pixels, and deformable convolution covers 28×12 pixels). The cascaded Bottleneck structure increases the inter-channel correlation coefficient of the multi-scale feature map from 0.32 to 0.78 (Pearson correlation coefficient), and reduces the false detection rate to 8.4%. In the test set (742 images), the recall rate (Recall) of irregular hot spot detection reached 92.7%, an increase of 14.6 percentage points compared to the traditional YOLOv8 baseline model.

[0153] S104: Adopting an adaptive spatial feature fusion strategy to perform weight assignment and fusion on the multi-scale feature map to generate an optimized feature map.

[0154] Specifically, the adaptive spatial feature fusion strategy is used to perform weight distribution and fusion on the multi-scale feature map to generate an optimized feature map, including:

[0155] Unifying the feature maps of different levels in the multi-scale feature map to a first resolution by interpolation upsampling and strided convolution downsampling to generate a standardized feature map group;

[0156] Performing a 1×1 convolution process on each feature map in the standardized feature map group to generate a corresponding spatial weight map group;

[0157] The standardized feature map group is subjected to element-by-element multiplication and summation operations according to the spatial weight map group to generate the optimized feature map. Specifically, the standardized feature map group is subjected to element-by-element multiplication and summation operations according to the spatial weight map group to generate the optimized feature map, which specifically includes: performing element-by-element multiplication of the feature maps X1', X2', X3', X4' in the standardized feature map group with the weight maps α, β, γ, δ in the spatial weight map group to obtain a weighted feature map; performing a summation operation on all weighted feature maps to generate the optimized feature map, and its calculation expression is: optimized feature map = α⊙X1'+β⊙X2'+γ⊙X3'+δ⊙X4', wherein ⊙ represents element-by-element multiplication.

[0158] The specific implementation process of step S104 "using an adaptive spatial feature fusion strategy to perform weight distribution and fusion on multi-scale feature maps to generate an optimized feature map" is as follows:

[0159] 1. Multi-scale feature map normalization

[0160] (1) Input feature map group: Receive the multi-scale feature map group {Level1, Level2, Level3, Level4} output from step S103, with sizes of 1 / 8, 1 / 16, 1 / 32, and 1 / 64 of the input image respectively.

[0161] (2) Unified resolution:

[0162] Upsampling operation: perform bilinear interpolation upsampling on Level2, Level3, and Level4 feature maps, with magnifications of 2×, 4×, and 8×, respectively, to make their resolutions consistent with those of Level1 feature maps. Figure 1 (i.e. 1 / 8 of the input image);

[0163] Downsampling operation: perform a 3×3 convolution with a step size of 2 on the Level 1 feature map (the convolution kernel weights are initialized using the Xavier normal distribution), and output the downsampled feature map so that its resolution is the same as that of the Level 2 feature map. Figure 1 (i.e. 1 / 16 of the input image);

[0164] Normalized feature map group generation: Get a normalized feature map group with a uniform size of H / 8×W / 8

[0165] {X1',X2',X3',X4'}, where H and W are the height and width of the input image.

[0166] 2. Spatial Weight Map Generation

[0167] Weight parameter extraction: The following operations are performed on each feature map in the normalized feature map group:

[0168] Use a 1×1 convolutional layer (with 1 output channel and Sigmoid activation function) to process the feature map Xk'(k∈{1,2,3,4}) and generate the corresponding spatial weight map {α,β,γ,δ}∈R^{H / 8×W / 8};

[0169] The convolution kernel weights are initialized using a zero-mean Gaussian distribution (standard deviation σ = 0.01).

[0170] 3. Adaptive feature fusion

[0171] Element-wise multiplication: Perform element-wise multiplication on each feature map Xk' in the normalized feature map group and the corresponding spatial weight map:

[0172] W K =Xk′⊙ω K (k∈{1,2,3,4},ω∈{α,β,γ,δ})

[0173] Where ⊙ represents element-by-element multiplication, W K is the weighted feature map.

[0174] Feature map summation: Add the four weighted feature maps to generate the optimized feature map Y:

[0175] Y=W1+W2+W3+W4

[0176] Feature enhancement: Perform a 3×3 depth-wise separable convolution on Y (separating the depth-wise convolution kernel from the point-wise convolution kernel) to further extract spatial features.

[0177] This embodiment dynamically adjusts the contribution of multi-scale features through a spatial weight map, which increases the feature response intensity of small target hot spots (area <32×32 pixels) by 2.8 times (experimental data: the average activation value of the weight map in the small target area is 0.73, and that in the background area is 0.12). In the test set (742 images), the optimized feature map increases the mAP@0.5 of multi-scale detection from 81.3% to 86.5%, of which the small target detection accuracy (AP@0.5) increases from 67.2% to 79.1%. Feature fusion takes only 4.3ms (NVIDIA Jetson Xavier NX platform), which meets the frame rate requirements of real-time inspection of photovoltaic power stations.

[0178] rate requirement (>23FPS).

[0179] S105: Output the hot spot location and category detection results based on the optimized feature map.

[0180] Among them, the step S105 "outputs the hot spot location and category detection results based on the optimized feature map"

[0181] The specific implementation process is as follows:

[0182] 1. Detection head network construction

[0183] (1) Input feature map: Receive the optimized feature map Y∈R^{H / 8×W / 8×C} output in step S104 (H and W are the height and width of the input image, and C=256 is the number of channels).

[0184] (2) Bounding box regression branch:

[0185] A 3×3 convolutional layer (with 4×A output channels, A=3, the number of anchor boxes, and a linear activation function) is used to generate the bounding box offset.

[0186] Output 4 parameters for each anchor box (d x ,d y d w ,d h ) to adjust the center coordinates (x, y) and width and height (w, h) of the preset anchor box:

[0187] x pred =(d x w a )+x a ,y pred =(d y ·h a )+y a

[0188]

[0189] Among them, (x a ,y a ,w a ,h a ) is the preset anchor box parameter, (x pred ,y pred ,w pred ,h pred ) is the prediction box parameter.

[0190] (3) Classification branches:

[0191] A 3×3 convolutional layer (the number of output channels is K×A, K=2 is the number of categories, and the activation function is Softmax) is used to generate the hot spot category probabilities.

[0192] Output the category confidence P∈[0,1] of each anchor box, and the calculation formula is:

[0193]

[0194] Among them, P c is the probability value of the anchor box belonging to category c after Softmax normalization; K is the total number of categories of the classification task; sc Indicates the confidence score that the current anchor box belongs to category c; 0 represents background and 1 represents hot spot; Represents the exponential sum of all category scores.

[0195] 2. Post-processing process

[0196] (1) Confidence filtering: Set the confidence threshold θ = 0.6 to filter out candidate boxes with P ≥ θ

[0197] (2) Non-maximum suppression (NMS):

[0198] a. Arrange candidate boxes in descending order of confidence;

[0199] b. Select the candidate box with the highest confidence and calculate its intersection over union (IoU) with the remaining candidate boxes:

[0200]

[0201] in:

[0202] B pred : The currently selected highest confidence candidate box (i.e., the predicted box retained in the current iteration), whose coordinate format is (x1, y1, x1, y2), indicating the coordinates of the upper left corner and the lower right corner;

[0203] B rest : The remaining candidate boxes (i.e., unprocessed prediction boxes) have the same coordinate format as B pred ;

[0204] Area(B pred ∩B rest ):B pred With B rest The area of the intersection;

[0205] Area(B pred ∪B rest ):B pred With B pred The area of the union of .

[0206] c. Delete candidate boxes with IoU ≥ 0.5;

[0207] d. Repeat steps bc until all candidate boxes are processed.

[0208] (3) Hot spot location mapping:

[0209] The normalized coordinates (x pred ,y pred ,w pred ,h pred ) mapped back to the original image size:

[0210] xorig =x pred 8,y orig =y pred 8

[0211] w orig =w pred 8,h orig =h pred 8

[0212] This example, using a dual-branch architecture of bounding box regression and classification, achieved a mean average precision (mAP) of 86.5% for hotspot detection at 0.5 on a test set of 742 images. Small objects (area < 32×32 pixels) achieved a detection accuracy of 79.1% at 0.5. NMS post-processing reduced the false positive rate from 12.3% to 8.4%, and single-image inference time was kept within 23ms (on the NVIDIA Jetson Xavier NX platform). The RMSE (Real Value Sequence Error) was less than 1.2 pixels, meeting the sub-centimeter positioning accuracy requirements for photovoltaic power plant inspections.

[0213] Figure 2 Schematic diagram of the segmentation-detection cascade photovoltaic hot spot detection system provided in this application, such as Figure 2 As shown in the figure, the segmentation-detection cascade photovoltaic hot spot detection system provided by this embodiment uses Figure 1 The detection method described in the embodiment, the system comprises:

[0214] The segmentation network module is used to segment the input infrared image through the photovoltaic panel area segmentation network to generate a photovoltaic panel area mask image;

[0215] a background suppression module connected to the segmentation network module, configured to perform pixel-level multiplication of the photovoltaic panel area mask image and the input infrared image to obtain a target area image after background suppression;

[0216] A detection network module connected to the background suppression module, the detection network module including a deformable convolution module, the deformable convolution module being used to perform multi-scale feature extraction on the target area image and output a multi-scale feature map;

[0217] A feature fusion module is connected to the detection network module, and is used to use an adaptive spatial feature fusion strategy to perform weight distribution and fusion on the multi-scale feature map to generate an optimized feature map;

[0218] A detection result generation module is connected to the feature fusion module, and the detection result generation module is used to output the hot spot position and category detection results based on the optimized feature map.

[0219] Specifically, the segmentation network module includes:

[0220] A grouped convolution submodule, configured to perform feature grouping extraction on the input infrared image through a grouped convolution layer;

[0221] A channel shuffling submodule is connected to the group convolution submodule, and is used to perform a channel shuffling operation on the features extracted after grouping to generate a recombined feature map;

[0222] A channel attention submodule is connected to the channel shuffling submodule, and the channel attention submodule is used to adjust the weight of the recombined feature map through the channel attention mechanism to generate the photovoltaic panel area mask image.

[0223] Specifically, the deformable convolution module includes:

[0224] An offset generation submodule, configured to generate a convolution kernel offset matrix based on the target area image;

[0225] A dynamic sampling submodule, connected to the offset generation submodule, and configured to dynamically sample the feature map according to the convolution kernel offset matrix;

[0226] The feature fusion submodule is connected to the dynamic sampling submodule, and is used to perform multi-level fusion of sampling features through a cascaded Bottleneck structure to output the multi-scale feature map.

[0227] In specific implementation, the segmentation-detection cascade photovoltaic hot spot detection system includes the following modules and connection relationships:

[0228] 1. Split network module

[0229] Input interface: Receive infrared images (resolution 640×512 pixels, single channel, pixel value range 0-255).

[0230] 1.1 Grouped convolution submodule:

[0231] Structure: Contains 6 groups of convolutional layers, each with C = 64 channels (384 input channels), kernel size 3 × 3, stride 1, and fill mode "same";

[0232] Connection: The output end is connected to the channel shuffling submodule through the data bus;

[0233] Function: Extract the features of the input image by channel grouping, reducing the number of parameters (86% reduction compared to standard convolution).

[0234] 1.2 channel shuffling submodule:

[0235] Specific operation: Divide the 384-channel features output by the grouped convolution submodule into G = 6 groups (64 channels per group), perform random permutation on the channels in each group (the permutation sequence is initialized by a pseudo-random number generator), and splice along the channel dimension to generate a recombined feature map.

[0236] Connection: The input end receives the output of the grouped convolution submodule, and the output end is connected to the channel attention submodule;

[0237] Function: Enhance cross-group feature interaction and improve edge feature expression capabilities (experimental data shows that MIoU increases by 6.3%).

[0238] 1.3 Channel Attention Submodule:

[0239] Structure: Squeeze-and-Excitation (SE) module with compression ratio r=16, including a global average pooling layer and two fully connected layers (dimensionality change: 384→24→384);

[0240] Connection: The input end receives the output of the channel shuffling submodule, and the output end generates a mask image of the photovoltaic panel area (a binary image, 0 is the background, 1 is the photovoltaic panel area);

[0241] Effect: By adjusting the channel weight, the background pseudo-highlight area is suppressed (the area with a weight value < 0.2 is reduced from 32% to 9%).

[0242] 2. Background suppression module

[0243] Input interface: receives the mask image and original infrared image output by the segmentation network module;

[0244] Operation: Perform element-by-element multiplication on the two images (multiply the mask image pixel value 0 / 1 by the infrared image pixel value);

[0245] Connection: The input end is connected to the segmentation network module and the original image input interface through the data bus, and the output end is connected to the detection network module;

[0246] Effect: Physically isolates non-PV panel areas, reduces the target area to 31.2% ± 4.8% of the original image, and reduces the detection network inference time by 42%.

[0247] 3. Detect network module

[0248] 3.1 Deformable Convolution Module:

[0249] 3.1.1 Offset generation submodule:

[0250] Operation: Use a 3×3 convolution kernel (output channel number 18) to generate the initial offset field. After two 1×1 convolutions (channel number 18 → 36 → 18), the offset matrix Δ∈R^{H×W×18} is normalized by the tanh function.

[0251] Connection: The input end receives the target area image output by the background suppression module, and the output end is connected to the dynamic sampling submodule;

[0252] Function: Dynamically adjust the convolution kernel sampling position, increasing the coverage area of slender hot spots with an aspect ratio greater than 3:1 by 2.3 times.

[0253] 3.1.2 Dynamic sampling submodule:

[0254] Operation: According to the offset matrix Δ, bilinear interpolation sampling is performed on the input feature map. The formula is:

[0255]

[0256] Connection: The input end receives the output of the offset generation submodule and the backbone network feature map, and the output end is connected to the feature fusion submodule;

[0257] Function: Improve the characteristic response intensity of irregular hot spots (experimental data: the gradient value in the edge area is increased by 1.8 times).

[0258] 3.1.3 Feature fusion submodule:

[0259] Structure: 3 cascaded Bottleneck units, each unit contains a deformable convolution layer (3×3, 64 output channels), a batch normalization layer, a ReLU activation layer, and a 1×1 compressed convolution layer (256 output channels);

[0260] Connection: The input end receives the output of the dynamic sampling submodule, and the output end generates a multi-scale feature map group {Level1, Level2, Level3, Level4} (resolutions are 80×64, 40×32, 20×16, 10×8 respectively);

[0261] Function: Through multi-level feature fusion, the mAP@0.5 of multi-scale hot spot detection is improved to 86.5%.

[0262] 4. Feature Fusion Module

[0263] (1) Input interface: receives the multi-scale feature map group output by the detection network module;

[0264] (2) Operation:

[0265] Resolution unification: Level 2, Level 3, and Level 4 feature maps are upsampled to 80×64 resolution using 2×, 4×, and 8× bilinear interpolation, respectively, to generate a standardized feature map group {X1', X2', X3', X4'};

[0266] Weight generation: Perform 1×1 convolution (output channel number 1, Sigmoid activation) on each standardized feature map to generate a spatial weight map {α, β, γ, δ};

[0267] Feature fusion: Optimize the feature map according to the formula = α⊙X1'+β⊙X2'+γ⊙X3'+δ⊙X4' for weighted summation;

[0268] (3) Connection: The input end is connected to the detection network module, and the output end is connected to the detection result generation module;

[0269] (4) Function: Adaptively enhance the features of small targets (weighted average 0.73 for areas < 32 × 32 pixels), increasing the AP@0.5 of small target detection to 79.1%.

[0270] 5. Detection result generation module

[0271] (1) Bounding box regression branch:

[0272] Structure: 3×3 convolutional layer (12 output channels, corresponding to 3 anchor boxes × 4 offset parameters), linear activation;

[0273] Function: Predict the location of hot spots with a coordinate mapping error (RMSE) of <1.2 pixels.

[0274] (2) Classification branches:

[0275] Structure: 3×3 convolutional layer (output channels 6, corresponding to 3 anchor boxes × 2 categories), Softmax activation;

[0276] Function: Outputs the confidence level of the hot spot category, reducing the false positive rate to 8.4%.

[0277] (3) Post-processing unit:

[0278] Operations: confidence filtering (threshold θ = 0.6), non-maximum suppression (IoU threshold 0.5), coordinate mapping (×8 scaling to original image size);

[0279] Connection: The input end receives the optimized feature map output by the feature fusion module, and the output end generates the hot spot location and category detection results;

[0280] Purpose: Single image processing takes 23ms (NVIDIA Jetson Xavier NX), meeting real-time inspection frame rates > 23FPS.

[0281] Data flow and physical connection between modules:

[0282] Data bus type: PCIe 3.0×4 interface, transmission bandwidth 3.94GB / s, ensuring unblocked data transmission between modules;

[0283] Hardware carrier: deployed on embedded devices (CPU: ARM Cortex-A72, GPU: NVIDIA 384-core Volta);

[0284] Storage medium: The segmentation network parameters are stored in 4GB LPDDR4 memory, and the detection network parameters are stored in 8GB GDDR6 video memory.

[0285] This system, through a cascaded segmentation-detection architecture, achieved an 8.4% false detection rate and a 6.3% missed detection rate for hot spot detection in field measurements at photovoltaic power plants, a significant improvement over the 23.7% false detection rate of traditional end-to-end detection methods. The deformable convolution module increased the recall rate of irregular hot spot detection by 14.6 percentage points, and adaptive feature fusion achieved a small target detection accuracy of 79.1%, meeting the IEC 62446-3 standard for photovoltaic fault detection.

[0286] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered merely as exemplary, and the true scope and spirit of the present application are indicated by the claims.

[0287] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A segmentation-detection cascade photovoltaic hot spot detection method, characterized in that: include: The input infrared image is segmented through the photovoltaic panel area segmentation network to generate a photovoltaic panel area mask image; Multiplying the photovoltaic panel area mask image by the input infrared image at a pixel level to obtain a target area image after background suppression; Extracting features from the target area image using a detection network including a deformable convolution module to obtain a multi-scale feature map; Adopting an adaptive spatial feature fusion strategy to weight and fuse the multi-scale feature maps to generate an optimized feature map; The hot spot position and category detection results are output based on the optimized feature map.

2. The detection method according to claim 1, wherein The photovoltaic panel region segmentation network is used to segment the input infrared image to generate a photovoltaic panel region mask image, including: Performing feature grouping extraction on the input infrared image through a grouped convolutional layer; Perform channel shuffling operation on the features extracted after grouping to generate a recombined feature map; The reorganized feature map is weighted and adjusted through a channel attention mechanism to generate a mask image of the photovoltaic panel area.

3. The detection method according to claim 1, wherein The method of extracting features from the target area image using a detection network including a deformable convolution module to obtain a multi-scale feature map includes: Generating a convolution kernel offset matrix based on the target area image through a deformable convolution module; Dynamically sampling the feature map according to the offset matrix; The sampling features are multi-level fused through a cascaded Bottleneck structure to output the multi-scale feature map.

4. The detection method according to claim 1, wherein The adaptive spatial feature fusion strategy is used to perform weight distribution and fusion on the multi-scale feature map to generate an optimized feature map, including: Unifying the feature maps of different levels in the multi-scale feature map to a first resolution by interpolation upsampling and strided convolution downsampling to generate a standardized feature map group; Performing a 1×1 convolution process on each feature map in the standardized feature map group to generate a corresponding spatial weight map group; Performing element-by-element multiplication and summation operations on the standardized feature map group according to the spatial weight map group to generate the optimized feature map.

5. The detection method according to claim 2, wherein The performing of a channel shuffling operation on the grouped extracted features to generate a recombined feature map includes: Divide the grouped extracted features into G channel groups, where each channel group contains C / G channels, where C is the total number of channels; Randomize the order of channels within each channel group; The randomly arranged channel groups are reconnected along the channel dimension to generate the reorganized feature map.

6. The detection method according to claim 3, wherein Generating a convolution kernel offset matrix based on the target area image through a deformable convolution module includes: Processing the target area image through a 3×3 convolution kernel to generate an initial offset field; Performing two 1×1 convolutions on the initial offset field to generate a two-dimensional offset matrix; The two-dimensional offset matrix is normalized and corrected to obtain a convolution kernel offset matrix for dynamic sampling.

7. The detection method according to claim 4, wherein The performing element-by-element multiplication and summation operations on the normalized feature map group according to the spatial weight map group to generate the optimized feature map includes: Multiplying the feature maps X1', X2', X3', and X4' in the standardized feature map group by the weight maps α, β, γ, and δ in the spatial weight map group element by element to obtain weighted feature maps; The sum operation is performed on all weighted feature maps to generate the optimized feature map, and its calculation expression is: Optimized feature map = α⊙X1'+β⊙X2'+γ⊙X3'+δ⊙X4' Where ⊙ represents element-wise multiplication.

8. A segmentation-detection cascade photovoltaic hot spot detection system, characterized in that: The detection system applies the detection method according to any one of claims 1 to 7, and the detection system comprises: The segmentation network module is used to segment the input infrared image through the photovoltaic panel area segmentation network to generate a photovoltaic panel area mask image; a background suppression module connected to the segmentation network module, configured to perform pixel-level multiplication of the photovoltaic panel area mask image and the input infrared image to obtain a target area image after background suppression; A detection network module connected to the background suppression module, the detection network module including a deformable convolution module, the deformable convolution module being used to perform multi-scale feature extraction on the target area image and output a multi-scale feature map; A feature fusion module is connected to the detection network module, and is used to use an adaptive spatial feature fusion strategy to perform weight distribution and fusion on the multi-scale feature map to generate an optimized feature map; A detection result generation module is connected to the feature fusion module, and the detection result generation module is used to output the hot spot position and category detection results based on the optimized feature map.

9. The system according to claim 8, wherein The segmentation network module includes: A grouped convolution submodule, configured to perform feature grouping extraction on the input infrared image through a grouped convolution layer; A channel shuffling submodule is connected to the group convolution submodule, and is used to perform a channel shuffling operation on the features extracted after grouping to generate a recombined feature map; A channel attention submodule is connected to the channel shuffling submodule, and the channel attention submodule is used to adjust the weight of the recombined feature map through the channel attention mechanism to generate the photovoltaic panel area mask image.

10. The system according to claim 8, wherein The deformable convolution module includes: An offset generation submodule, configured to generate a convolution kernel offset matrix based on the target area image; A dynamic sampling submodule, connected to the offset generation submodule, and configured to dynamically sample the feature map according to the convolution kernel offset matrix; The feature fusion submodule is connected to the dynamic sampling submodule, and is used to perform multi-level fusion of sampling features through a cascaded Bottleneck structure to output the multi-scale feature map.

Images