Photovoltaic support structure state detection and classification method

Abstract

This invention discloses a method for detecting and classifying the structural state of photovoltaic (PV) support structures. The method includes: using a Swing Transformer-modified Detectron2 instance segmentation model to obtain pixel-level masks and target regions of the PV support; performing mask reconstruction and anomaly scoring using the ViT-AE autoencoder anomaly detection model to obtain an anomaly severity index; introducing an attention gating mechanism into the Swing Transformer classification network and constructing an SPDG constraint loss with anomaly severity as auxiliary supervision to achieve robust classification of the support structure state. This invention solves the problems of false positives and false negatives caused by overlapping and shading of slender components, strong background interference, and diverse anomaly patterns in PV support structures. It can achieve automated, high-precision, and highly generalizable detection and classification of rail-mounted and flat single-axis supports and their columns, and is suitable for unmanned inspection and intelligent operation and maintenance of PV power plants in complex scenarios such as deserts and mountains.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection technology for photovoltaic power plants, specifically to a method for detecting and classifying the structural condition of photovoltaic support structures. Background Technology

[0002] Existing technologies in visual recognition systems generally fall into several categories: extracting contour features and making judgments using traditional image processing methods such as threshold segmentation, edge detection, and morphology. However, in the context of photovoltaic (PV) support structures, which typically include columns, torsion beams, and purlins, the structures are complex and diverse, often densely arranged in multiple rows. These rows easily overlap and occlude in the image, and in desert environments, strong background interference such as vegetation and cables further exacerbates the problem, leading to unstable contour features and frequent false positives and false negatives. Therefore, relying solely on traditional methods like threshold segmentation, edge detection, and morphology is insufficient for accurately identifying individual support targets, making it difficult to determine the integrity of the support structure. In contrast, compared to common convolutional backbones, Transformers are better at fusing global contextual information, improving instance segmentation of slender components even under overlapping occlusion and background interference. Therefore, the Detectron2 detection model, based on the SwinTransformer backbone, possesses stronger feature representation capabilities, achieving instance-level segmentation and outputting the component contours even under complex background and target overlap conditions. To further improve the status determination of the support structure, a classification branch was introduced based on the instance segmentation results, and a binary classification model based on Swin Transformer was used to determine whether the support structure is complete. However, photovoltaic support structures have many types of components and complex assembly relationships, and incomplete forms are diverse. Simply using conventional binary classification or fixed multi-class classification models is difficult to cover all abnormal patterns and is prone to misjudgment and missed judgment.

[0003] To address this, this invention provides a high-precision, interference-resistant, and highly generalizable method for detecting and classifying the structural state of photovoltaic support structures. In the classification stage, this method employs a discriminant network with a Swing Transformer as its backbone and introduces a gating mechanism in the attention calculation to suppress desert background interference and highlight the structural discrimination features of key components. Simultaneously, a ViT-based autoencoder anomaly detection model is introduced as an auxiliary teacher signal, and combined with SPDG constraint loss, weighted constraints are applied to the classification output during the training stage, thereby improving the robustness and generalization ability for recognizing diverse incomplete states. Summary of the Invention

[0004] The purpose of this invention is to overcome the problems existing in the prior art and provide a method for detecting and classifying the structural status of photovoltaic support structures, so as to automatically identify the integrity / incompleteness of guide rail type and flat single-axis type photovoltaic support structures and their columns.

[0005] To achieve the above-mentioned technical objectives and effects, the present invention is implemented through the following technical solution: A method for detecting and classifying the structural condition of photovoltaic support structures, the method comprising the following steps: Step S1: Detect the support frame and column based on the instance segmentation model Detectron2 with Transformer as the backbone network, obtain the target segmentation mask, and provide accurate target region information for subsequent structural state discrimination; Step S2: For the obtained target segmentation mask, use the ViT-based anomaly detection model to evaluate the degree of anomaly of the target and output the anomaly score as an auxiliary discrimination basis in the structural state training stage; Step S3: Using the obtained anomaly assessment information as auxiliary supervision, a classification network based on Swin Transformer is used, and SPDG constraint loss is introduced to jointly constrain the classification training, thereby realizing the classification of the photovoltaic support structure state.

[0006] Furthermore, in step S1, the photovoltaic bracket detection based on the instance segmentation model Detectron2 includes the following sub-steps: Step S101: The robot is positioned at the center of the photovoltaic power station row, and the RGB camera is installed in the middle of the robot to acquire a real-time one-frame image; Step S102: The feature extraction backbone of the instance segmentation model Detectron2 is replaced with the Transformer architecture to detect photovoltaic support targets. Instance segmentation inference is performed on an acquired frame image to obtain instance results of the photovoltaic support targets. The instance results include target category, bounding box and corresponding instance segmentation mask. The support targets are selected based on the instance results, and the target region image ROI is cropped in the original image according to its bounding box. The target region image ROI is input into the subsequent classification model for structural state discrimination.

[0007] Furthermore, in step S2, the anomaly detection model based on ViT assesses the degree of anomaly of the target, which includes the following sub-steps: Step S201: Obtain the target region image of the current frame and input the target image into the ViT encoder: , in, E(·) is the ViT encoder, and the image feature representation is T. t-1 It consists of one global feature and multiple image patch features. After removing the global feature, a block-level feature sequence composed of multiple image patch features is obtained: , in, N is the number of image patches, and D is the feature dimension; Step S202: Generate a mask vector on the feature grid of the image. Based on the generated masking markers, the masked features are suppressed, and a sequence of missing image features is constructed. Specifically, it is expressed as: , in, This is element-wise multiplication; Step S203: Subsequently, the original feature sequence and the missing feature sequence are input into the decoder D(·) respectively for image reconstruction, resulting in a complete reconstructed image and a missing reconstructed image: , in, This represents the reconstruction result based on the original features. This indicates the reconstruction results based on the defect features. By comparing the reconstruction differences between the two, the degree of anomaly in the target region can be characterized, and supervision can be provided for subsequent anomaly scoring and structural state discrimination training. Step S204: Aggregate the local feature sequences to obtain image-level feature vectors, and output the anomaly score through the anomaly scoring branch g(·): , , Where pool(·) is the pooling operation, and g(·) is the linear layer. and The score indicates the degree of abnormality. The higher the abnormality score, the more likely the structure is to be incomplete or abnormal.

[0008] Furthermore, in step S2, to ensure consistency during model training, the ability to reconstruct occluded parts, and the separability of anomaly scoring, a joint loss function is constructed: , in, and Used to constrain complete image reconstruction Compared to the original image x t The consistency is composed of a weighted average of pixel-level mean square error and perceptual feature error: The mean square error and the perceived feature error function are: , , in, This represents the features extracted by the pre-trained VGG16.

[0009] Furthermore, in step S2, when constructing the joint loss function, to avoid the model relying solely on the reconstruction of unmasked regions, a masked region feature reconstruction loss is introduced. Further emphasis is placed on the quality of restoration of the masked areas; specifically, the damaged reconstructed image... Then input the encoder to obtain its local feature sequence. And only when masked by marker M i At the position corresponding to =1, the reconstructed features are constrained to be consistent with the original features: , Where, p t,i For the original local feature sequence p t-1 The i-th feature unit, The corresponding feature units are obtained by re-encoding after reconstruction, where N is the number of local feature units. Simultaneously, to enable the anomaly score to distinguish between complete and incomplete inputs, a ranking constraint loss is constructed. Make the anomaly score for incomplete inputs higher than the anomaly score for complete inputs: , in, and These are the anomaly scores output by the anomaly scoring branch for complete and missing features, respectively, where m is the interval constant, based on the aforementioned joint loss. The model parameters are iteratively updated to obtain the model. The anomaly evaluation model is used as an auxiliary guide in the subsequent training phase of the classification model to guide the classification model to learn a more stable discrimination boundary.

[0010] Furthermore, in step S3, the SwinViT-ae multi-classification method based on Swin Transformer includes the following sub-steps: Step S301: Obtain the final output target region image and perform preprocessing operations on it. During training, the target region image is randomly cropped and then randomly scaled to a fixed size and randomly horizontally flipped. Tensor quantization and normalization are then performed. During the inference stage, scaling and center cropping are used followed by normalization. The normalization is expressed as: ; Step S302: Input the preprocessed image x' into the classification network f(·) with the Swing Transformer as the backbone to obtain the category output: z=f(x'), The probability of each structural state category is obtained through Softmax: p = Softmax(z), and the output category is... Confidence level is ; Step S303: In the window-based multi-head self-attention computation process, an attention gating mechanism is introduced into the attention module to adaptively adjust the response intensity of different attention heads, and to generate gating weights for the feature sequence X composed of several image patch features within the window: , Among them W g With b g For learnable parameters, The sigmoid function is used to restrict the gating coefficients to the interval [0,1]. Subsequently, after obtaining the outputs of each head of the multi-head self-attention system, these outputs are scaled and modulated to obtain the gated attention output. , Attn represents the multi-head self-attention mechanism. This indicates element-wise multiplication; Step S304: During the training phase, the ViT-AE obtained in step S2 is introduced as the teacher network AE(·), and its parameters are frozen, used only to output the degree of anomaly in the target region image. Specifically, the target region image x' is input into the network to obtain the reconstructed image. x rec =AE(x') And calculate pixel-level reconstruction error as an indicator of anomaly: e = Mean(|x rec -x'|), Mean represents the average value, which is then normalized for the degree of anomalies within the same batch: , Among them, the degree of abnormality Used to apply weighted constraints to the output of the classification network, while... To prevent the denominator from being zero, an SPDG-constrained loss is then constructed, which suppresses the probability of anomalous data for complete samples and the probability of non-nominal data for incomplete samples, and is adaptively weighted according to the degree of anomalousness. , , , , in, represents the probability that the i-th sample is classified as an incomplete category by the classification network, which is obtained from the Softmax output; Ⅱ[·] is an indicator function, which takes the value 1 when the condition in parentheses is true, and 0 otherwise, and is used to select the complete sample set and the incomplete sample set from the batch sample respectively; the denominator contains is a small constant used to prevent the denominator from being zero when a certain type of sample is missing in a batch, and to play a role in numerical stability; weight W normal With W abn W is an adaptive weighting coefficient based on the degree of anomaly e, β is the weight sensitivity coefficient, and m is the offset constant; when the sample anomaly degree is high, W abn Increase W to strengthen the constraint on incomplete samples; when the sample anomaly level is low, W... normal Increase the size to strengthen the constraint on the complete sample, thereby achieving consistency constraint; The final loss function is: During training, the network parameters are iteratively updated using an optimizer.

[0011] Furthermore, in step S302, during the training process, the algorithm calculates the cross-entropy loss between z and the true label y: , The AdamW optimization strategy was then used to update the network parameters.

[0012] The beneficial effects of this invention are: 1. This invention utilizes the Detectron2 instance segmentation model to detect components such as photovoltaic brackets and columns. Compared to conventional target detection schemes that only provide rectangular bounding boxes, it outputs instance results with pixel-level segmentation masks, thereby stably separating components from the background and accurately depicting the true contours. The backbone network is replaced with a Swing Transformer, which leverages its window self-attention mechanism to model fine-grained structural features within local regions. Furthermore, hierarchical feature aggregation introduces more comprehensive global contextual information. This enhances the representation of key component boundaries and structural textures, even when slender bracket components are easily obscured by the background or when there is significant overlap between rows of components. This improves the localization accuracy and stability of instance segmentation, and reduces false positives and false negatives caused by complex backgrounds such as vegetation and cables.

[0013] 2. The present invention, based on ViT encoding-decoding reconstruction and anomaly scoring, introduces a masking mechanism to construct defects in the target region image and reconstructs and compares the original features and defect features respectively. This explicitly enhances the model's sensitivity to anomaly patterns from the training level. At the same time, by constraining the reconstruction of masked region features, the network focuses on learning the structural recovery ability of the masked position, thereby improving the detection ability, robustness and generalization ability of unknown anomalies.

[0014] 3. This invention introduces a gating modulation mechanism in the attention calculation of the SwinViT-ae multi-classification method based on the Swin Transformer. This mechanism adaptively suppresses and enhances the responses of different attention heads, reducing interference from desert backgrounds, cables, and other factors on the discriminative features. Furthermore, the anomaly level output by ViT-AE is used as an auxiliary supervision signal, and the classification output is weighted and constrained by the SPDG constraint loss function. This allows the model to use anomaly strength information to correct the decision boundary during the training phase, thereby reducing false positives and false negatives and improving classification robustness and generalization ability, even in cases of incomplete and diverse morphologies where fixed categories cannot cover all anomaly patterns. Attached Figure Description

[0015] Figure 1 This is a segmentation and detection effect diagram of an example of the guide rail bracket of the present invention; Figure 2 This is a segmented inspection diagram of an example of the flat single-axis bracket of the present invention; Figure 3 This is a flowchart of the ViT-AE anomaly detection algorithm of the present invention; Figure 4 This is a photomask diagram of the photovoltaic support structure of the present invention; Figure 5 This is a diagram showing the convergence effect of the ViT-AE training loss function of this invention; Figure 6 This is a flowchart of the SwinViT-ae algorithm of the present invention; Figure 7 This is a Grad-CAM diagram of the photovoltaic bracket of the present invention; Figure 8 This is a final classification effect diagram of the guide rail photovoltaic bracket of the present invention; Figure 9 This is a final classification effect diagram of the single-axis photovoltaic bracket of the present invention; Figure 10 This is a diagram illustrating the final classification effect from the first special perspective of the present invention. Figure 11 This is a final classification effect diagram from the second special perspective of the present invention; Figure 12 This is an overall flowchart of the photovoltaic support structure condition detection and classification method of the present invention; Figure 13 This is a flowchart of the photovoltaic support structure condition detection and classification method of the present invention. Detailed Implementation

[0016] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0017] A method for detecting and classifying the structural condition of photovoltaic support structures, such as Figure 12 and Figure 13 As shown, the method includes the following steps: Step S1: Detect the support frame and column based on the instance segmentation model Detectron2 with Transformer as the backbone network, obtain the target segmentation mask, and provide accurate target region information for subsequent structural state discrimination; Step S2: For the obtained target segmentation mask, use the ViT-based anomaly detection model to evaluate the degree of anomaly of the target and output the anomaly score as the basis for structural state discrimination; Step S3: Using the obtained anomaly assessment information as auxiliary supervision, a classification network based on Swin Transformer is used, and SPDG constraint loss is introduced to jointly constrain the classification training, thereby realizing the classification of the photovoltaic support structure state.

[0018] In step S1, the photovoltaic support detection based on the instance segmentation model Detectron2 includes the following sub-steps: Step S101: The robot is positioned at the center of the photovoltaic power station row, and the RGB camera is installed in the middle of the robot to acquire a real-time one-frame image; Step S102: Using the instance segmentation model Detectron2, its feature extraction backbone is replaced with a Transformer architecture to detect photovoltaic support targets (e.g., rail-mounted photovoltaic supports, flat single-axis photovoltaic supports, and their columns). Instance segmentation inference is performed on an acquired image frame to obtain instance results for the photovoltaic support targets. Instance results include target category, bounding box, and corresponding instance segmentation mask. Support targets are selected based on the instance results, and the target region image ROI is cropped from the original image according to its bounding box. The target region image ROI is then input into the subsequent classification model for structural state discrimination. The instance segmentation detection effect is as follows: Figure 1 , Figure 2 As shown.

[0019] In step S2, the anomaly detection model based on ViT evaluates the degree of anomaly of the target, which includes the following sub-steps: Step S201: Obtain the target region image of the current frame and input the target image into the ViT encoder: , in, E(·) is the ViT encoder, and the image feature representation is T. t-1 It consists of one global feature and multiple image patch features. After removing the global feature, a block-level feature sequence composed of multiple image patch features is obtained: , in, N is the number of image patches, and D is the feature dimension; Step S202: To enable the anomaly detection method to better identify incomplete scaffolds, a mask vector is generated on the feature grid of the image. Based on the generated masking markers, the masked features are suppressed, and a sequence of missing image features is constructed. Specifically, it is expressed as: , in, For element-wise multiplication, the effect is as follows: Figure 4 As shown; Step S203: Subsequently, the original feature sequence and the missing feature sequence are input into the decoder D(·) respectively for image reconstruction, resulting in a complete reconstructed image and a missing reconstructed image: , in, This represents the reconstruction result based on the original features. This indicates the reconstruction results based on the defect features. By comparing the reconstruction differences between the two, the degree of anomaly in the target region can be characterized, and supervision can be provided for subsequent anomaly scoring and structural state discrimination training. Step S204: To obtain a directly usable anomaly quantification index, the local feature sequences are aggregated to obtain an image-level feature vector, and the anomaly score is output through the anomaly scoring branch g(·). , , Where pool(·) is the pooling operation, and g(·) is the linear layer. and The score indicates the degree of abnormality. The higher the abnormality score, the more likely the structure is to be incomplete or abnormal.

[0020] In step S2, to ensure consistency during model training, the ability to reconstruct occluded parts, and the separability of anomaly scoring, a joint loss function is constructed: , in, and Used to constrain complete image reconstruction Compared to the original image x t The consistency is composed of a weighted average of pixel-level mean square error and perceptual feature error: The mean square error and the perceived feature error function are: , , in, This represents the features extracted by the pre-trained VGG16.

[0021] In step S2, when constructing the joint loss function, to avoid the model relying solely on the reconstruction of unmasked regions, a masked region feature reconstruction loss is introduced. Further emphasis is placed on the quality of restoration of the masked areas; specifically, the damaged reconstructed image... Then input the encoder to obtain its local feature sequence. And only when masked by marker M i At the position corresponding to =1, the reconstructed features are constrained to be consistent with the original features: , Where, p t,i For the original local feature sequence p t-1 The i-th feature unit, The corresponding feature units are obtained by re-encoding after reconstruction, where N is the number of local feature units. Simultaneously, to enable the anomaly score to distinguish between complete and incomplete inputs, a ranking constraint loss is constructed. Make the anomaly score for incomplete inputs higher than the anomaly score for complete inputs: , in, and These are the anomaly scores output by the anomaly scoring branch for complete and missing features, respectively, where m is the interval constant, based on the aforementioned joint loss. The model parameters are iteratively updated to obtain the model, and the loss function converges as follows: Figure 5 As shown, the anomaly assessment model is used as an auxiliary guide in the subsequent training phase of the classification model, in order to guide the classification model to learn a more stable discrimination boundary.

[0022] In step S3, such as Figure 6 As shown, the SwinViT-ae multi-classification method based on Swin Transformer includes the following sub-steps: Step S301: Obtain the final output target region image and perform preprocessing operations on it. During training, the target region image is randomly cropped and then randomly scaled to a fixed size and randomly horizontally flipped. Tensor quantization and normalization are then performed. During the inference stage, scaling and center cropping are used followed by normalization. The normalization is expressed as: ; Step S302: Input the preprocessed image x' into the classification network f(·) with the Swing Transformer as the backbone to obtain the category output: z=f(x'), The probability of each structural state category is obtained through Softmax: p = Softmax(z), and the output category is... Confidence level is Specifically, during training, the algorithm calculates the cross-entropy loss between z and the true label y: , The AdamW optimization strategy was then used to update the network parameters. Step S303: In the window-based multi-head self-attention computation process, to address the feature interference problem caused by complex backgrounds and multi-component occlusion, this method introduces an attention gating mechanism in the attention module to adaptively adjust the response intensity of different attention heads and generate gating weights for the feature sequence X composed of several image patch features within the window: , Among them W g With b g For learnable parameters, The sigmoid function is used to restrict the gating coefficients to the interval [0,1]. Subsequently, after obtaining the outputs of each head of the multi-head self-attention system, these outputs are scaled and modulated to obtain the gated attention output. , Attn represents the multi-head self-attention mechanism. This indicates element-wise multiplication; through gating modulation, attention heads with strong background interference can be suppressed, while attention heads related to the key structure of the scaffold can be enhanced, thereby improving the discrimination robustness and generalization ability in complex scenes, with effects such as... Figure 7 As shown; Step S304: During the training phase, to enhance the robustness of recognizing diverse incomplete states, this method introduces the ViT-AE trained in step S2 as the teacher network AE(·), and freezes its parameters, using it only to output the degree of anomaly in the target region image. Specifically, the target region image x' is input into the network to obtain the reconstructed image: x rec =AE(x') And calculate pixel-level reconstruction error as an indicator of anomaly: e = Mean(|x rec -x'|), Mean represents the average value, which is then normalized for the degree of anomalies within the same batch: , Among them, the degree of abnormality Used to apply weighted constraints to the output of the classification network, while... To prevent the denominator from being zero, an SPDG-constrained loss is then constructed, which suppresses the probability of anomalous data for complete samples and the probability of non-nominal data for incomplete samples, and is adaptively weighted according to the degree of anomalousness. , , , , in, represents the probability that the i-th sample is classified as an incomplete category by the classification network, which is obtained from the Softmax output; Ⅱ[·] is an indicator function, which takes the value 1 when the condition in parentheses is true, and 0 otherwise, and is used to select the complete sample set and the incomplete sample set from the batch sample respectively; the denominator contains is a small constant used to prevent the denominator from being zero when a certain type of sample is missing in a batch, and to play a role in numerical stability; weight W normal With W abn W is an adaptive weighting coefficient based on the degree of anomaly e, β is the weight sensitivity coefficient, and m is the offset constant; when the sample anomaly degree is high, W abn Increase W to strengthen the constraint on incomplete samples; when the sample anomaly level is low, W... normal Increase the size to strengthen the constraint on the complete sample, thereby achieving consistency constraint; The final loss function is: During training, the network parameters are iteratively updated using an optimizer.

[0023] The final effect of the model obtained by the above method is as follows: Figure 8 , Figure 9 , Figure 10 , Figure 11 As shown.

[0024] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting and classifying the structural condition of photovoltaic support structures, characterized in that, The method includes the following steps: Step S1: Detect the support frame and column based on the instance segmentation model Detectron2 with Transformer as the backbone network, obtain the target segmentation mask, and provide accurate target region information for subsequent structural state discrimination; Step S2: For the obtained target segmentation mask, use the ViT-based anomaly detection model to evaluate the degree of anomaly of the target and output the anomaly score as the basis for structural state discrimination; Step S3: Using the obtained anomaly assessment information as auxiliary supervision, a classification network based on Swin Transformer is used, and SPDG constraint loss is introduced to jointly constrain the classification training, thereby realizing the classification of the photovoltaic support structure state.

2. The photovoltaic support structure condition detection and classification method according to claim 1, characterized in that, In step S1, the photovoltaic support detection based on the instance segmentation model Detectron2 includes the following sub-steps: Step S101: The robot is positioned at the center of the photovoltaic power station row, and the RGB camera is installed in the middle of the robot to acquire a real-time one-frame image; Step S102: The feature extraction backbone of the instance segmentation model Detectron2 is replaced with the Transformer architecture to detect photovoltaic support targets. Instance segmentation inference is performed on an acquired frame image to obtain instance results of the photovoltaic support targets. The instance results include target category, bounding box and corresponding instance segmentation mask. The support targets are selected based on the instance results, and the target region image ROI is cropped in the original image according to its bounding box. The target region image ROI is input into the subsequent classification model for structural state discrimination.

3. The photovoltaic support structure condition detection and classification method according to claim 2, characterized in that, In step S2, the anomaly detection model based on ViT evaluates the degree of anomaly of the target, which includes the following sub-steps: Step S201: Obtain the target region image of the current frame and input the target image into the ViT encoder: ， in, E(·) is the ViT encoder, and the image feature representation is T. t-1 It consists of one global feature and multiple image patch features. After removing the global feature, a block-level feature sequence composed of multiple image patch features is obtained: ， in, N is the number of image patches, and D is the feature dimension; Step S202: Generate a mask vector on the feature grid of the image. Based on the generated masking markers, the masked features are suppressed, and a sequence of missing image features is constructed. Specifically, it is expressed as: ， in, This is element-wise multiplication; Step S203: Subsequently, the original feature sequence and the missing feature sequence are input into the decoder D(·) respectively for image reconstruction, resulting in a complete reconstructed image and a missing reconstructed image: ， in, This represents the reconstruction result based on the original features. This indicates the reconstruction results based on the defect features. By comparing the reconstruction differences between the two, the degree of anomaly in the target region can be characterized, and supervision can be provided for subsequent anomaly scoring and structural state discrimination training. Step S204: Aggregate the local feature sequences to obtain image-level feature vectors, and output the anomaly score through the anomaly scoring branch g(·): ， ， Where pool(·) is the pooling operation, and g(·) is the linear layer. and The score indicates the degree of abnormality. The higher the abnormality score, the more likely the structure is to be incomplete or abnormal.

4. The photovoltaic support structure condition detection and classification method according to claim 3, characterized in that, In step S2, to ensure consistency during model training, the ability to reconstruct occluded parts, and the separability of anomaly scoring, a joint loss function is constructed: ， in, and Used to constrain complete image reconstruction Compared to the original image x t The consistency is composed of a weighted average of pixel-level mean square error and perceptual feature error: The mean square error and the perceived feature error function are: ， ， in, This represents the features extracted by the pre-trained VGG16.

5. The photovoltaic support structure condition detection and classification method according to claim 4, characterized in that, In step S2, when constructing the joint loss function, to avoid the model relying solely on the reconstruction of unmasked regions, a masked region feature reconstruction loss is introduced. Further emphasis is placed on the quality of restoration of the masked areas; specifically, the damaged reconstructed image... Then input the encoder to obtain its local feature sequence. And only when masked by marker M i At the position corresponding to =1, the reconstructed features are constrained to be consistent with the original features: ， Where, p t,i For the original local feature sequence p t-1 The i-th feature unit, The corresponding feature units are obtained by re-encoding after reconstruction, where N is the number of local feature units. Simultaneously, to enable the anomaly score to distinguish between complete and incomplete inputs, a ranking constraint loss is constructed. Make the anomaly score for incomplete inputs higher than the anomaly score for complete inputs: ， in, and These are the anomaly scores output by the anomaly scoring branch for complete and missing features, respectively, where m is the interval constant, based on the aforementioned joint loss. The model parameters are iteratively updated to obtain the model. The anomaly evaluation model is used as an auxiliary guide in the subsequent training phase of the classification model to guide the classification model to learn a more stable discrimination boundary.

6. The method for detecting and classifying the structural condition of photovoltaic support structures according to claim 3, characterized in that, In step S3, the SwinViT-ae multi-classification method based on Swin Transformer includes the following sub-steps: Step S301: Obtain the final output target region image and perform preprocessing operations on it. During training, the target region image is randomly cropped and then randomly scaled to a fixed size and randomly horizontally flipped. Tensor quantization and normalization are then performed. During the inference stage, scaling and center cropping are used followed by normalization. The normalization is expressed as: ； Step S302: Input the preprocessed image x' into the classification network f(·) with the Swing Transformer as the backbone to obtain the category output: z=f(x'), The probability of each structural state category is obtained through Softmax: p = Softmax(z), and the output category is... Confidence level is ; Step S303: In the window-based multi-head self-attention computation process, an attention gating mechanism is introduced into the attention module to adaptively adjust the response intensity of different attention heads, and to generate gating weights for the feature sequence X composed of several image patch features within the window: ， Among them W g With b g For learnable parameters, The sigmoid function is used to restrict the gating coefficients to the interval [0,1]. Subsequently, after obtaining the outputs of each head of the multi-head self-attention system, these outputs are scaled and modulated to obtain the gated attention output. ， Attn represents the multi-head self-attention mechanism. This indicates element-wise multiplication; Step S304: During the training phase, the ViT-AE obtained in step S2 is introduced as the teacher network AE(·), and its parameters are frozen, used only to output the degree of anomaly in the target region image. Specifically, the target region image x' is input into the network to obtain the reconstructed image. x rec =AE(x'), And calculate pixel-level reconstruction error as an indicator of anomaly: e=Mean(|x rec -x'|), Mean represents the average value, which is then normalized for the degree of anomalies within the same batch: ， Among them, the degree of abnormality Used to apply weighted constraints to the output of the classification network, while... To prevent the denominator from being zero, an SPDG-constrained loss is then constructed, which suppresses the probability of anomalous data for complete samples and the probability of non-nominal data for incomplete samples, and is adaptively weighted according to the degree of anomalousness. ， ， ， ， in, represents the probability that the i-th sample is classified as an incomplete category by the classification network, which is obtained from the Softmax output; Ⅱ[·] is an indicator function, which takes the value 1 when the condition in parentheses is true, and 0 otherwise, and is used to select the complete sample set and the incomplete sample set from the batch sample respectively; the denominator contains is a small constant used to prevent the denominator from being zero when a certain type of sample is missing in a batch, and to play a role in numerical stability; weight W normal With W abn W is an adaptive weighting coefficient based on the degree of anomaly e, β is the weight sensitivity coefficient, and m is the offset constant; when the sample anomaly degree is high, W abn Increase W to strengthen the constraint on incomplete samples; when the sample anomaly level is low, W... normal Increase the size to strengthen the constraint on the complete sample, thereby achieving consistency constraint; The final loss function is: During training, the network parameters are iteratively updated using an optimizer.

7. The photovoltaic support structure condition detection and classification method according to claim 6, characterized in that, In step S302, during the training process, the cross-entropy loss between z and the true label y is calculated: ， The AdamW optimization strategy was then used to update the network parameters.

Images