Photovoltaic panel surface defect detection method based on hybrid visual converter architecture

Abstract

The invention discloses a photovoltaic panel surface defect detection method based on a hybrid visual converter framework. The method comprises the following steps: collecting and sorting typical defect images shot by a photovoltaic power station; and in combination with artificial visual analysis and data statistical characteristics, performing defect type labeling on the image data set by using a LabelImg tool. In order to improve the detection precision, structural optimization is carried out on the basis of an original YOLO11n model, and a fast hybrid vision converter FastViT (Fast Hybrid Vision Transformation) adopting a structural re-parameterization mechanism is introduced as a backbone network so as to enhance the feature extraction capability; the method comprises the following steps of: introducing a spatial cross scale attention mechanism (SCSA, Spatial CrossScale Attention) into a neck network, so as to improve the fusion effect of multi-scale features; and an original feature pyramid module is replaced by a lightweight and efficient SimSPPF structure, so that an improved model with better performance is constructed. After a photovoltaic panel image containing three typical defects is input into the trained improved model, a defect detection result can be output, and the category, the position and the confidence coefficient of the defects can be accurately positioned.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer technology, in particular to target detection technology, and more particularly to a method for accurate and rapid detection of multiple types of defects of photovoltaic panels. BACKGROUND

[0002] As an important part of new energy power generation systems, the safe and stable operation of photovoltaic power stations is of great significance to the efficiency of the power system. However, defects on the surface of photovoltaic panels (such as cracks, dust, bird droppings, etc.) can significantly affect the conversion efficiency of photovoltaic modules, and in severe cases, may even cause safety accidents such as malfunctions. Therefore, how to achieve real-time and accurate detection of defects on the surface of photovoltaic panels has become a key technical problem in ensuring the safe operation of photovoltaic power stations.

[0003] Existing photovoltaic panel defect detection methods mostly rely on manual inspection or traditional sensors (such as temperature, smoke, etc.) for anomaly recognition. These methods are inefficient and easily limited by environmental changes, and have obvious problems of missed detection and false detection. At the same time, in the face of complex background lighting (such as day-night changes, obstructions, etc.), the detection accuracy and robustness of traditional image recognition methods cannot meet the actual application requirements.

[0004] In recent years, with the development of computer vision and deep learning, target detection algorithms have made significant achievements in the field of image recognition. The YOLO series (You Only Look Once) model has become one of the mainstream target detection frameworks due to its fast detection speed and lightweight structure. However, the original YOLO model still has insufficient detection ability in dealing with small targets or complex backgrounds, especially in the fine-grained recognition task of photovoltaic panel defects, which is prone to inaccurate recognition or missed detection. Therefore, there is an urgent need for a detection method that can adapt to the special environment of photovoltaic power stations and balance accuracy and real-time performance, in order to improve the overall level of photovoltaic panel defect prevention and control.

[0005] In order to solve the above technical defects, the patent with application number CN202410784086.5 proposes a solar photovoltaic panel defect detection method based on YOLOv8n, which includes the following steps: 1) Data augmentation is performed on the defect image; 2) A Dyc_C2 module is constructed, replacing the C2f module of the Backbone network of YOLOv8n with the Dyc_C2 module; 3) The Dyc_C2 module first passes the input feature map through a first Conv module, then through a Split module, and then through a Bottleneck block, followed by channel dimension tensor concatenation with the feature map branch passing through the Split module and the component passing through the second Conv module, and finally passing through a third Conv module to obtain the feature map; 4) Evaluate the YOLOv8n network model using the map index.

[0006] The prior art solves the problem that the existing model backbone network cannot efficiently extract features, and the robustness and accuracy still need to be further improved. Although it can realize the recognition of photovoltaic panel defects, the image recognition benchmark model used has low accuracy, and it is difficult to accurately and timely recognize, and cannot effectively predict the photovoltaic panel defect situation, which may lead to missed detection.

[0007] Therefore, the applicant proposes a photovoltaic panel surface defect detection method based on a hybrid visual transformer architecture. SUMMARY

[0008] The present application aims to break through the limitations of existing photovoltaic panel defect recognition technology in complex environments. The existing method often has difficulty in accurately recognizing and timely detecting defect types when facing changing light conditions, dust obstruction, surface reflection and other environmental interference factors. At the same time, the feature difference between different types of defects is small, and it is also easy to be confused with environmental noise, leading to misjudgment or missed detection of the recognition result, which seriously restricts the intelligent level and detection efficiency of photovoltaic system operation and maintenance. Therefore, the present application proposes a high-precision and strong-robustness photovoltaic panel defect recognition method to improve the detection performance and reliability in actual complex working conditions.

[0009] In order to solve the above technical problems, the technical scheme adopted by the present application is: A photovoltaic panel surface defect detection method based on a hybrid visual transformer architecture, comprising the following steps: Step 1: Obtain photovoltaic cell panel data set images in a photovoltaic power station, and construct a photovoltaic panel surface defect data set; Step 2: Label the target data set, and label each collected image, construct a photovoltaic panel surface defect data set, and divide the data set into a training set, a validation set and a test set; Step 3: Obtain an improved YOLO11 model, and use the model to recognize the collected photovoltaic panel surface defect images; Step 4: Verify the model, including testing its robustness and detection accuracy under a single target; ensure that the model can effectively recognize photovoltaic panel surface defects in simple and complex target cases, and guarantee accurate detection capability.

[0010] In step 1, the following steps are specifically included: Step S1.1: Collect typical photovoltaic panel surface defect case data; Step S1.2: Carefully select and organize related case images; Step S1.3: Preprocess the data set to reduce the inaccuracy and contingency of part of the data set images.

[0011] In step S1.1, the case includes different sizes, different types, different application environments of photovoltaic panel defects; In step S1.2, the image should clearly show the specific features of the photovoltaic panel surface defects, including three typical defect forms of photovoltaic panel cracks, photovoltaic panel bird droppings and photovoltaic panel dirt accumulation; In step S1.3, the data preprocessing method includes histogram equalization, Gaussian filtering and Gamma correction.

[0012] In step S2, the following steps are included: S2.1: Labeling in Anaconda environment, using labelimg image labeling tool to label the collected photovoltaic panel surface defect images, identifying objects into four categories of photovoltaic panel, bird droppings, dust and cracks, among which photovoltaic panel is labeled as "panel", bird droppings is labeled as "bird-drop", dust is labeled as "dusty", and cracks are labeled as "cracked"; get the PascalVOC format data label, and then convert it into a txt file suitable for training through the data format conversion code written by python, which includes most of the data, including the x coordinate, y coordinate, width and height of the bounding box center point and other labeling information; S2.2: After completing the labeling work, the data set is divided to obtain the training set, validation set and test set of the photovoltaic panel defect data set.

[0013] The improved YOLO11 model is specifically: The initialized input image is input to the FastViT backbone network module, the output of the FastViT backbone network module is connected to the input of the SimSPPF module, the output of the SimSPPF module is connected to the input of the C2PSA module, the output of the C2PSA module is connected to the input of the first Upsample module and the input of the fourth C3k2 module, the output of the first Upsample module is connected to the input of the first Concat module, the output of the first Concat module is connected to the input of the first C3k2 module, the output of the first C3k2 module is connected to the input of the first SCSA module, the output of the first SCSA module is connected to the input of the second Upsample module, the output of the second Upsample module is connected to the input of the second Concat module, the output of the second Concat module is connected to the input of the second C3k2 module, the output of the second C3k2 module is connected to the input of the second SCSA module, the output of the second SCSA module is connected to the input of the first 11Detect detection head module and the input of the first Conv module, the output of the first Conv module is connected to the input of the third Concat module, the output of the third Concat module is connected to the input of the third C3k2 module, the output of the third C3k2 module is connected to the input of the third SCSA module, the output of the third SCSA module is connected to the input of the second 11Detect detection head module and the input of the second Conv module, the output of the second Conv module is connected to the input of the fourth Concat module, the output of the fourth Concat module is connected to the input of the fourth C3k2 module, and the output of the fourth C3k2 module is connected to the input of the third 11Detect detection head module; The SimSPPF feature pyramid structure is specifically: The initialized input feature of the SimSPPF feature pyramid module is input to the first ConvBNReLU module, the output of the first ConvBNReLU module is connected to the input of the first Maxpool2d module, the output of the first Maxpool2d module is connected to the input of the splicing module and the input of the second Maxpool2d module, the output of the second Maxpool2d module is connected to the input of the splicing module and the input of the third Maxpool2d module, the output of the third Maxpool2d module is connected to the input of the splicing module, the output of the splicing module is connected to the input of the second ConvBNReLU module, and the output of the second ConvBNReLU module is the refined feature.

[0014] The SCSA spatial cross-scale attention mechanism module is specifically: The initialization input of the SCSA module is input to the input of the X Avgpool module, the input of the Y Avgpool module, and the input of the first matrix point multiplication module. The output of the X Avgpool module is connected to the input of the first Spilt module. The output of the first Spilt module is connected to the input of the DWConv1d module with a convolution kernel of 3*3, the input of the DWConv1d module with a convolution kernel of 5*5, the input of the DWConv1d module with a convolution kernel of 7*7, and the input of the DWConv1d module with a convolution kernel of 9*9. The output of the DWConv1d module with a convolution kernel of 3*3, the output of the DWConv1d module with a convolution kernel of 5*5, the output of the DWConv1d module with a convolution kernel of 7*7, and the output of the DWConv1d module with a convolution kernel of 9*9 are connected to the input of the first Concat module under the corresponding input branch of the X Avgpool module. The output of the first Concat module is connected to the input of the first GroupNorm module. The output of the first GroupNorm module is connected to the input of the first Sigmoid module. The output of the first Sigmoid module is connected to the input of the first matrix point multiplication module. The output of the Y Avgpool module is connected to the input of the second Spilt module. The output of the second Spilt module is connected to the input of the DWConv1d module with a convolution kernel of 3*3, the input of the DWConv1d module with a convolution kernel of 5*5, the input of the DWConv1d module with a convolution kernel of 7*7, and the input of the DWConv1d module with a convolution kernel of 9*9. The output of the DWConv1d module with a convolution kernel of 3*3, the output of the DWConv1d module with a convolution kernel of 5*5, the output of the DWConv1d module with a convolution kernel of 7*7, and the output of the DWConv1d module with a convolution kernel of 9*9 are connected to the input of the second Concat module under the corresponding input branch of the Y Avgpool module. The output of the second Concat module is connected to the input of the second GroupNorm module. The output of the second GroupNorm module is connected to the input of the second Sigmoid module. The output of the second Sigmoid module is connected to the input of the first matrix point multiplication module. This process is the whole process of the SMSA module. Then, the output of the first matrix point multiplication module is connected to the input of the first Avgpool, the output of the first Avgpool is connected to the input of the third GroupNorm module, the output of the third GroupNorm module is connected to the input of the DWConv2d module with the first convolution kernel of 1x1, the input of the DWConv2d module with the second convolution kernel of 1x1, the input of the DWConv2d module with the third convolution kernel of 1x1, the output of the DWConv2d module with the first convolution kernel of 1x1, the output of the DWConv2d module with the second convolution kernel of 1x1 is connected to the input of the first matrix multiplication module, the output of the first matrix multiplication module is connected to the input of the first Softmax module, the output of the first Softmax module and the output of the third DWConv2d module with the connection kernel of 1x1 are connected to the input of the second Avgpool module, the output of the second Avgpool module is connected to the input of the third Sigmoid module, the output of the third Sigmoid module and the output of the first matrix point multiplication module are connected to the input of the second matrix point multiplication module; this process is the whole process of the PSCA module. The output of the second matrix point multiplication module is the refined feature.

[0015] The Fast Hybrid Vision Transformer using Structural Reparameterization (FastViT) is specifically: The initialization input feature of the FastViT module is input to the input of the Stem module, the output of the Stem module is connected to the input of the Stage1 module, the output of the Stage1 module is connected to the input of the Patch Emb module with the first step of 2, the output of the Patch Emb module with the first step of 2 is connected to the input of the Stage2 module, the output of the Stage2 module is connected to the input of the Patch Emb module with the second step of 2, the output of the Patch Emb module with the second step of 2 is connected to the input of the Stage3 module, the output of the Stage3 module is connected to the input of the Patch Emb module with the third step of 2, the output of the Patch Emb module with the third step of 2 is connected to the input of the Stage4 module, the output of the Stage4 module is connected to the input of the DW MobileOne Block module with the convolution kernel of 3x3, the output of the DW MobileOne Block module with the convolution kernel of 3x3 is connected to the input of the Avg Pool module, the output of the Avg Pool module is connected to the input of the FC module, and the output of the FC module is the refined feature.

[0016] The structure of the Patch Emb module with a step of 2 is specifically as follows: the initial input feature of the Patch Emb module with a step of 2 is input to an input of a DW Conv module with a first convolution kernel of 7*7 and a step of 2, and an input of a DW Conv module with a second convolution kernel of 7*7 and a step of 2; an output of the DW Conv module with the first convolution kernel of 7*7 and the step of 2 is connected to an input of a first BatchNorm module, an output of the first BatchNorm module is connected to an input of a first matrix element-wise addition module, an output of the DW Conv module with the second convolution kernel of 7*7 and the step of 2 is connected to an input of a second BatchNorm module, and an output of the second BatchNorm module is connected to an input of the first matrix element-wise addition module; an output of the first matrix element-wise addition module is connected to an input of a first Attention module, an output of the first Attention module is connected to an input of a Conv module with a convolution kernel of 1*1 and an input of a fourth BatchNorm module, an output of the Conv module with the convolution kernel of 1*1 is connected to an input of a third BatchNorm module, an output of the third BatchNorm module and an output of the fourth BatchNorm module are connected to an input of a second matrix element-wise addition module, and an output of the second matrix element-wise addition module is connected to an input of a second Attention module. An output of the second Attention module is the refined feature.

[0017] The structure of the Stage1, Stage2 and Stage3 modules is specifically as follows: the initial input feature of the Stage1, Stage2 and Stage3 is input to an input of a BatchNorm module and an input of a matrix element-wise addition module, an output of the BatchNorm module is connected to an input of a DW Conv module with a convolution kernel of 3*3, an output of the DW Conv module with the convolution kernel of 3*3 is connected to an input of the matrix element-wise addition module, an output of the matrix element-wise addition module is connected to an input of a ConvFFN module, and an output of the ConvFFN module is the refined feature. The structure of the Stage4 module is specifically as follows: the initial input feature of the Stage4 is input to an input of a CPE module and an input of a first matrix element-wise addition module, an output of the CPE module is connected to an input of the first matrix element-wise addition module, an output of the first matrix element-wise addition module is connected to an input of a BatchNorm module, an output of the BatchNorm module is connected to an input of an Attention module, an output of the Attention module is connected to an input of a second matrix element-wise addition module, an output of the second matrix element-wise addition module is connected to an input of a ConvFFN module, and an output of the ConvFFN module is the refined feature.

[0018] In step 4, the following steps are included: Step S4.1: Verify the accuracy of the model and the photovoltaic panel defect detection ability; Step S4.2: Real-time target detection and response ability.

[0019] In step S4.1, the model is ensured to accurately detect various defects on the photovoltaic panel, regardless of the size, shape or color difference of the defect. Especially considering the subtle features of the defect, such as cracks, stains, scratches, etc., the model should be able to identify early or hidden signs of defects. Even if the defect area is small or just emerging, the model should be able to capture and mark the defect location in time, achieving early warning and maintenance; In step S4.2, in key new energy facilities such as photovoltaic power stations, the development of defects may quickly affect the performance and safety of equipment. In order to effectively ensure the stable operation of photovoltaic panels, the model needs to have rapid response ability, which can detect and locate defects in real time, and ensure timely warning before the defect expands or worsens.

[0020] Compared with the prior art, the present application has the following technical effects: 1) The present application has stable defect recognition ability in various complex scenes, especially in the presence of background interference, image occlusion, and changes in viewing angle, etc. The model can still effectively extract key features, maintain high detection accuracy and robustness. Compared with traditional methods, the present application significantly improves the adaptability of the model in a changing environment, and is suitable for diversified photovoltaic panel detection tasks in actual engineering; 2) In the scene with clear background and less interference, the model can accurately identify the typical defect features of the photovoltaic panel surface, such as the edge profile of the crack, the occlusion shape of the bird droppings, and the diffusion distribution of the dust, with high detection accuracy and positioning accuracy. This effect significantly reduces the risk of missed detection in the identification process, and improves the identification efficiency of high-quality images; 3) In the scene with more interference or poor image quality, such as complex background, target occlusion or blurred details, the model still has strong anti-interference ability. Its deep feature fusion structure can mine potential defect feature information, effectively alleviate the negative impact of occlusion and interference, significantly reduce the false detection rate, and enhance the stability and reliability of detection; 4) Even if the image contains only a single defect or the feature information is incomplete, the model can still realize independent identification and classification of different types of defects due to its feature extraction and semantic understanding ability. This ability enables the system to make reasonable judgments even in conditions of insufficient or partially missing information, avoiding false positives and false negatives, and further improving the intelligent level and practical value of the overall detection system. In summary, the present application improves the stability and accuracy of the system in various complex environments. Whether in daytime, nighttime, or different light intensity environments, the model can efficiently identify defects, ensuring superior performance in practical applications; In summary, the present application exhibits excellent photovoltaic panel defect recognition ability in various complex scenarios by introducing a multi-level feature extraction mechanism and an optimized network structure. Whether in ideal conditions with less background interference, or in complex environments with obstacles, blurring, noise, and other adverse factors, the model can stably extract key features and accurately determine defect types and locations. At the same time, even if the image information is incomplete or the defect features are single, the model also has good independent recognition ability and fault tolerance, significantly improving the accuracy, robustness, and practical value of the overall detection system.

[0021] The present application proposes a photovoltaic panel surface defect detection method based on a hybrid visual transformer architecture. The spatial cross-scale attention mechanism-SCSA is added to the neck network, and the feature pyramid structure is replaced with SimSPPF to further improve the performance of the model. The improved model can efficiently identify multiple types of defects on the surface of photovoltaic panels, especially in typical complex scenarios such as occlusion interference, complex background, and blurred details, exhibiting significant detection accuracy and robustness. By improving the YOLO11 architecture, the model enhances the attention ability to key defect areas, achieving precise positioning and classification of multiple defects such as cracks, dust, and bird droppings. This method not only has high recognition accuracy but also has good real-time detection capability, providing strong technical support for intelligent operation and fault warning of photovoltaic power stations. BRIEF DESCRIPTION OF DRAWINGS

[0022] The present application will be further described below in conjunction with the drawings and examples: Figure 1 The network structure of the present application is shown in the figure; Figure 2 The FastViT structure schematic diagram is shown in the figure; Figure 3 The SCSA structure schematic diagram is shown in the figure; Figure 4 The SimSPPF structure schematic diagram is shown in the figure; Figure 5 The photovoltaic cell panel model prediction graph containing crack defects in the present application is shown in the figure; Figure 6 Model prediction graph for a photovoltaic panel soiled in the present invention; Figure 7 Model prediction graph for multi-objective verification in the present invention; Figure 8 Model prediction graph for a photovoltaic panel with bird droppings defect in the present invention; Figure 9 Model prediction graph for a normal photovoltaic panel without defects. DETAILED DESCRIPTION

[0023] A photovoltaic panel surface defect detection method based on a hybrid visual transformer architecture, comprising the following steps: Step S1: Collect and organize relevant case images by comprehensively summarizing typical photovoltaic panel defect cases commonly found on the network, aiming to build a high-quality photovoltaic panel defect dataset. This dataset should not only cover a large number of cases, but also include various scenarios to ensure sufficient diversity and adapt to analysis needs in different environments. The images in the dataset have high resolution and excellent clarity, ensuring rich details and no distortion, facilitating subsequent image processing and analysis.

[0024] To ensure the representativeness and completeness of the dataset, it needs to be collected from multiple dimensions, including normal photovoltaic panels, photovoltaic panel cracks, bird droppings, and photovoltaic panel fouling. By ensuring that the dataset covers enough real-world scenarios, the learning ability of subsequent models can be effectively improved, providing more accurate and reliable data support for photovoltaic panel defect detection systems, automatic detection systems, and other applications; Step S2: Use the LabelImg tool to finely label the target dataset, and label each collected image. Divide the dataset according to a reasonable proportion, and finally build a substation image dataset containing photovoltaic panel defect scenarios. This dataset will be further divided into training, validation, and test sets to ensure the comprehensiveness and accuracy of subsequent model training; Step S3: Obtain the improved YOLO11n model and use it to identify the collected photovoltaic panel defect images. When evaluating model performance, use precision (Precision), recall (Recall), mean average precision (mAP), and frames per second (FPS) as main indicators.

[0025] Step S4: Verify the real-time and accuracy of the model to ensure that it can quickly and accurately identify various defects of photovoltaic panels.

[0026] YOLO11n (You Only Look Once version 11 nano) is a lightweight real-time object detection model, which is the latest variant of the YOLO series. This model continues the advantages of end-to-end and single-stage detection in the YOLO architecture, and by introducing improved feature extraction modules and efficient attention mechanisms, it significantly reduces the computational complexity while improving detection accuracy, especially suitable for edge devices and real-time application scenarios. Compared with methods such as Faster R-CNN and DETR, YOLO11n eliminates the generation of candidate regions and complex post-processing operations, and performs well in terms of detection speed and energy consumption control.

[0027] In the YOLO11 series, the official provides multiple variants to meet the performance requirements in different scenarios, such as YOLO11n, YOLO11s, YOLO11m, YOLO11l, etc. This invention selects YOLO11n as the basic model for photovoltaic panel defect detection, which has good performance in detection accuracy with smaller network size.

[0028] Specifically, step S1 includes: S1.1: Collect typical photovoltaic panel defect case data widely Through the network platform, typical photovoltaic panel defect cases occurring at home and abroad in recent years are collected widely. In the data collection process, the authenticity and reliability of the obtained information need to be verified to avoid the influence of false or misleading information on the accuracy of subsequent analysis and model training. In addition, in order to improve the comprehensiveness and adaptability of the data set, it is necessary to ensure that the collected defect cases cover photovoltaic panel defects in different scales, different types, and different application environments, so as to enhance the diversity and representativeness of the data set and meet the adaptation requirements of the model in multiple scenarios.

[0029] S1.2: Carefully select and organize related photovoltaic panel defect images After collecting a large amount of photovoltaic panel defect case data, the obtained image data is selected and processed. Preferably, image samples with high resolution and good image clarity are selected as the basis for constructing the defect detection data set. The images should clearly show the specific features of the photovoltaic panel surface defects, including photovoltaic panel cracks, photovoltaic panel bird droppings, and photovoltaic panel dirt accumulation.

[0030] S1.3: Preprocess the data set to reduce the inaccuracy and contingency of part of the data set images. The data preprocessing methods include histogram equalization, Gaussian filtering, and Gamma correction, finally obtaining 2171 images.

[0031] The image preprocessing method is as follows: 1) Histogram equalization is a commonly used image enhancement technique that can effectively improve the contrast of images. By redistributing the gray values of pixels in the image, the image is more evenly distributed in the brightness range, thereby enhancing the visibility of the detail area. This method is especially suitable for scenes with uneven lighting or dark images, and helps the model to more accurately identify detailed information and improve the perception of small defects on the surface of photovoltaic panels.

[0032] 2) Gaussian filtering is a classic image smoothing method, mainly used to remove high-frequency noise in images. Applying Gaussian filtering in the image preprocessing stage can effectively suppress random interference introduced by the acquisition device or the environment in the image, while preserving the main edge and structure information in the image. This operation helps to improve image quality and enhance the model's attention to key areas, thereby improving the accuracy of defect detection.

[0033] 3) Gamma correction is a non-linear brightness adjustment technique that can achieve detailed adjustment of light and dark areas in the image by performing power transformation on the pixel values of the image. Introducing Gamma correction in the image preprocessing stage helps to improve the visual performance of the image, especially in less than ideal lighting conditions. Proper Gamma adjustment not only improves the distinguishability of the image, but also enhances the model's ability to extract defect features under different brightness conditions.

[0034] After the above preprocessing steps, the constructed dataset has been significantly improved in terms of image quality, content diversity and expression accuracy, and a total of 2171 image samples have been obtained. This data size can effectively support subsequent model training and performance evaluation, and also lays a solid foundation for the improvement of algorithm accuracy and robustness.

[0035] To further enhance the generalization ability of the model and the diversity of the data, more image preprocessing and enhancement strategies can be introduced on the basis of the existing ones, such as scaling, cropping, flipping operations, or simulating noise interference and image blur conditions that may occur in actual application scenarios. Through the above expansion means, the size of the dataset can be doubled, thereby providing more sufficient and comprehensive data support for subsequent algorithm optimization and model iteration.

[0036] Specifically, step S2 includes: S2.1: In the Anaconda environment, the collected images are labeled using the labelimg image labeling tool. The identified objects are classified into four categories: photovoltaic panels, cracks, bird droppings, and dust accumulation. The photovoltaic panel is labeled as "panel", the crack is labeled as "cracked", the bird droppings are labeled as "bird_drop", and the dust accumulation is labeled as "dusty". The Pascal VOC format data label (XML label) is obtained, and then the data format conversion code written in python is used to convert it into a txt file suitable for training. The file includes most of the data, including the x coordinate, y coordinate, width, and height of the bounding box center point, and other labeling information.

[0037] S2.2: After completing the labeling work, the data set is divided to obtain the training set, validation set, and test set of the photovoltaic panel defect data set, resulting in 1948 training sets, 158 validation sets, and 65 test sets. Specifically, step S3 includes: S3.1: Improve the YOLO11n model to enhance its accuracy and real-time performance to adapt to complex environmental changes.

[0038] (1) Optimize the original backbone network and introduce a fast hybrid visual transformer FastViT backbone network with a structure reparameterization mechanism to replace the original backbone network.

[0039] In the YOLO11n model, the original backbone network is responsible for extracting multi-scale features from the input image, providing basic semantic support for subsequent detection tasks. However, this backbone structure has certain limitations in capturing long-distance dependencies and complex contextual information, affecting the model's recognition accuracy and generalization ability in complex scenarios. To address this issue, this paper proposes an improvement scheme: introducing a fast hybrid visual transformer FastViT with a structure reparameterization mechanism as a new backbone network to replace the original YOLO11n model's basic backbone structure, constructing a new efficient detection network.

[0040] The improved YOLO11n model is as follows: The initialized input image is input to the FastViT backbone network module, the output of the FastViT backbone network module is connected to the input of the SimSPPF module, the output of the SimSPPF module is connected to the input of the C2PSA module, the output of the C2PSA module is connected to the input of the first Upsample module and the input of the fourth C3k2 module, the output of the first Upsample module is connected to the input of the first Concat module, the output of the first Concat module is connected to the input of the first C3k2 module, the output of the first C3k2 module is connected to the input of the first SCSA module, the output of the first SCSA module is connected to the input of the second Upsample module, the output of the second Upsample module is connected to the input of the second Concat module, the output of the second Concat module is connected to the input of the second C3k2 module, the output of the second C3k2 module is connected to the input of the second SCSA module, the output of the second SCSA module is connected to the input of the first 11Detect detection head module and the input of the first Conv module, the output of the first Conv module is connected to the input of the third Concat module, the output of the third Concat module is connected to the input of the third C3k2 module, the output of the third C3k2 module is connected to the input of the third SCSA module, the output of the third SCSA module is connected to the input of the second 11Detect detection head module and the input of the second Conv module, the output of the second Conv module is connected to the input of the fourth Concat module, the output of the fourth Concat module is connected to the input of the fourth C3k2 module, and the output of the fourth C3k2 module is connected to the input of the third 11Detect detection head module; The SimSPPF feature pyramid structure is specifically: The initialized input feature of the SimSPPF feature pyramid module is input to the first ConvBNReLU module, the output of the first ConvBNReLU module is connected to the input of the first Maxpool2d module, the output of the first Maxpool2d module is connected to the input of the splicing module and the input of the second Maxpool2d module, the output of the second Maxpool2d module is connected to the input of the splicing module and the input of the third Maxpool2d module, the output of the third Maxpool2d module is connected to the input of the splicing module, the output of the splicing module is connected to the input of the second ConvBNReLU module, and the output of the second ConvBNReLU module is the refined feature.

[0041] The SCSA spatial cross-scale attention mechanism module is specifically: The initialization input of the SCSA module is input to the input of the X Avgpool module, the input of the Y Avgpool module, and the input of the first matrix point multiplication module. The output of the X Avgpool module is connected to the input of the first Spilt module. The output of the first Spilt module is connected to the input of the DWConv1d module with a convolution kernel of 3x3, the input of the DWConv1d module with a convolution kernel of 5x5, the input of the DWConv1d module with a convolution kernel of 7x7, and the input of the DWConv1d module with a convolution kernel of 9x9. The output of the DWConv1d module with a convolution kernel of 3x3, the output of the DWConv1d module with a convolution kernel of 5x5, the output of the DWConv1d module with a convolution kernel of 7x7, and the output of the DWConv1d module with a convolution kernel of 9x9 are connected to the input of the first Concat module under the corresponding input branch of the X Avgpool module. The output of the first Concat module is connected to the input of the first GroupNorm module. The output of the first GroupNorm module is connected to the input of the first Sigmoid module. The output of the first Sigmoid module is connected to the input of the first matrix point multiplication module. The output of the Y Avgpool module is connected to the input of the second Spilt module. The output of the second Spilt module is connected to the input of the DWConv1d module with a convolution kernel of 3x3, the input of the DWConv1d module with a convolution kernel of 5x5, the input of the DWConv1d module with a convolution kernel of 7x7, and the input of the DWConv1d module with a convolution kernel of 9x9. The output of the DWConv1d module with a convolution kernel of 3x3, the output of the DWConv1d module with a convolution kernel of 5x5, the output of the DWConv1d module with a convolution kernel of 7x7, and the output of the DWConv1d module with a convolution kernel of 9x9 are connected to the input of the second Concat module under the corresponding input branch of the Y Avgpool module. The output of the second Concat module is connected to the input of the second GroupNorm module. The output of the second GroupNorm module is connected to the input of the second Sigmoid module. The output of the second Sigmoid module is connected to the input of the first matrix point multiplication module. This process is the whole process of the SMSA module. Then, the output of the first matrix point multiplication module is connected to the input of the first Avgpool, the output of the first Avgpool is connected to the input of the third GroupNorm module, the output of the third GroupNorm module is connected to the input of the DWConv2d module with the first convolution kernel of 1x1, the input of the DWConv2d module with the second convolution kernel of 1x1, the input of the DWConv2d module with the third convolution kernel of 1x1, the output of the DWConv2d module with the first convolution kernel of 1x1, the output of the DWConv2d module with the second convolution kernel of 1x1 is connected to the input of the first matrix multiplication module, the output of the first matrix multiplication module is connected to the input of the first Softmax module, the output of the first Softmax module and the output of the third DWConv2d module with the connection kernel of 1x1 are connected to the input of the second Avgpool module, the output of the second Avgpool module is connected to the input of the third Sigmoid module, the output of the third Sigmoid module and the output of the first matrix point multiplication module are connected to the input of the second matrix point multiplication module; this process is the whole process of the PSCA module. The output of the second matrix point multiplication module is the refined feature.

[0042] The Fast Hybrid Vision Transformer using Structural Reparameterization (FastViT) is specifically: The initialization input feature of the FastViT module is input to the input of the Stem module, the output of the Stem module is connected to the input of the Stage1 module, the output of the Stage1 module is connected to the input of the Patch Emb module with the first step of 2, the output of the Patch Emb module with the first step of 2 is connected to the input of the Stage2 module, the output of the Stage2 module is connected to the input of the Patch Emb module with the second step of 2, the output of the Patch Emb module with the second step of 2 is connected to the input of the Stage3 module, the output of the Stage3 module is connected to the input of the Patch Emb module with the third step of 2, the output of the Patch Emb module with the third step of 2 is connected to the input of the Stage4 module, the output of the Stage4 module is connected to the input of the DW MobileOne Block module with the convolution kernel of 3x3, the output of the DW MobileOne Block module with the convolution kernel of 3x3 is connected to the input of the Avg Pool module, the output of the Avg Pool module is connected to the input of the FC module, and the output of the FC module is the refined feature.

[0043] The structure of the Patch Emb module with a step of 2 is specifically as follows: the initial input feature of the Patch Emb module with a step of 2 is input to an input of a DW Conv module with a first convolution kernel of 7*7 and a step of 2, and an input of a DW Conv module with a second convolution kernel of 7*7 and a step of 2; an output of the DW Conv module with the first convolution kernel of 7*7 and the step of 2 is connected to an input of a first BatchNorm module, an output of the first BatchNorm module is connected to an input of a first matrix element-wise addition module, an output of the DW Conv module with the second convolution kernel of 7*7 and the step of 2 is connected to an input of a second BatchNorm module, and an output of the second BatchNorm module is connected to an input of the first matrix element-wise addition module; an output of the first matrix element-wise addition module is connected to an input of a first Attention module, an output of the first Attention module is connected to an input of a Conv module with a convolution kernel of 1*1 and an input of a fourth BatchNorm module, an output of the Conv module with the convolution kernel of 1*1 is connected to an input of a third BatchNorm module, an output of the third BatchNorm module and an output of the fourth BatchNorm module are connected to an input of a second matrix element-wise addition module, and an output of the second matrix element-wise addition module is connected to an input of a second Attention module. An output of the second Attention module is the refined feature.

[0044] The structure of the Stage1, Stage2 and Stage3 modules is specifically as follows: the initial input feature of the Stage1, Stage2 and Stage3 is input to an input of a BatchNorm module and an input of a matrix element-wise addition module, an output of the BatchNorm module is connected to an input of a DW Conv module with a convolution kernel of 3*3, an output of the DW Conv module with the convolution kernel of 3*3 is connected to an input of the matrix element-wise addition module, an output of the matrix element-wise addition module is connected to an input of a ConvFFN module, and an output of the ConvFFN module is the refined feature. The structure of the Stage4 module is specifically as follows: the initial input feature of the Stage4 is input to an input of a CPE module and an input of a first matrix element-wise addition module, an output of the CPE module is connected to an input of the first matrix element-wise addition module, an output of the first matrix element-wise addition module is connected to an input of a BatchNorm module, an output of the BatchNorm module is connected to an input of an Attention module, an output of the Attention module is connected to an input of a second matrix element-wise addition module, an output of the second matrix element-wise addition module is connected to an input of a ConvFFN module, and an output of the ConvFFN module is the refined feature.

[0045] FastViT backbone network combines the efficiency of convolutional neural networks with the global modeling capabilities of Transformer structure. It introduces a multi-branch structure during training through structural reparameterization to enhance expression capabilities, and reparameterizes to a single-branch structure during inference to effectively reduce computational overhead and speed up inference.

[0046] The design of FastViT not only improves the ability to capture photovoltaic panel defect features, but also enhances the recognition effect of small targets, occluded areas, and complex textures.

[0047] In specific implementation, the original backbone network of YOLO11n is completely replaced by the depth separable hybrid transformation module of FastViT to build a new backbone structure. The modular design of FastViT allows more detailed modeling of local details and global semantics in images, improving the contextual understanding ability of target detection.

[0048] The advantages of this structure replacement mainly lie in the following aspects: 1) Improve target detection accuracy: FastViT uses multi-scale attention mechanism and global context information integration method to significantly improve the detection accuracy of photovoltaic panel defects such as fine cracks, surface scratches, and hot spots; 2) Optimize computational efficiency: Through structural reparameterization technology, maintain strong expression ability during training, and run with lighter structure during inference, effectively improve the actual deployment performance of the model; 3) Enhance model robustness and generalization ability: The introduction of global context perception ability makes the model adapt to different shooting angles, lighting conditions, and complex situations such as dirt occlusion; 4) Simplify training difficulty: More stable gradient flow and efficient attention path design help to alleviate the gradient vanishing problem and accelerate model convergence.

[0049] In summary, based on the YOLO11n model, the introduction of FastViT as a new backbone network improvement scheme not only improves the feature expression ability of the model, but also optimizes the balance between detection performance and computational resources, providing a more practical solution for precise identification of photovoltaic panel surface defects.

[0050] S3.2: Introduce spatial cross-scale attention mechanism (SCSA, Spatial Cross-Scale Attention) in the neck network to improve the fusion effect of multi-scale features.

[0051] In the neck network of the improved model, a spatial cross-scale attention mechanism (SCSA) is introduced to enhance the fusion of multi-scale features. This mechanism effectively captures the dependencies between features of different resolutions through cross-scale spatial information interaction, improving the model's ability to perceive and locate defects on photovoltaic panels.

[0052] Specifically, the SCSA mechanism enables dynamic weight distribution of multi-scale features at the spatial level in the neck network, enhancing the expression of key information while suppressing redundant and irrelevant background features. By modeling the spatial correlation between different scale feature maps, SCSA significantly improves the model's ability to recognize small defects and complex backgrounds.

[0053] After introducing SCSA, the model has stronger context understanding ability when fusing multi-level features, greatly improving the accuracy and robustness of detection, especially in real-world application environments such as varying lighting conditions and complex surface textures.

[0054] In summary, the spatial cross-scale attention mechanism, as an important improvement to the neck network, not only enhances the interaction and fusion of multi-scale information, but also provides strong support for improving the overall performance of photovoltaic panel defect detection models.

[0055] S3.3: Replace the original feature pyramid module with a lightweight and efficient SimSPPF structure.

[0056] SimSPPF (Simplified Spatial Pyramid Pooling Fast) is an improvement and optimization of the traditional SPPF (Spatial Pyramid Pooling Fast) module, aiming to improve the efficiency and effectiveness of feature fusion while reducing computational complexity.

[0057] SimSPPF simplifies the original multi-level pooling operation by optimizing the spatial pyramid pooling structure. It uses a lighter combination of pooling layers to reduce redundant and repetitive calculations, and through a reasonable connection method, it effectively integrates multi-scale information. Specifically, SimSPPF retains the advantages of different scale pooling in capturing spatial context information, but is more compact in structure, avoiding excessive channel expansion and computational bottlenecks. In addition, SimSPPF usually uses fast convolution or lightweight convolution strategies to further improve processing speed.

[0058] The SimSPPF replaces the original SPPF module in YOLO11n, bringing significant advantages in multiple aspects. First, SimSPPF improves the efficiency of multi-scale feature fusion through a simplified series and parallel structure, reducing redundant calculations and operations, while maintaining or even enhancing feature expression capabilities while reducing computational complexity. In addition, SimSPPF improves the feature pooling and merging strategy, more reasonably integrating feature information from different receptive fields, enabling the model to better capture fine-grained and global information in photovoltaic panel defects, improving the sensitivity to details. At the same time, the optimization and simplification of the structure effectively reduces the number of model parameters and floating-point operations, thereby speeding up the inference and meeting the real-time detection requirements. More efficient feature fusion also enhances the model's robustness in complex backgrounds and diverse defect environments, reducing the probability of false positives and false negatives, and improving detection accuracy. Especially when combined with the FastViT backbone network using a structural reparameterization mechanism, SimSPPF can better accept and integrate the multi-scale features output by the backbone, exerting a synergistic optimization effect and further improving overall performance. In summary, as an optimization and upgrade of the SPPF in YOLO11n, SimSPPF not only improves the quality and efficiency of feature fusion, but also considers model lightweight and real-time, making it very suitable for photovoltaic panel defect applications that require accurate and rapid identification.

[0059] Specifically, step S4 includes: S4.1: Verify the accuracy and photovoltaic panel defect detection ability of the model Ensure that the model can accurately detect various defects on the photovoltaic panel, regardless of defect size, shape, or color difference. It is particularly important to consider the subtle features of defects such as cracks, stains, scratches, etc. The model should be able to identify early or hidden defect signs. Even if the defect area is small or just emerging, the model should be able to timely and accurately capture and mark the defect location, achieving early warning and maintenance. Photovoltaic panel defects are diverse and may be complex and varied in the background environment. It is also important to verify whether the model can distinguish between different defect types and their severity, which helps to accurately locate and classify defects and guide subsequent maintenance decisions.

[0060] S4.2: Real-time target detection and response capability In key new energy facilities such as photovoltaic power stations, the development of defects can quickly affect equipment performance and safety. In order to effectively ensure the stable operation of photovoltaic panels, the model needs to have fast response capability, which can detect and locate defects in real time and ensure timely warning before defects expand or worsen. This requires the model not only to have high detection accuracy, but also to complete detection and feedback in a very short time to ensure that maintenance personnel can take prompt action.

[0061] In addition, the model also needs to have defect tracking capability, that is, after the occurrence of defects, the location, size and change trend can be continuously monitored, and the maintenance strategy is dynamically adjusted according to the monitoring result. This is of great significance to prevent defects from expanding, reduce losses and prolong the service life of photovoltaic panels. The model verifies the detection speed of the verification model to determine whether the model is real-time during verification.

[0062] Through the ablation experiment as shown in Table 1, it can be known that, by simultaneously adding the FastViT backbone network, the SCSA attention mechanism module and the SimSPPF pyramid module, compared with the baseline model and other improved methods, the model of the application is far higher than other detection models in accuracy and speed, and can meet the requirements of real-time classification detection.

[0063] The following is a comparison result analysis table with the original model Table 1 Comparison test result table

[0064] The platform used for testing of the application is Pycharm 11.0.15, the model framework is Pytorch 1.13.1, the CPU is 15vCPU Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60GHz, and the GPU is RTX3090, 24GB.

[0065] When setting the hyperparameters, the number of training rounds is set to 200 rounds, and the batchsize is set to 32.

[0066] As can be seen from Table 1, after introducing the FastViT backbone network, the SCSA attention mechanism and the SimSPPF feature enhancement module on the basis of the benchmark YOLO11 model, the model has significant improvement in various performance indicators. Specifically, the improved model is superior to the original model in terms of detection accuracy, recall rate, mAP50 value and inference speed, and has stronger comprehensive detection capability, fully illustrating that the proposed structural improvement is effective and feasible in the research task. In order to further verify the independent contribution of each module to the performance improvement of the model, we designed and carried out a systematic ablation experiment. The experimental results show that the introduction of the FastViT, SCSA and SimSPPF three modules can effectively improve the mAP50 value of the model alone, which shows that each improvement measure enhances the extraction and expression ability of the model to the target features to different degrees, thereby significantly improving the overall detection performance.

[0067] The backbone network as the core component of deep learning model, its structure design largely determines the performance of the model in feature extraction, expression ability and overall performance. Therefore, in order to verify the superiority of the selected backbone network FastViT in the target detection task, it is necessary to compare it with other mainstream backbone network structures. For this purpose, several widely used backbone networks (such as MobileNetV3, ShuffleNetV2, FastNet) are used to replace the original model FastViT, and training and testing are carried out under the unified data set and experimental setting. Table 2 summarizes the key performance indicators of the models corresponding to each alternative backbone network, including mAP, parameter quantity, inference speed, etc., to comprehensively evaluate the performance advantage of FastViT in this task.

[0068] Table 2 backbone network replacement experiment results table

[0069] From table 2, in the comparison experiment, the model using FastViT as the backbone network performs well in multiple key performance indicators. Specifically, it achieves the best results in precision, recall, mean average precision (mAP) and inference speed (FPS), which are significantly better than other mainstream backbone network structures. This shows that FastViT has achieved a good balance between feature extraction efficiency and representation ability, and can more effectively adapt to the complex and variable scenes and target characteristics in the data set. Therefore, considering the detection performance and running efficiency, selecting FastViT as the backbone network structure of the model in this study is the relatively optimal choice under the current conditions.

[0070] In order to verify the superiority of the SimSPPF spatial feature pyramid structure, the present application carries out comparison experiments on various spatial pyramid structures such as SPPF, SPP, SPPCSPC and SimSPPF, and the experimental results are shown in table 3.

[0071] Table 3 feature pyramid network comparison experiment results table

[0072] From table 3, when the feature pyramid network is SimSPPF, the model performance is the best. Therefore, it can be concluded that the SimSPPF feature pyramid structure is the best in this model, which is better than other feature pyramid structure models.

[0073] In summary, the present application is based on improving the YOLO11n model, which realizes efficient and accurate identification of photovoltaic panel defects. Compared with traditional detection methods, this method shows higher detection accuracy and real-time performance in complex environments, enhancing the reliability of the model. By introducing the FastViT backbone network, SCSA attention mechanism and SimSPPF feature pyramid module, the model enhances the attention ability to key features, reduces the inaccuracy of model detection, and optimizes the regression accuracy of target bounding box. In addition, the present application can effectively distinguish different types of photovoltaic panel defects, and can accurately detect even in the case of multiple targets, improving the intelligent level of photovoltaic panel defect monitoring. This method not only improves the efficiency of defect detection, but also provides important technical support for the safe operation of photovoltaic systems, with significant engineering application value and social and economic benefits.

Claims

1. A method for detecting surface defects in photovoltaic panels based on a hybrid vision converter architecture, characterized in that, Includes the following steps: Step 1: Obtain a dataset of photovoltaic panel images from a photovoltaic power station and construct a dataset of photovoltaic panel surface defects; Step 2: Label the target dataset, process the labels of each collected image, construct a dataset containing photovoltaic panel surface defects, and divide the dataset into training set, validation set and test set; Step 3: Obtain the improved YOLO11 model and use the model to identify the collected images of defects on the surface of photovoltaic panels; Step 4: Validate the model, including testing its robustness and detection accuracy under a single target; To ensure that the model can effectively identify surface defects of photovoltaic panels in both simple and complex scenarios, thus guaranteeing accurate detection capabilities.

2. The method according to claim 1, characterized in that, Step 1 specifically includes the following steps: Step S1.1: Collect extensive data on typical photovoltaic panel surface defect cases; Step S1.2: Carefully select and organize relevant case images; Step S1.3: Preprocess the dataset to reduce the inaccuracy and randomness of some dataset images.

3. The method according to claim 2, characterized in that, In step S1.1, the cases include photovoltaic panel defects of different scales, types, and application environments; In step S1.2, the image should be able to clearly show the specific characteristics of the defects on the photovoltaic panel surface, including three typical defect forms: photovoltaic panel cracks, photovoltaic panel bird droppings, and photovoltaic panel dirt accumulation. In step S1.3, the data preprocessing methods include histogram equalization, Gaussian filtering, and Gamma correction.

4. The method according to claim 1, characterized in that, Step S2 includes the following steps: S2.1: Annotation was performed in the Anaconda environment. The labelimg image annotation tool was used to annotate the collected images of defects on the surface of photovoltaic panels. The objects were identified into four categories: photovoltaic panels, bird droppings, dust, and cracks. Photovoltaic panels were labeled "panel", bird droppings were labeled "bird-drop", dust was labeled "dusty", and cracks were labeled "cracked". PascalVOC format data labels were obtained, and then the data format conversion code written in Python was used to convert them into a txt file suitable for training. The file contained most of the data, including the x-coordinate, y-coordinate, width, and height of the bounding box center point. S2.2: After completing the annotation work, the dataset is divided to obtain the training set, validation set, and test set of the photovoltaic panel defect dataset.

5. The method according to claim 1, characterized in that, The improved YOLO11 model is specifically as follows: The initial input image is fed into the FastViT backbone network module. The output of the FastViT backbone network module is connected to the input of the SimSPPF module. The output of the SimSPPF module is connected to the input of the C2PSA module. The output of the C2PSA module is connected to the input of the first Upsample module and the fourth C3k2 module. The output of the first Upsample module is connected to the input of the first Concat module. The output of the first Concat module is connected to the input of the first C3k2 module. The output of the first C3k2 module is connected to the input of the first SCSA module. The output of the first SCSA module is connected to the input of the second Upsample module. The output of the second Upsample module is connected to the input of the second Concat module. The output of the second Concat module is connected to the second C3k2 module. The input of the module, the output of the second C3k2 module is connected to the input of the second SCSA module, the output of the second SCSA module is connected to the input of the first 11Detect detection head module and the input of the first Conv module, the output of the first Conv module is connected to the input of the third Concat module, the output of the third Concat module is connected to the input of the third C3k2 module, the output of the third C3k2 module is connected to the input of the third SCSA module, the output of the third SCSA module is connected to the input of the second 11Detect detection head module and the input of the second Conv module, the output of the second Conv module is connected to the input of the fourth Concat module, the output of the fourth Concat module is connected to the input of the fourth C3k2 module, and the output of the fourth C3k2 module is connected to the input of the third 11Detect detection head module.

6. The method according to claim 5, characterized in that, The SimSPPF feature pyramid structure is as follows: The initial input features of the SimSPPF feature pyramid module are fed into the first ConvBNReLU module. The output of the first ConvBNReLU module is connected to the input of the first Maxpool2d module. The output of the first Maxpool2d module is connected to the input of the concatenation module and the second Maxpool2d module. The output of the second Maxpool2d module is connected to the input of the concatenation module and the third Maxpool2d module. The output of the third Maxpool2d module is connected to the input of the concatenation module. The output of the concatenation module is connected to the input of the second ConvBNReLU module. The output of the second ConvBNReLU module is the refined feature.

7. The method according to claim 5, characterized in that, The SCSA spatial cross-scale attention mechanism module is specifically as follows: The initialization input features of the SCSA module are fed into the inputs of the X Avgpool module, the Y Avgpool module, and the first matrix dot product module. The output of the X Avgpool module is connected to the input of the first Split module. The output of the first Split module is connected to the inputs of the following DWConv1d modules: one with a 3×3 kernel, one with a 5×5 kernel, one with a 7×7 kernel, and one with a 9×9 kernel. The outputs of the following DWConv1d modules are connected to the following DWConv1d modules: one with a 3×3 kernel, one with a 5×5 kernel, one with a 7×7 kernel, and one with a 9×9 kernel. The input branch of the Avgpool module connects to the input of the first Concat module. The output of the first Concat module connects to the input of the first GroupNorm module. The output of the first GroupNorm module connects to the input of the first Sigmoid module. The output of the first Sigmoid module connects to the input of the first matrix dot product module. The output of the Avgpool module connects to the input of the second Split module. The output of the second Split module connects to the inputs of the DWConv1d modules with 3×3, 5×5, 7×7, and 9×9 kernels. The outputs of the DWConv1d modules with 3×3, 5×5, 7×7, and 9×9 kernels are also connected to the input of the Y module. The input branch of the Avgpool module is connected to the input of the second Concat module. The output of the second Concat module is connected to the input of the second GroupNorm module. The output of the second GroupNorm module is connected to the input of the second Sigmoid module. The output of the second Sigmoid module is connected to the input of the first matrix dot product module. This process is the entire flow of the SMSA module. Then, the output of the first matrix dot product module is connected to the input of the first Avgpool module, the output of the first Avgpool module is connected to the input of the third GroupNorm module, the output of the third GroupNorm module is connected to the inputs of the first DWConv2d module with a 1×1 convolution kernel, the second DWConv2d module with a 1×1 convolution kernel, and the third DWConv2d module with a 1×1 convolution kernel, the outputs of the first DWConv2d module with a 1×1 convolution kernel and the second DWConv2d module with a 1×1 convolution kernel are connected to the input of the first matrix multiplication module, the output of the first matrix multiplication module is connected to the input of the first Softmax module, the output of the first Softmax module and the output of the third DWConv2d module with a 1×1 concatenation kernel are connected to the input of the second Avgpool module, the output of the second Avgpool module is connected to the input of the third Sigmoid module, and the output of the third Sigmoid module and the output of the first matrix dot product module are connected to the input of the second matrix dot product module; this process is the complete flow of the PSCA module; The output of the second matrix dot product module is the refined feature.

8. The method according to claim 5, characterized in that, The FastViT, a fast hybrid vision transformer with reparameterized structure, is as follows: The initial input features of the FastViT module are fed into the input of the Stem module. The output of the Stem module is connected to the input of the Stage1 module. The output of the Stage1 module is connected to the input of the first Patch Emb module with a length of 2. The output of the first Patch Emb module with a length of 2 is connected to the input of the Stage2 module. The output of the Stage2 module is connected to the input of the second Patch Emb module with a length of 2. The output of the second Patch Emb module with a length of 2 is connected to the input of the Stage3 module. The output of the Stage3 module is connected to the input of the third Patch Emb module with a length of 2. The output of the third Patch Emb module with a length of 2 is connected to the input of the Stage4 module. The output of the Stage4 module is connected to the input of the DW MobileOneBlock module with a 3×3 convolution kernel. The output of the DW MobileOne Block module with a 3×3 convolution kernel is connected to the input of the Avg Pool module. The output of the Avg Pool module is connected to the input of the FC module. The output of the FC module is the refined feature.

9. The method according to claim 8, characterized in that, in, The structure of the Patch Emb module with a stride of 2 is as follows: the initial input features of the Patch Emb module with a stride of 2 are fed into the input of the first DWConv module with a 7×7 kernel and a stride of 2, and into the input of the second DW Conv module with a 7×7 kernel and a stride of 2; the output of the first DW Conv module with a 7×7 kernel and a stride of 2 is connected to the input of the first BatchNorm module; the output of the first BatchNorm module is connected to the input of the first matrix element-wise addition module; and the second DW Conv module with a 7×7 kernel and a stride of 2... The output of the Conv module is connected to the input of the second BatchNorm module, and the output of the second BatchNorm module is connected to the input of the first matrix element-wise addition module. The output of the first matrix element-wise addition module is connected to the input of the first Attention module. The output of the first Attention module is connected to the input of the Conv module with a 1×1 kernel and the input of the fourth BatchNorm module. The output of the Conv module with a 1×1 kernel is connected to the input of the third BatchNorm module. The outputs of the third BatchNorm module and the fourth BatchNorm module are connected to the input of the second matrix element-wise addition module. The output of the second matrix element-wise addition module is connected to the input of the second Attention module. The output of the second Attention module is the refined feature.

10. The method according to claim 8, characterized in that, The specific structure of Stage1, Stage2, and Stage3 modules is as follows: the initial input features of Stage1, Stage2, and Stage3 are input to the input of the BatchNorm module and the matrix element-wise addition module. The output of the BatchNorm module is connected to the input of the DW Conv module with a 3×3 convolution kernel. The output of the DW Conv module with a 3×3 convolution kernel is connected to the input of the matrix element-wise addition module. The output of the matrix element-wise addition module is connected to the input of the ConvFFN module. The output of the ConvFFN module is the refined feature. The Stage4 module has the following structure: the initial input features of Stage4 are fed into the input of the CPE module and the input of the first matrix element-wise addition module. The output of the CPE module is connected to the input of the first matrix element-wise addition module. The output of the first matrix element-wise addition module is connected to the input of the BatchNorm module. The output of the BatchNorm module is connected to the input of the Attention module. The output of the Attention module is connected to the input of the second matrix element-wise addition module. The output of the second matrix element-wise addition module is connected to the input of the ConvFFN module. The output of the ConvFFN module is the refined features.

Images