Two-stage photovoltaic panel defect detection method and system based on multi-spectral state space model

Abstract

The application provides a kind of based on multi-spectrum state space model's two-stage photovoltaic panel defect detection method and system, comprising: high-resolution RGB aerial photovoltaic panel image is input based on variable aggregation convolution light weight photovoltaic panel positioning detection network, and the position of photovoltaic panel is output;RGB aerial photovoltaic panel image is used as reference image, and multi-spectrum state aerial photovoltaic panel image is used as to be calibrated image, input time sequence dynamic image registration module, utilize the accurate registration of single frame RGB image and multi-spectrum image based on the deep learning registration method of jump network, and obtain multi-spectrum calibration image by the time sequence dynamic accumulation operation of multiple images;According to the position of photovoltaic panel, multi-spectrum calibration image is cut, and multi-spectrum photovoltaic panel defect image is obtained and input photovoltaic panel defect detection network, utilize multi-spectrum space model to linear modeling on image spectrum and spatial dimension, and output the defect class and defect detection frame of to-be-tested photovoltaic panel.The application can detect photovoltaic panel defect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of photovoltaic panel defect detection, in particular to a two-stage photovoltaic panel defect detection method and system based on a multi-spectral state space model. BACKGROUND

[0002] As a clean and renewable energy solution, solar photovoltaic technology has received extensive attention and application. With the large-scale deployment of photovoltaic power generation systems, the problem of routine maintenance of photovoltaic panels has become increasingly prominent. In actual operation, photovoltaic panels often expose various potential faults and defects such as hot spots and fragments, which can cause energy loss, system efficiency decline, and even system failure in extreme cases. Therefore, developing accurate and efficient photovoltaic panel defect detection technology is crucial for ensuring the reliability and stability of photovoltaic systems.

[0003] Currently, photovoltaic panel defect detection technology mainly falls into two categories. The first category is the detection method based on electrical characteristics, which mainly collects electrical operation data through various sensors and uses mathematical models or machine learning to analyze whether there are defects. However, the large number of sensors leads to high detection costs of photovoltaic panels. The second category is the detection method based on aerial images, which mainly relies on machine learning or deep learning technology and has gradually become the mainstream solution for photovoltaic panel defect detection. Algorithms based on traditional machine learning, such as support vector machines and random forest algorithms, usually rely on manually designed extractors and require different and complex recognition relationships to be constructed manually in different scenarios, resulting in poor generalization ability and robustness. With the rapid development of deep learning, detection methods based on deep learning, such as Faster R-CNN, YOLO, and DETR models, have significantly improved feature representation capabilities. However, these methods usually directly recognize collected images, but aerial images have large scales, high resolutions, and a large number of irrelevant background pixels, and processing high-resolution images is limited by the structure size of the model, which severely affects the detection ability of the model. In addition, defects can cause uneven heating of photovoltaic panels, resulting in significant differences in spectral characteristics in the infrared, near-infrared, and far-infrared bands compared to normal photovoltaic panels. Therefore, using multi-spectral images containing infrared, near-infrared, far-infrared, and RGB bands for photovoltaic panel defect detection has great advantages. However, existing algorithms fail to fully exploit and utilize the spectral information in multi-spectral images under resource constraints, resulting in limited accuracy of photovoltaic panel defect detection. SUMMARY

[0004] To solve the technical problems existing in the prior art, the present application provides a two-stage photovoltaic panel defect detection method based on a multi-spectral state space model, and the technical solution is as follows:

[0005] In one aspect, a two-stage photovoltaic panel defect detection method based on a multi-spectral state space model is provided, the method comprising:

[0006] S1, obtaining a high-resolution RGB photovoltaic panel aerial image to be detected and a multi-spectral aerial photovoltaic panel image;

[0007] S2, inputting the RGB aerial photovoltaic panel image into a first-stage lightweight photovoltaic panel positioning detection network based on a variable aggregation convolution, the photovoltaic panel positioning detection network performing positioning detection on the position of the photovoltaic panel in the RGB aerial photovoltaic panel image, and outputting the position of the photovoltaic panel;

[0008] S3, inputting the RGB aerial photovoltaic panel image as a reference image and the multi-spectral aerial photovoltaic panel image as a to-be-calibrated image into a time-series dynamic image registration module, the time-series dynamic image registration module performing accurate registration of a single-frame RGB image and a multi-spectral image by using a deep learning registration method based on a skip network, and obtaining a multi-spectral calibration image through time-series dynamic accumulation operation of multiple images;

[0009] S4, cutting the multi-spectral calibration image according to the position of the photovoltaic panel output by the photovoltaic panel positioning detection network, to obtain a multi-spectral photovoltaic panel defect image of a corresponding region;

[0010] S5, inputting the multi-spectral photovoltaic panel defect image into a second-stage photovoltaic panel defect detection network, the photovoltaic panel defect detection network performing linear modeling on image spectrum and spatial dimension by using a multi-spectral state space model, effectively adapting to the characteristics of spectral order change, enhancing the representation ability of defect features, and outputting a defect category and a defect detection frame of the photovoltaic panel to be detected, to complete photovoltaic panel defect detection.

[0011] Optionally, the photovoltaic panel positioning detection network comprises a backbone network, a neck network, and a detection output end.

[0012] The backbone network is divided into four stages, and each stage is connected in turn, wherein each stage network of the first three stages is composed of a convolution module and a cross-stage feature extraction module in series, and an SPPF module is added to the last stage on this basis; the convolution module comprises a convolution layer, a batch normalization layer, and a Leaky ReLU activation function, wherein the Leaky ReLU activation function is an improved version of ReLU, and the formula is as follows: In the formula, a is a small constant, when the input is a negative number, the output still remains negative, but will not be a complete zero, and provides a non-zero gradient a for the negative input, which helps the gradient to propagate more smoothly in the entire network, effectively solving the problem of dead neurons that may occur in the training process of the standard ReLU; the cross-stage feature extraction module, by averaging the channels of the input feature map into two parts, respectively inputting two branches, one branch passing through the variable aggregation convolution, and the other branch passing through the variable aggregation convolution and the bottleneck module, the outputs of the two branches are spliced and then passed through the variable aggregation convolution to obtain the final result, the bottleneck module is composed of two convolution modules and adopts residual connection to add the input and the output, the SPPF module includes three maximum pooling layers connected in series, and the output of each maximum pooling layer is spliced with the original input and then passed through a convolution module to obtain the final output; the last three stages of the backbone network output small, medium and large scale feature maps respectively, and there is a two times down-sampling relationship between the sizes, and these feature maps will be transmitted as input to the neck network;

[0013] The neck network inputs small, medium and large scale feature maps from the backbone network, and processes them through the cross-stage feature extraction module, convolution and up-sampling operation: first, the small scale feature map is restored to the same spatial resolution as the medium scale feature map through up-sampling, and is spliced with the medium scale feature map and input to the cross-stage feature extraction module L1, the output of L1 is restored to the same resolution as the large scale feature map through up-sampling, and is spliced with the large scale feature map and input to the cross-stage feature extraction module L2 to generate large scale fusion features, the output of L2 is down-sampled to medium scale resolution through convolution, and is spliced with the output of L1 and input to the cross-stage feature extraction module L3 to obtain medium scale fusion features, the output of L3 is restored to small scale resolution through convolution, and is spliced with the small scale feature map and input to the cross-stage feature extraction module L4 to obtain a small scale fusion feature map, through the top-down feature fusion and bottom-up feature enhancement path, the high-level semantic information and low-level detail features are effectively fused, and more accurate target detection is realized in different scales and complex scenes;

[0014] The detection output end splices the three scale fusion features from the neck network, restores them to the original image size through up-sampling, and outputs the position of the photovoltaic panel through convolution operation.

[0015] Optionally, the variable aggregation convolution, in which k parallel convolution kernels are embedded in a convolution layer, dynamically adjusts the weight of each convolution kernel through an attention mechanism composed of an average pooling layer, a full connection layer, a Leaky ReLU activation function, a full connection layer, and a softmax, to identify photovoltaic panels of different sizes and distinguish irrelevant backgrounds on a high-resolution image. Each convolution kernel is multiplied by its respective weight after convolution operation on the input image tensor, and the final convolution result is obtained by aggregation and addition. The result is output through batch normalization and a Leaky ReLU activation function.

[0016] Optionally, the time-series dynamic image registration module, in the single-frame image registration process, inputs the reference RGB image I F and the image to be calibrated I M into a skip network to extract respective feature maps, concatenates the obtained two feature maps, down-samples and flattens them through a convolution layer, inputs them into a full connection layer, and obtains geometric transformation parameters P{t x ,t y ,θ,s x ,s y ,β x ,β y}, wherein t x , t y are translation parameters, θ is a rotation angle, s x , s y are scaling factors, β x , β y are shear factors, and the image to be calibrated is geometrically transformed and adjusted through these parameters to achieve registration, to obtain a calibrated image IR.

[0017] The skip network takes an image as input, first extracts preliminary features through a convolution and a max pooling layer, then further extracts deep features through three double-branch residual structures and a cross-stage feature extraction module. The double-branch residual structure includes a single-layer convolution branch and a three-layer convolution branch. After fusion, the features are processed by a Leaky ReLU activation function, and then further extracted by two layers of cross-stage feature extraction modules to extract deep features of the image. Finally, the feature map is output.

[0018] Optionally, the time-series dynamic image registration module, in the time-series dynamic accumulation operation, inputs n frames of RGB images {I F1 , I F2 , I F3 ...I Fn} and corresponding n frames of multi-spectral images {I M1 , I M2 , I M3 ...I Mnimage registration, to obtain a registered image IR1, IR2, IR3...IR n , and then frame accumulation is performed on IR1 and IR2 to obtain IR 12 , and image registration is performed on IR 12 and I F2 to obtain IRR 12 ; frame accumulation is performed on IRR 12 and IR3 to obtain IRR 123 , and image registration is performed on IRR 123 and I F3 to obtain IRRR 123 ; through repeating the above steps, a final multispectral calibration image IRR..R 12..n is obtained, wherein the frame accumulation operation refers to adding and averaging each pixel value of two images.

[0019] Optionally, the photovoltaic panel defect detection network is composed of a photovoltaic panel defect detection backbone network and a multiscale fusion detection output end.

[0020] The photovoltaic panel defect detection backbone network first extracts photovoltaic panel defect features through a 3-dimensional convolution module, the 3-dimensional convolution module is stacked by 3-dimensional convolution, batch normalization and Leaky ReLU activation function in turn, and the input and the output are connected through a residual connection; next, the feature map is sent to a subsequent stage, the subsequent stage is divided into four stages, each stage is connected in turn, the first stage is composed of a state space feature extraction module; the second and third stages are composed of a down-sampling module and a state space feature extraction module in series; the fourth stage adds an SPPF module on the basis of the first two stages to enhance the extraction of key features and multiscale feature fusion, through the above four stages, feature maps of different scales are generated, including large, medium and small scales, the feature map size decreases by a two-fold down-sampling relationship, and the three scales of generated features will be input to the multiscale fusion detection output end.

[0021] The multiscale fusion detection output end inputs small, medium and large scale feature maps from the photovoltaic panel defect detection backbone network, and first performs multiscale fusion processing through a state space feature extraction module, convolution and up-sampling operation. The small scale feature map is restored to the same spatial resolution as the medium scale feature map through up-sampling, spliced with the medium scale feature map, input to the state space feature extraction module F1, and the output of F1 is restored to the same resolution as the large scale feature map through up-sampling, spliced with the large scale feature map, and input to the state space feature extraction module F2 to generate large scale fusion features. The output of F2 is down-sampled to a medium scale resolution through convolution, spliced with the output of F1, input to the state space feature extraction module F3 to obtain a medium scale fusion feature, and the output of F3 is restored to a small scale resolution through convolution, spliced with the small scale feature map, and input to the state space feature extraction module F4 to obtain a small scale fusion feature map. Then, the fusion feature maps of the three scales are spliced and restored to the original image size through up-sampling. Then, the feature map dimension is adjusted using 1x1 convolution, and finally the defect target class and position are output after two 3x3 convolution and one 1x1 convolution processing, and the photovoltaic panel defect detection is completed.

[0022] Optionally, the state space feature extraction module inputs the feature map first through convolution, batch normalization and LeakyReLU activation function for preliminary extraction processing. The preliminary processed features are input to a local space module. The local space module is sequentially stacked by depth separable convolution, batch normalization, convolution, Leaky ReLU activation function and convolution, and the output of the module is added to the input through a residual structure.

[0023] Then, the output of the local space module is input to a multispectral state space model after layer normalization processing, extracts the information combining the spectral dimension and the spatial dimension, and enhances the photovoltaic panel defect feature expression.

[0024] Finally, the obtained feature map is input to a linear layer for further processing, and residual connection is adopted to add and connect the output of the Leaky ReLU activation function in the preliminary processing to the output of the multispectral state space model and the linear layer respectively to obtain the output feature map.

[0025] Optionally, the multispectral state space model input is first subjected to image serialization processing, the feature map is cut into 9 spatial slices, and converted into an image sequence.

[0026] Then the image sequences are respectively input to the spectral dimension branch and the spatial dimension branch for processing. In the spatial dimension branch, a multi-spectrum spatial model linearly models 9 slices of the image sequence along the spatial dimension. Considering that the image sequence has context relationships and spatial dependencies in four directions, the image sequence is first scanned forward and backward by rows and columns respectively, and block matrix rearrangement is performed. The four groups of image sequences after rearrangement are input to the SSM module. Four feature sequences output by the SSM module are processed by summation and averaging, decoded back to an image tensor, and finally a spatial dimension feature is obtained after a convolution layer.

[0027] In the spectral dimension branch, a multi-spectrum spatial model linearly models c spectral channels of the image sequence. First, the spectral dimension sequence is adjusted, and block matrix rearrangement is performed by forward and backward scanning. The two groups of image sequences after rearrangement are input to the SSM module. Two image feature sequences output by the SSM module are also processed by summation and averaging, adjusted to the original dimension, decoded back to an image tensor, and finally a spectral dimension feature is obtained after a convolution layer.

[0028] Finally, the spatial dimension feature and the spectral dimension feature are fused to learn the joint features in space and spectrum, and finally a feature map is output.

[0029] Optionally, the SSM module is a discrete state space model, which is obtained by discretizing a continuous state space model. By discretizing the continuous-time state transition process, the original continuous state transition matrix and projection matrix are converted into a form suitable for discrete time, effectively modeling time series data.

[0030] On the other hand, a two-stage photovoltaic panel defect detection system based on a multi-spectrum spatial model is provided. The system comprises:

[0031] An acquisition module is configured to acquire a high-resolution RGB aerial photovoltaic panel image to be detected and a multi-spectrum aerial photovoltaic panel image.

[0032] A positioning detection module is configured to input the RGB aerial photovoltaic panel image into a first-stage lightweight photovoltaic panel positioning detection network based on a variable aggregation convolution. The photovoltaic panel positioning detection network performs positioning detection on the position of the photovoltaic panel in the RGB aerial photovoltaic panel image, and outputs the position of the photovoltaic panel.

[0033] A registration module is configured to input the RGB aerial photovoltaic panel image as a reference image and the multi-spectrum aerial photovoltaic panel image as a to-be-calibrated image into a time-series dynamic image registration module. The time-series dynamic image registration module uses a deep learning registration method based on a skip network to accurately register a single-frame RGB image and a multi-spectrum image, and obtains a multi-spectrum calibrated image through time-series dynamic accumulation operation of multiple images.

[0034] a cutting module configured to cut the multi-spectral calibration image according to the position of the photovoltaic panel output by the photovoltaic panel positioning detection network to obtain a multi-spectral photovoltaic panel defect image of a corresponding region;

[0035] a defect detection module configured to input the multi-spectral photovoltaic panel defect image into a second-stage photovoltaic panel defect detection network, and the photovoltaic panel defect detection network uses a multi-spectral spatial model to linearly model the spectral and spatial dimensions of the image, effectively adapt to the characteristics of the spectral order change, enhance the representation ability of the defect features, and output a defect category and a defect detection frame of the photovoltaic panel to be detected, thereby completing the photovoltaic panel defect detection.

[0036] In another aspect, an electronic device is provided, which includes a processor and a memory having at least one instruction stored therein, the at least one instruction being loaded and executed by the processor to implement the above-mentioned two-stage photovoltaic panel defect detection method based on a multi-spectral spatial model.

[0037] In another aspect, a computer-readable storage medium is provided, which has at least one instruction stored therein, the at least one instruction being loaded and executed by a processor to implement the above-mentioned two-stage photovoltaic panel defect detection method based on a multi-spectral spatial model.

[0038] The technical solutions provided by the present application have at least the following beneficial effects:

[0039] 1. The present application proposes a two-stage photovoltaic panel defect detection algorithm, which is different from the common deep learning algorithm directly performing single-stage defect detection on high-resolution aerial images. First, a first-stage lightweight photovoltaic panel positioning detection network based on variable aggregation convolution is used to extract the photovoltaic panel region in the RGB image, and the corresponding multi-spectral calibration image is cut according to the region, thereby minimizing irrelevant background interference.

[0040] 2. Compared with the traditional image registration method, the timing dynamic image registration module of the present application can realize accurate registration of single-frame RGB images and multi-spectral images through a registration network based on a skip network in a complex photovoltaic panel aerial scene, and effectively improve the signal-to-noise ratio by combining the timing dynamic accumulation operation of multiple images to improve the quality of the calibration image, thereby realizing high-precision alignment of RGB and multi-spectral high-resolution images in the aerial scene, and having strong robustness.

[0041] 3. The second-stage photovoltaic panel defect detection network of the present application combines a multi-spectral spatial model to linearly model the spatial and spectral dimensions, thereby enhancing the representation ability of the photovoltaic panel defect features and improving the detection accuracy. Meanwhile, the multi-spectral spatial model is designed based on the SSM architecture and has linear computational complexity, thereby ensuring the feasibility of the method in actual deployment. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0043] Figure 1 is a flow chart of a two-stage photovoltaic panel defect detection method based on a multi-spectrum state space model provided by an embodiment of the present application.

[0044] Figure 2 is a general block diagram of a two-stage photovoltaic panel defect detection method based on a multi-spectrum state space model provided by an embodiment of the present application.

[0045] Figure 3 is a photovoltaic panel positioning detection network structure block diagram provided by an embodiment of the present application.

[0046] Figure 4 is a variable aggregation convolution structure block diagram provided by an embodiment of the present application.

[0047] Figure 5 is a time sequence dynamic image registration module structure block diagram provided by an embodiment of the present application.

[0048] Figure 6 is a photovoltaic panel defect detection network structure block diagram provided by an embodiment of the present application.

[0049] Figure 7 is a state space feature extraction module structure block diagram provided by an embodiment of the present application.

[0050] Figure 8 is a multi-spectrum state space model structure block diagram provided by an embodiment of the present application.

[0051] Figure 9 is a two-stage photovoltaic panel defect detection system block diagram based on a multi-spectrum state space model provided by an embodiment of the present application.

[0052] Figure 10 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the technical problems, technical solutions and advantages of the present application more clear, the following will be described in detail with reference to the drawings and specific embodiments.

[0054] An embodiment of the present application provides a two-stage photovoltaic panel defect detection method based on a multi-spectrum state space model, which can be realized by an electronic device, which can be a terminal or a server.Figure 1 The flowchart of this method is shown below. Figure 2 The diagram shown is an overall block diagram of the method. The processing flow may include the following steps:

[0055] S1. Acquire high-resolution RGB aerial images and multispectral aerial images of the photovoltaic panels to be detected;

[0056] This invention utilizes a drone equipped with a high-resolution RGB camera and a multispectral camera to perform high-altitude imaging of a photovoltaic power station area, acquiring RGB and multispectral image data of the photovoltaic panels to be inspected. The RGB images are used for the first stage of photovoltaic panel location detection, with relatively low channel data dimensions, containing only three channels (red, green, and blue), requiring less computation and storage, thus ensuring a lightweight system. The multispectral images cover infrared, near-infrared, and far-infrared bands, which are more sensitive to uneven heat sources in defective parts of the photovoltaic panels, and are therefore used for the second stage of identifying potential defects in the photovoltaic panels.

[0057] S2. Input the RGB aerial image of the photovoltaic panel into the first-stage lightweight photovoltaic panel positioning and detection network based on variable aggregation convolution. The photovoltaic panel positioning and detection network performs positioning and detection on the position of the photovoltaic panel in the RGB aerial image of the photovoltaic panel and outputs the position of the photovoltaic panel.

[0058] Optionally, such as Figure 3 As shown, the photovoltaic panel positioning and detection network includes a backbone network, a neck network, and a detection output terminal;

[0059] The backbone network is divided into four stages, which are connected sequentially. The first three stages each consist of a convolutional module and a cross-stage feature extraction module connected in series. The last stage adds an SPPF module. Each convolutional module includes convolutional layers, batch normalization layers, and a Leaky ReLU activation function. The Leaky ReLU activation function is an improved version of ReLU (the ReLU activation function formula is as follows:). During training, the ReLU activation function outputs 0 for negative inputs, causing some neurons to remain inactive and have no impact on the network's final output. Furthermore, the derivative of ReLU with respect to negative inputs is always 0, meaning the gradient in the negative input region is zero during backpropagation, preventing the weights of these neurons from being updated. This phenomenon is known as the "dead neuron" phenomenon, directly affecting the model's learning ability. The formula is as follows: In the formula, a is a small constant (usually 0.01), when the input is negative, the output still remains negative, but will not be a complete zero, and provides a non-zero gradient a for the negative input, which helps the gradient to propagate more smoothly in the entire network, effectively solving the problem of dead neurons that may occur in the training process of the standard ReLU; the cross-stage feature extraction module, by splitting the channel average of the input feature map into two parts, respectively inputting two branches, one branch passing through the variable aggregation convolution, and the other branch passing through the variable aggregation convolution and the bottleneck module, the outputs of the two branches are spliced, and then the final result is obtained through the variable aggregation convolution, the bottleneck module is composed of two convolution modules, and the input and the output are added through residual connection, the SPPF module includes three maximum pooling layers connected in series, and the output of each maximum pooling layer is spliced with the original input, and then the final output is obtained through the convolution module; the last three stages of the backbone network output small, medium and large scale feature maps respectively, and there is a two times down-sampling relationship between the sizes, and the feature maps will be transmitted as input to the neck network;

[0060] The neck network inputs small, medium and large scale feature maps from the backbone network, and processes them through the cross-stage feature extraction module, convolution and up-sampling operation: first, the small scale feature map is restored to the same spatial resolution as the medium scale feature map through up-sampling, and is spliced with the medium scale feature map, and is input to the cross-stage feature extraction module L1, the output of L1 is restored to the same resolution as the large scale feature map through up-sampling, and is spliced with the large scale feature map, and is input to the cross-stage feature extraction module L2 to generate large scale fusion features, the output of L2 is down-sampled to medium scale resolution through convolution, and is spliced with the output of L1, and is input to the cross-stage feature extraction module L3 to obtain medium scale fusion features, the output of L3 is restored to small scale resolution through convolution, and is spliced with the small scale feature map, and is input to the cross-stage feature extraction module L4 to obtain small scale fusion feature map, through the top-down feature fusion (transferring semantic information from high layer to low layer, enhancing the expression ability of low layer features) and bottom-up feature enhancement path (starting from low layer features, gradually adding semantic information, enhancing the comprehensive understanding of details and semantics of the network), the high layer semantic information and low layer detail features are effectively fused, and more accurate target detection is realized in different scales and complex scenes;

[0061] The detection output end splices the three scale fusion features from the neck network, restores them to the original image size through up-sampling, and outputs the position of the photovoltaic panel through convolution operation.

[0062] Optionally, as Figure 4As shown, the variable aggregation convolution, which is embedded with k parallel convolution kernels in a convolution layer, dynamically adjusts the weight of each convolution kernel through the attention mechanism stacked by average pooling layer, fully connected layer, Leaky ReLU activation function, fully connected layer, and softmax, to identify photovoltaic panels of different sizes and distinguish irrelevant backgrounds on high-resolution images. Each convolution kernel is multiplied by its respective weight after convolution operation on the input image tensor, and the final convolution result is obtained by aggregation and addition. Then, the result is output through batch normalization and Leaky ReLU activation function.

[0063] S3, input the RGB aerial photovoltaic panel image as a reference image and the multi-spectral state aerial photovoltaic panel image as a to-be-calibrated image into a time-series dynamic image registration module. The time-series dynamic image registration module uses a deep learning registration method based on a skip network to accurately register single-frame RGB images and multi-spectral state images, and obtains a multi-spectral state calibration image through time-series dynamic accumulation operation of multiple images.

[0064] Optionally, as shown in Figure 5 , in the single-frame image registration process, the reference RGB image I F and the to-be-calibrated multi-spectral state image I M are input into the skip network respectively to extract respective feature maps. After the obtained two feature maps are spliced, down-sampling and flattening are performed through the convolution layer, and the fully connected layer is input to obtain geometric transformation parameters P{t x ,t y ,θ,s x ,s y ,β x ,β y}, wherein t x and t y are translation parameters, θ is a rotation angle, s x and s y are scaling factors, and β x and β y are shear factors. The to-be-calibrated multi-spectral state image is geometrically converted and adjusted (including translation, rotation, scaling, and shearing) through these parameters to achieve registration, and a calibration image IR is obtained.

[0065] The skip network takes an image as input, first extracts preliminary features through convolution and maximum pooling layers, then further extracts deep features through three double-branch residual structures and cross-stage feature extraction modules. The double-branch residual structure includes a single-layer convolution branch and a three-layer convolution branch. After fusion, the Leaky ReLU activation function is used for processing, and the deep features of the image are further extracted through two cross-stage feature extraction modules. Finally, the feature map is output.

[0066] Optionally, the time-series dynamic image registration module, in the time-series dynamic accumulation operation, performs image registration on n frames of RGB images {I F1 ,I F2 ,I F3 ...I Fn} and corresponding n frames of multi-spectral images {I M1 ,I M2 ,I M3 ...I Mn} to obtain registered images {IR1, IR2, IR3...IR n}, then performs frame accumulation on IR1 and IR2 to obtain IR 12 , performs image registration on IR 12 and I F2 to obtain IRR 12 , performs frame accumulation on IRR 12 and IR3 to obtain IRR 123 , and performs image registration on IRR 123 and I F3 to obtain IRRR 123 ; through repeating the above steps, a final multi-spectral calibration image IRR..R 12..n is obtained, wherein the frame accumulation operation refers to adding and averaging corresponding pixel values of two images.

[0067] The embodiment of the application can realize accurate alignment between high-resolution aerial photovoltaic panel RGB images and corresponding multi-spectral images through the time-series dynamic image registration module, and has strong robustness.

[0068] S4, cutting the multi-spectral calibration image according to the position of the photovoltaic panel output by the photovoltaic panel positioning detection network to obtain a multi-spectral photovoltaic panel defect image of a corresponding region;

[0069] S5, inputting the multi-spectral photovoltaic panel defect image into a second-stage photovoltaic panel defect detection network, the photovoltaic panel defect detection network utilizing a multi-spectral spatial model to perform linear modeling on image spectrum and spatial dimensions, effectively adapting to the characteristics of spectral order change, enhancing the representation ability of defect features, and outputting a defect category and a defect detection frame of a photovoltaic panel to be detected, thereby completing photovoltaic panel defect detection.

[0070] Optionally, as shown in Figure 6 , the photovoltaic panel defect detection network is composed of a photovoltaic panel defect detection backbone network and a multi-scale fusion detection output end;

[0071] The photovoltaic panel defect detection backbone network first extracts photovoltaic panel defect features through a 3-dimensional convolution module, the 3-dimensional convolution module is stacked by 3-dimensional convolution, batch normalization and Leaky ReLU activation function in turn, and the input and output are connected by residual connection; next, the feature map is sent to the subsequent stage, the subsequent stage is divided into four stages, each stage is connected in turn, the first stage is composed of a state space feature extraction module; the second and third stages are composed of a down-sampling module and a state space feature extraction module in series; the fourth stage adds an SPPF module on the basis of the previous two stages to enhance the extraction of key features and multi-scale feature fusion, through the above four stages, different scale feature maps are generated, including large, medium and small scales, the feature map size decreases by a factor of two down-sampling relationship, and the generated three scale features will be used as the input of the multi-scale fusion detection output end;

[0072] The multi-scale fusion detection output end inputs small, medium and large scale feature maps from the photovoltaic panel defect detection backbone network, first performs multi-scale fusion processing through a state space feature extraction module, convolution and up-sampling operation, wherein the small scale feature map is restored to the same spatial resolution as the medium scale feature map through up-sampling, and is spliced with the medium scale feature map, and is input to the state space feature extraction module F1, the output of F1 is restored to the same resolution as the large scale feature map through up-sampling, and is spliced with the large scale feature map, and is input to the state space feature extraction module F2 to generate large scale fusion features, the output of F2 is down-sampled to medium scale resolution through convolution, and is spliced with the output of F1, and is input to the state space feature extraction module F3 to obtain medium scale fusion features, the output of F3 is restored to small scale resolution through convolution, and is spliced with the small scale feature map, and is input to the state space feature extraction module F4 to obtain a small scale fusion feature map; then the three scale fusion feature maps are spliced and restored to the original image size through up-sampling; then the feature map dimension is adjusted using a 1*1 convolution, and then processed twice through a 3*3 convolution and once through a 1*1 convolution, finally the defect target class and position are output, and the photovoltaic panel defect detection is completed.

[0073] Optionally, as shown in Figure 7 The state space feature extraction module, the input feature map is first processed through convolution, batch normalization and Leaky ReLU activation function, and then the processed feature is input to a local space module, the local space module is stacked by depth separable convolution, batch normalization, convolution, Leaky ReLU activation function and convolution in turn, and the output of the module is added to the input through a residual structure;

[0074] Then the output of the local space module is input to a multi-spectrum state space model after layer normalization processing, extracts the information combining the spectral dimension and the spatial dimension, and enhances the photovoltaic panel defect feature expression;

[0075] Finally, the obtained feature map is input into a linear layer for further processing, and a residual connection is adopted to add the output of the Leaky ReLU activation function in the preliminary processing to the output of the multi-spectrum state space model and the linear layer, respectively, to obtain an output feature map.

[0076] Optionally, as shown in Figure 8 The multi-spectrum state space model is shown in the figure, and the input is first subjected to image serialization processing, the feature map is cut into 9 spatial slices, and is converted into an image sequence.

[0077] Then the image sequence is input into a spectral dimension branch and a spatial dimension branch for processing, in the spatial dimension branch, the multi-spectrum state space model linearly models the 9 slices of the image sequence along the spatial dimension, considering that the image sequence has context relationship and spatial dependence in four directions, the image sequence is first scanned in a forward direction and a reverse direction respectively, and block matrix rearrangement is performed, the four groups of image sequences after rearrangement are input into an SSM module, four feature sequences output by the SSM module are processed by addition and averaging, and then are decoded back to an image tensor, and finally a spatial dimension feature is obtained through a convolution layer.

[0078] In the spectral dimension branch, the multi-spectrum state space model linearly models c spectral channels of the image sequence, first adjusts the spectral dimension sequence, performs block matrix rearrangement through forward and reverse scanning, and inputs the two groups of image sequences after rearrangement into the SSM module, two image feature sequences output by the SSM module are also processed by addition and averaging, and then are adjusted to the original dimension, decoded back to an image tensor, and finally a spectral dimension feature is obtained through a convolution layer.

[0079] Finally, the spatial dimension feature and the spectral dimension feature are fused to learn the joint features in the space and the spectrum, and finally a feature map is output.

[0080] Optionally, the SSM module is a discrete state space model, which is obtained by discretizing a continuous state space model, by discretizing the state transition process of continuous time, the original continuous state transition matrix and projection matrix are converted into a form suitable for discrete time, and effective modeling of time series data is realized.

[0081] In the implementation of the multi-spectrum state space model, the SSM module can better capture the spatial features and spectral information of the input data by sequentially modeling the input data in the spatial dimension and the spectral dimension, especially in the spectral dimension, the sequentially modeled SSM can adapt to the sequential change of the spectral band, improve the modeling accuracy of the model on the spectral information, and improve the performance of the overall feature learning.

[0082] In the embodiment of the application, the training process of the overall model composed of the first-stage variable aggregation convolution-based lightweight photovoltaic panel positioning detection network, the time-series dynamic image registration module, and the second-stage photovoltaic panel defect detection network is as follows:

[0083] 1) Data collection.

[0084] In the context of a photovoltaic power station, a UAV equipped with a high-resolution RGB camera and a multispectral camera is used to take aerial photographs of the photovoltaic power station area, obtaining multiple sets of RGB images and multispectral image data at different times and angles.

[0085] Photovoltaic panel positioning dataset and photovoltaic panel defect detection dataset construction.

[0086] The photovoltaic panel positioning dataset is labeled by marking the photovoltaic panel area as the detection positioning target and the remaining area as the background in the high-resolution RGB aerial image. The photovoltaic panel defect detection dataset is labeled based on the cropped multispectral photovoltaic panel defect image, marking the defect types such as hot spots and fragments as the detection target and the other areas as the background. After labeling, the photovoltaic panel positioning dataset and the defect detection dataset are divided into training set and test set in the ratio of 8:2 respectively.

[0087] 3) Model training.

[0088] The outputs of photovoltaic panel position detection and photovoltaic panel defect detection are compared with the corresponding true labels, and the respective detection loss functions are calculated. Then, the network parameters are updated by backpropagation according to the detection loss function, and the model is optimized. After multiple rounds of iterative training, the photovoltaic panel positioning and photovoltaic panel defect detection models with optimal detection accuracy are obtained.

[0089] In the training process of the time-series dynamic image registration module, the MSD loss function of the reference image I F and the calibration image IR is calculated, and the network parameters are updated by backpropagation using an optimization algorithm and multiple rounds of training optimization. The MSD loss function formula is as follows:

[0090]

[0091] where I F and I M represent the reference image and the corresponding multispectral image respectively, n represents the total number of pixels in the image, I F(i) represents the value of the i-th pixel in the reference image I F , and I M (T (i,p) ) represents the value of the i-th pixel in the corresponding multispectral image I (i,p) after geometric image transformation T M .

[0092] As Figure 9 shown, the embodiment of the present application also provides a two-stage photovoltaic panel defect detection system based on a multi-spectral spatial model, which comprises:

[0093] The acquisition module 910 is configured to acquire a high-resolution RGB aerial photovoltaic panel image to be detected and a multi-spectral aerial photovoltaic panel image.

[0094] The positioning detection module 920 is configured to input the RGB aerial photovoltaic panel image into a first-stage lightweight photovoltaic panel positioning detection network based on a variable aggregation convolution, the photovoltaic panel positioning detection network performs positioning detection on the position of the photovoltaic panel in the RGB aerial photovoltaic panel image, and outputs the position of the photovoltaic panel.

[0095] The registration module 930 is configured to input the RGB aerial photovoltaic panel image as a reference image and the multi-spectral aerial photovoltaic panel image as a to-be-calibrated image into a time-series dynamic image registration module, the time-series dynamic image registration module performs accurate registration of a single-frame RGB image and a multi-spectral image by using a deep learning registration method based on a skip network, and obtains a multi-spectral calibration image through time-series dynamic accumulation operation of multiple images.

[0096] The cutting module 940 is configured to cut the multi-spectral calibration image according to the position of the photovoltaic panel output by the photovoltaic panel positioning detection network, and obtain a multi-spectral photovoltaic panel defect image of a corresponding region.

[0097] The defect detection module 950 is configured to input the multi-spectral photovoltaic panel defect image into a second-stage photovoltaic panel defect detection network, the photovoltaic panel defect detection network performs linear modeling on image spectrum and spatial dimension by using a multi-spectral spatial model, effectively adapts to the characteristics of spectral order change, enhances the representation ability of defect features, and outputs a defect category and a defect detection frame of the photovoltaic panel to be detected, thereby completing photovoltaic panel defect detection.

[0098] The two-stage photovoltaic panel defect detection system based on the multi-spectral spatial model provided by the embodiment of the present application corresponds to the two-stage photovoltaic panel defect detection method based on the multi-spectral spatial model provided by the embodiment of the present application in terms of functional structure, and thus will not be described here.

[0099] Figure 10is a structural schematic diagram of an electronic device 1000 provided by an embodiment of the present application. The electronic device 1000 can be quite different in configuration or performance, and can include one or more processors (central processing units, CPU) 1001 and one or more memories 1002, wherein the memory 1002 stores at least one instruction, and the processor 1001 loads and executes the at least one instruction to implement the steps of the above-mentioned two-stage photovoltaic panel defect detection method based on a multi-spectrum state space model.

[0100] In an exemplary embodiment, a computer readable storage medium, for example, a memory including instructions, is also provided, and the instructions can be executed by a processor in a terminal to complete the above-mentioned two-stage photovoltaic panel defect detection method based on a multi-spectrum state space model. For example, the computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0101] Those of ordinary skill in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by a program instructing relevant hardware, and the program can be stored in a computer readable storage medium, and the storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0102] The above-mentioned is only a preferred embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A two-stage photovoltaic panel defect detection method based on multi-spectral state space model, characterized in that, The method comprises: S1, obtaining high-resolution RGB aerial photovoltaic panel images to be detected and multispectral aerial photovoltaic panel images; S2, inputting the RGB aerial photovoltaic panel images into a first-stage lightweight photovoltaic panel positioning detection network based on variable aggregation convolution, the photovoltaic panel positioning detection network performing positioning detection on the positions of photovoltaic panels in the RGB aerial photovoltaic panel images, and outputting the positions of the photovoltaic panels; S3, inputting the RGB aerial photovoltaic panel images as reference images and the multispectral aerial photovoltaic panel images as to-be-calibrated images into a time-series dynamic image registration module, the time-series dynamic image registration module performing accurate registration of single-frame RGB images and multispectral images by using a deep learning registration method based on a skip network, and obtaining multispectral calibration images through time-series dynamic accumulation operation of multiple images; S4, cutting the multispectral calibration images according to the positions of the photovoltaic panels output by the photovoltaic panel positioning detection network to obtain multispectral photovoltaic panel defect images of corresponding regions; S5, inputting the multispectral photovoltaic panel defect images into a second-stage photovoltaic panel defect detection network, the photovoltaic panel defect detection network performing linear modeling on image spectrum and spatial dimensions by using a multispectral spatial model, effectively adapting to the characteristics of spectral order change, enhancing the representation ability of defect features, and outputting defect categories and defect detection boxes of the to-be-detected photovoltaic panels to complete photovoltaic panel defect detection.

2. The method of claim 1, wherein, The photovoltaic panel positioning detection network comprises a backbone network, a neck network, and a detection output end. The backbone network is divided into four stages, and each stage is connected in turn, wherein the network of each of the first three stages is composed of a convolution module and a cross-stage feature extraction module in series, and the last stage adds an SPPF module on this basis; the convolution module comprises a convolution layer, a batch normalization layer and a Leaky ReLU activation function, wherein the Leaky ReLU activation function is an improved version of ReLU, and the formula is as follows: In the formula, alpha is a constant, when the input is negative, the output still remains negative, but will not be completely zero, and provides a non-zero gradient alpha for the negative input, which helps the gradient to be more smoothly propagated in the entire network, effectively solving the problem of dead neurons that may occur in the training process of the standard ReLU; the cross-stage feature extraction module, by averaging the channels of the input feature map into two parts, respectively input into two branches, one branch is processed through a variable aggregation convolution, and the other branch is processed through a variable aggregation convolution and a bottleneck module, the outputs of the two branches are spliced, and then the final result is obtained through a variable aggregation convolution, the bottleneck module is composed of two convolution modules, and the input and the output are added through residual connection, the SPPF module comprises three maximum pooling layers connected in series, the output of each maximum pooling layer is spliced with the original input, and then the final output is obtained through a convolution module; the last three stages of the backbone network output small, medium and large scale feature maps respectively, and there is a two times down sampling relationship between the sizes, and these feature maps will be transmitted as input to the neck network. The neck network inputs small, medium, and large scale feature maps from the backbone network and is processed through a cross-stage feature extraction module and convolution and up-sampling operation: first, the small scale feature map is restored to the same spatial resolution as the medium scale feature map through up-sampling and spliced with the medium scale feature map to be input into the cross-stage feature extraction module L1, the output of L1 is restored to the same resolution as the large scale feature map through up-sampling, spliced with the large scale feature map, and then input into the cross-stage feature extraction module L2 to generate large scale fusion features, the output of L2 is down-sampled to a medium scale resolution through convolution and spliced with the output of L1, and then input into the cross-stage feature extraction module L3 to obtain medium scale fusion features, the output of L3 is restored to a small scale resolution through convolution and spliced with the small scale feature map, and then input into the cross-stage feature extraction module L4 to obtain a small scale fusion feature map, through the top-down feature fusion and bottom-up feature enhancement paths, high-level semantic information and low-level detailed features are effectively fused, and more accurate target detection is realized in different scales and complex scenes; The detection output end splices the three scale fusion features from the neck network, restores them to the original image size through up-sampling, and outputs the positions of the photovoltaic panels through convolution operation.

3. The method of claim 1, wherein, The variable aggregation convolution in one convolution layer is embedded with k parallel convolution kernels, and the attention mechanism stacked by an average pooling layer, a full connection layer, a Leaky ReLU activation function, a full connection layer and a softmax is used to dynamically adjust the weight of each convolution kernel, so as to identify photovoltaic panels of different sizes and distinguish irrelevant backgrounds on a high-resolution image, multiply the convolution operation of each convolution kernel on the input image tensor by the respective weight, and finally aggregate and add the final convolution result, and then output the result through batch normalization and a Leaky ReLU activation function.

4. The method of claim 1, wherein, The time sequence dynamic image registration module inputs the reference RGB image I F and the multi-spectrum image I M to be calibrated into a skip network respectively to extract respective feature maps, splices the obtained two feature maps, performs down-sampling and flattening through a convolution layer, inputs a full connection layer, and obtains geometric transformation parameters P{t x ,t y ,θ,s x ,s y ,β x ,β y}, wherein t x , t y are translation parameters, θ is a rotation angle, s x , s y are scaling factors, β x , β y are shearing factors, the multi-spectrum image to be calibrated is adjusted through the parameters to realize geometric image conversion, and a calibrated image IR is obtained. The skip network takes an image as input, first extracts preliminary features through a convolution and a maximum pooling layer, then further extracts deep features through three double-branch residual structures and a cross-stage feature extraction module, the double-branch residual structure includes a single-layer convolution branch and a three-layer convolution branch, after fusion, the features are processed by a Leaky ReLU activation function, and then the deep features of the image are further extracted through two cross-stage feature extraction modules, and finally a feature map is output.

5. The method of claim 1, wherein, The time sequence dynamic image registration module, in the time sequence dynamic accumulation operation, carries out image registration on n frames of RGB images {I F1 , F2 , F3 ...I Fn} and corresponding n frames of multi-spectrum images {I M1 , M2 , M3 ...I Mn} in the image video segment, to obtain registered images {IR1, IR2, IR3...IR n}, then carries out frame accumulation on IR1 and IR2 to obtain IR 12 , carries out image registration on IR 12 and I F2 to obtain IRR 12 ; carries out frame accumulation on IRR 12 and IR3 to obtain IRR 123 , carries out image registration on IRR 123 and I F3 to obtain IRRR 123 ; through repeating the above frame accumulation and image registration steps, obtains the final multi-spectrum calibration image IRR..R 12..n , wherein the frame accumulation operation refers to taking average after adding corresponding pixel values of two images.

6. The method of claim 1, wherein, The photovoltaic panel defect detection network is composed of a photovoltaic panel defect detection backbone network and a multi-scale fusion detection output end. The photovoltaic panel defect detection backbone network first extracts photovoltaic panel defect features through a 3D convolution module, the 3D convolution module is stacked by 3D convolution, batch normalization and Leaky ReLU activation function in turn, and the input and output are connected by a residual connection; next, the feature map is sent to the subsequent stage, the subsequent stage is divided into four stages, each stage is connected in turn, the first stage is composed of a state space feature extraction module; the second and third stages are composed of a down-sampling module and a state space feature extraction module in series; the fourth stage adds an SPPF module on the basis of the first two stages to enhance the extraction of key features and multi-scale feature fusion, through the above four stages, feature maps of different scales are generated, including large, medium and small scales, the feature map size decreases by a factor of two, and the three scales of generated features will be input to the multi-scale fusion detection output end. The multiscale fusion detection output end inputs small, medium and large scale feature maps from the photovoltaic panel defect detection backbone network, and first performs multiscale fusion processing through a state space feature extraction module, convolution and up-sampling operation, wherein the small scale feature map is restored to the same spatial resolution as the medium scale feature map through up-sampling, spliced with the medium scale feature map, input to the state space feature extraction module F1, the output of F1 is restored to the same resolution as the large scale feature map through up-sampling, spliced with the large scale feature map, and input to the state space feature extraction module F2 to generate a large scale fusion feature, the output of F2 is down-sampled to a medium scale resolution through convolution, spliced with the output of F1, input to the state space feature extraction module F3 to obtain a medium scale fusion feature, and the output of F3 is restored to a small scale resolution through convolution, spliced with the small scale feature map, and input to the state space feature extraction module F4 to obtain a small scale fusion feature map; then the fusion feature maps of the three scales are spliced and restored to the original image size through up-sampling; then the feature map dimension is adjusted using 1*1 convolution, and finally the defect target class and position are output after two times of 3*3 convolution and once 1*1 convolution processing, and the photovoltaic panel defect detection is completed.

7. The method of claim 6, wherein, The state space feature extraction module inputs the feature map first through convolution, batch normalization and Leaky ReLU activation function for preliminary extraction processing, and the preliminary processed feature is input to a local space module, the local space module is sequentially stacked by depth separable convolution, batch normalization, convolution, Leaky ReLU activation function and convolution in sequence, and the output of the module is added to the input through a residual structure; Then the output of the local space module is input into a multispectral state space model after layer normalization processing, extracts the spectral dimension and spatial dimension information, and enhances the photovoltaic panel defect feature expression; Finally, the obtained feature map is input into a linear layer for further processing, and residual connection is adopted, and the output of the Leaky ReLU activation function in the preliminary processing is added to the output of the multispectral state space model and the linear layer respectively, to obtain the output feature map.

8. The method of claim 7, wherein, The multispectral state space model first performs image serialization processing, cuts the feature map into 9 spatial slices, and converts them into image sequences; Then the image sequences are input into the spectral dimension branch and the spatial dimension branch for processing, in the spatial dimension branch, the multispectral state space model linearly models the 9 slices of the image sequence along the spatial dimension, considering the context relationship and four-direction spatial dependence of the image sequence, first scanning the image sequence in row and column respectively, and performing block matrix rearrangement, the four groups of rearranged image sequences are input into the SSM module, the output four feature sequences are processed by addition and averaging, decoded back to image tensors, and finally the spatial dimension features are obtained through a layer of convolution; In the spectral dimension branch, the multi-spectral state space model linearly models the c spectral channels of the image sequence. First, the spectral dimension sequence is adjusted, and square matrix rearrangement is performed through forward and reverse scanning. The two rearranged image sequences are input into the SSM module, and the output two image feature sequences are also adjusted to the original dimension after addition and averaging processing. The image tensor is decoded back, and after passing through a convolution layer, the final spectral dimension feature is obtained. Finally, the spatial dimension feature and the spectral dimension feature are fused, and the joint feature in space and spectrum is learned to generate the final feature map output.

9. The method of claim 8, wherein, The SSM module is a discrete state space model obtained by discretizing the continuous state space model. By discretizing the continuous-time state transition process, the original continuous state transition matrix and projection matrix are converted into a form suitable for discrete time, effectively modeling the time series data.

10. A two-stage photovoltaic panel defect detection system based on multi-spectral state space model, characterized in that, The system comprises: An acquisition module configured to acquire a high-resolution RGB aerial photovoltaic panel image to be detected and a multi-spectral aerial photovoltaic panel image; A positioning detection module configured to input the RGB aerial photovoltaic panel image into a first-stage light-weight photovoltaic panel positioning detection network based on a variable aggregation convolution, and to output the position of the photovoltaic panel by positioning and detecting the position of the photovoltaic panel in the RGB aerial photovoltaic panel image; A registration module configured to input the RGB aerial photovoltaic panel image as a reference image and the multi-spectral aerial photovoltaic panel image as a to-be-calibrated image into a time-series dynamic image registration module, to perform accurate registration of a single-frame RGB image and a multi-spectral image by using a deep learning registration method based on a jump network, and to obtain a multi-spectral calibration image through time-series dynamic accumulation of multiple images; A cutting module configured to cut the multi-spectral calibration image according to the position of the photovoltaic panel output by the photovoltaic panel positioning detection network to obtain a multi-spectral photovoltaic panel defect image corresponding to a region; A defect detection module configured to input the multi-spectral photovoltaic panel defect image into a second-stage photovoltaic panel defect detection network, and to output the defect category and defect detection frame of the photovoltaic panel to be detected by using a multi-spectral state space model to linearly model the spectral and spatial dimensions of the image, effectively adapt to the characteristics of spectral order changes, and enhance the representation ability of defect features, thereby completing photovoltaic panel defect detection.

Images