Photovoltaic array inspection method and system based on unmanned aerial vehicle image semantic segmentation

Abstract

The present application relates to the field of photovoltaic array inspection, and provides a photovoltaic array inspection method and system based on unmanned aerial vehicle image semantic segmentation, which uses unmanned aerial vehicle remote sensing pose data to construct an inverse perspective mapping matrix to correct the perspective distortion of the unmanned aerial vehicle visible light image of the photovoltaic array; two parallel independent branches are designed: a structure perception branch explicitly extracts the grid topological arrangement and component physical boundary of the photovoltaic array to form an array structure priori through a differentiable edge operator and a graph convolution network; a texture discrimination branch specially captures local pixel-level abnormal textures through multiple gradient-sensitive asymmetric convolution kernels at specific angles; then, the two features are semantically fused through a cross-attention mechanism; finally, an uncertainty perception gating mechanism is introduced to adaptively enhance the high-ambiguity areas in the fused features that are easy to confuse, and a multi-class defect pixel-level semantic mask is output after decoding, so as to realize accurate identification of photovoltaic array defects.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic array inspection technology, and in particular to a photovoltaic array inspection method and system based on UAV image semantic segmentation. Background Technology

[0002] As a crucial component of clean energy, photovoltaic (PV) power generation has seen continuous expansion in installed capacity, with large-scale ground-mounted PV power plants often covering several square kilometers. During long-term outdoor operation, PV modules are inevitably affected by mechanical stress, thermal cycling, and environmental pollution, making them prone to various defects such as cracks, hot spots, and contamination. If these defects are not detected and addressed promptly, they will lead to a continuous decline in the power generation efficiency of the PV modules, and in severe cases, even cause localized overheating or fires. Therefore, efficient and accurate periodic inspections of large-scale PV arrays are a core operational requirement for ensuring the safe and stable operation of PV power plants. However, traditional manual ground-based inspection methods are inefficient and ill-suited to the inspection needs of large-scale arrays.

[0003] In existing technologies, aerial inspection solutions using drones equipped with visible light cameras have gradually become the industry mainstream due to their wide coverage and high mobility. Building on this, researchers have further introduced deep learning semantic segmentation technology into drone image analysis in order to achieve automated pixel-level identification of defects in photovoltaic modules.

[0004] Existing photovoltaic array inspection solutions based on UAV image semantic segmentation typically employ a general semantic segmentation network with an encoder-decoder architecture, directly performing end-to-end feature extraction and classification prediction on the raw images acquired by the UAV. However, these existing solutions suffer from a core technical bottleneck: when processing photovoltaic array images, the general semantic segmentation network cannot effectively distinguish and utilize two distinct types of visual information in the image. One type is the macroscopic geometric structure information formed by the regular grid arrangement of the photovoltaic array; the other is the microscopic local texture anomaly information presented by defects such as cracks, hot spots, and dirt. The root cause of this problem lies in the fact that when the UAV performs inspection tasks, it is affected by changes in flight attitude, including pitch, yaw, and roll angles. The acquired images suffer from significant perspective distortion, causing irregular geometric deformations of the physical boundaries of the photovoltaic modules in the image, making it difficult for the network to reliably extract macroscopic structural information. At the same time, defects such as cracks have extremely narrow pixel widths and varied orientations, while hot spots and dirt have high similarity in color features. These microscopic texture anomalies are easily confused with normal structural features such as module grid lines and borders. When macroscopic structural information is extracted inaccurately, the network cannot utilize the physical arrangement of the photovoltaic array to constrain the spatial location range of defects, leading to false detections of defects that cross physical boundaries. Conversely, when the discriminative power of microscopic texture features is insufficient, the network struggles to accurately separate real defects from similar normal structures in complex backgrounds, resulting in missed detections. This coupling and confusion of the two types of information makes it difficult for existing solutions to meet the actual needs of industrial-grade operation and maintenance in terms of segmentation accuracy and reliability when facing complex photovoltaic array scenarios with multiple types of defects. Summary of the Invention

[0005] The present invention aims to solve at least one of the problems existing in the prior art, and provides a photovoltaic array inspection method and system based on UAV image semantic segmentation.

[0006] One aspect of the present invention provides a photovoltaic array inspection method based on UAV image semantic segmentation, comprising: Step 1: Based on the camera intrinsic parameter matrix and attitude extrinsic parameter information constructed from the UAV telemetry pose data, the UAV visible light image of the photovoltaic array is subjected to perspective correction and channel normalization processing to obtain the normalized input tensor. Step 2: Perform array topology encoding and geometric feature aggregation on the normalized input tensor to extract the spatial constraint relationship of the photovoltaic module boundary and grid line direction, and generate the array structure prior feature map; Step 3: By pre-setting multiple multi-directional gradient-sensitive convolution kernels with specific angles, local pixel-level anomaly representations of crack direction, hot spot gradient and dirt diffusion pattern are captured from the normalized input tensor to obtain defect texture feature map. Step 4: Perform structure-guided cross-attention semantic fusion on the array structure prior feature map and the defect texture feature map to obtain a cross-attention fusion feature map; Step 5: Perform uncertainty-aware gating and defect semantic mask generation on the cross-attention fusion feature map to obtain multi-class defect semantic masks.

[0007] Optionally, step one includes: Step 1.1: Based on the Euler angles and translation vectors in the UAV telemetry pose data, construct the extrinsic transformation matrix, and multiply the extrinsic transformation matrix with the camera intrinsic matrix to obtain the inverse perspective mapping matrix; Step 1.2: Based on the inverse of the inverse perspective mapping matrix, perform bilinear interpolation projection transformation on the pixel coordinates in the UAV visible light image to obtain a perspective-corrected image; Step 1.3: Scale the perspective-corrected image and standardize the mean and variance of each channel to obtain a normalized input tensor.

[0008] Optionally, step two includes: Step 2.1: Based on the differentiable horizontal and vertical convolution kernels, perform differentiable edge feature extraction on the normalized input tensor to obtain the edge response feature tensor. Step 2.2: Perform patch partitioning and center feature aggregation on the edge response feature tensor, and calculate the topological connection weights between nodes by combining the Gaussian radial basis function and the connectivity distance threshold to obtain the array graph node feature matrix and the array graph adjacency matrix. Step 2.3: Perform graph convolution geometric feature aggregation and back projection on the array graph node feature matrix and array graph adjacency matrix to obtain the array structure prior feature map.

[0009] Optionally, the multi-directional gradient-sensitive convolutional kernels at multiple specific angles include asymmetric convolutional kernels at 0 degrees, 45 degrees, 90 degrees, and 135 degrees.

[0010] Optionally, step three includes: Step 3.1: Perform multi-directional asymmetric convolution calculation on the normalized input tensor using multi-directional gradient-sensitive convolution kernels at multiple specific angles to obtain a set of multi-directional intermediate feature tensors; Step 3.2: Concatenate multi-directional feature channels on the multi-directional intermediate feature tensor set to obtain a concatenated high-dimensional texture tensor; Step 3.3: The cascaded high-dimensional texture tensor is subjected to cross-channel dimensionality reduction projection using pointwise convolution kernels, and then subjected to feature distribution correction and nonlinear recombination through batch normalization layers and nonlinear activation functions to obtain the defect texture feature map.

[0011] Optionally, step four includes: Step 4.1: Perform linear mapping and spatial dimension flattening operations on the prior feature map of the array structure and the defect texture feature map to obtain the query matrix, key matrix and value matrix; The linear mapping between the prior feature map of the array structure and the defect texture feature map includes: A linear mapping is performed between the prior feature map of the array structure and the defect texture feature map according to the following formula: ; ; ; in, This is the 3D query tensor after query projection; To query the pointwise convolution weight parameter matrix of the projection layer; This is the spatial convolution operator; For the prior feature map of the array structure; To query the bias parameter vector of the projection layer; This is the three-dimensional bond tensor after bond projection; It is the three-dimensional value tensor after longitude projection; and These are the pointwise convolutional weight parameter matrices for the key projection layer and the value projection layer, respectively; For defect texture feature maps; and These are the bias parameter vectors for the key projection layer and the value projection layer, respectively. Step 4.2: Calculate the cross-attention weight graph between the query matrix and the key matrix; where the attention weight value in the i-th row and j-th column of the cross-attention weight graph is... Represented as: ; in, This is the structure query vector for the i-th row of the query matrix. Let be the transpose of the texture key vector in the j-th row of the key matrix. This is the transpose of the texture key vector in the nth row of the key matrix. For attention feature dimension, It is an exponential function, where N is the total number of spatial pixels in the defect texture feature map; Step 4.3: Based on the cross-attention weight map, perform weighted aggregation multiplication on the value matrix to recover the two-dimensional spatial structure, thereby obtaining the structure-guided enhanced texture tensor. Then, add the structure-guided enhanced texture tensor to the defect texture feature map pixel-by-pixel residual to obtain the cross-attention fusion feature map; wherein, the feature vector of the cross-attention fusion feature map at spatial coordinates (i,j) is... Represented as: ; in, For the defect texture feature map in spatial coordinates The original texture feature vector at that location, For dimension reshaping operation functions, This is a cross-attention weight graph; For value matrices, For the structure-guided enhanced texture tensor in spatial coordinates The eigenvector at that location.

[0012] Optionally, step five includes: Uncertainty quantization of cross-attention fusion feature maps is performed by pixel-wise Shannon entropy calculation to obtain an uncertainty weight map; Based on the uncertainty weight map as a spatial gate, feature scaling enhancement is performed on easily confused regions in the cross-attention fusion feature map, and then multi-class defect semantic masks are obtained by pixel-by-pixel decoding through the full-convolution integral class head.

[0013] Optionally, step five includes: Step 5.1: Perform initial pixel-level probability distribution estimation on the cross-attention fusion feature map to obtain the initial segmentation probability tensor; Step 5.2: Quantize the initial segmentation probability tensor with information entropy to obtain an uncertain weight map. Then, based on the uncertain weight map, scale the cross-attention fusion feature map to obtain a gated augmented fusion feature map. The gated augmented fusion feature map is located in spatial coordinates... eigenvectors at location Represented as: ; in, For cross-attention fusion feature maps in spatial coordinates The eigenvector at point , where ⊙ is the element-wise multiplication operator. For normalized activation functions, For the uncertainty weighting graph in spatial coordinates The information entropy value at that location, and , This represents the preset total number of semantic segmentation categories for photovoltaic arrays. The initial segmentation probability tensor in spatial coordinates The predicted probability value for belonging to class c. It is the natural logarithm function. A very small positive floating-point constant is introduced to prevent the input of the natural logarithm function from being zero; Step 5.3: Perform full-convolution integral decoding on the gated enhanced fusion feature map and perform maximum value index decoding operation pixel by pixel along the channel dimension to obtain multi-class defect semantic mask.

[0014] Another aspect of the present invention provides a photovoltaic array inspection system based on UAV image semantic segmentation, comprising: The UAV image perspective correction module is used to perform perspective correction and channel normalization processing on the UAV visible light image of the photovoltaic array based on the inverse perspective mapping matrix constructed from the camera intrinsic parameter matrix and attitude extrinsic parameter information in the UAV telemetry pose data, so as to obtain the normalized input tensor. The array topology feature aggregation module is used to perform array topology encoding and geometric feature aggregation on the normalized input tensor to extract the spatial constraint relationship of the photovoltaic module boundary and grid line direction, and generate the array structure prior feature map. The local pixel-level anomaly representation module is used to capture local pixel-level anomaly representations of crack direction, hot spot gradient and dirt dispersion pattern from the normalized input tensor by using multi-directional gradient sensitive convolution kernels with multiple preset specific angles, so as to obtain defect texture feature map. The structure-guided semantic fusion module is used to perform structure-guided cross-attention semantic fusion on the array structure prior feature map and the defect texture feature map to obtain a cross-attention fusion feature map. The defect semantic mask generation module is used to perform uncertainty-aware gating and defect semantic mask generation on cross-attention fusion feature maps to obtain multi-class defect semantic masks.

[0015] Optionally, the defect semantic mask generation module includes: A pixel-level probability distribution estimation unit is used to perform initial pixel-level probability distribution estimation on the cross-attention fusion feature map to obtain the initial segmentation probability tensor; The uncertainty feature scaling unit is used to quantize the information entropy of the initial segmentation probability tensor to obtain the uncertainty weight map, and to scale the uncertainty feature of the cross-attention fusion feature map based on the uncertainty weight map to obtain the gated enhanced fusion feature map. The maximum value index decoding unit is used to perform full-convolution integral class decoding on the gated enhanced fusion feature map and perform maximum value index decoding operation pixel by pixel along the channel dimension to obtain multi-class defect semantic masks.

[0016] Compared to existing technologies, this invention first utilizes UAV telemetry pose data to construct an inverse perspective mapping matrix, correcting perspective distortion in the UAV visible light images of the acquired photovoltaic array to eliminate interference from flight attitude changes on subsequent structural information extraction. Based on this, two parallel independent branches are designed: a structure perception branch explicitly extracts the grid topology and component physical boundaries of the photovoltaic array using differentiable edge operators and graph convolutional networks, forming an array structure prior. The texture discrimination branch specifically captures local pixel-level abnormal textures such as crack direction, hot spot gradients, and dirt dispersion using gradient-sensitive asymmetric convolutional kernels at multiple specific angles. Subsequently, a cross-attention mechanism is used to perform structure-guided semantic fusion of the two feature paths, constraining the spatial scope of micro-texture features with macro-structural priors, fundamentally suppressing false detections across physical boundaries. Finally, an uncertainty-aware gating mechanism based on Shannon entropy is introduced to adaptively enhance highly ambiguous regions in the fused features, followed by full-convolution integral class head decoding to output multi-class defect pixel-level semantic masks, thereby achieving accurate identification of photovoltaic array defects. Attached Figure Description

[0017] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0018] Figure 1 A flowchart of a photovoltaic array inspection method based on UAV image semantic segmentation is provided in one embodiment of the present invention; Figure 2 This is a data flow diagram illustrating a photovoltaic array inspection method based on UAV image semantic segmentation, provided in another embodiment of the present invention. Figure 3 A flowchart of step two in a photovoltaic array inspection method based on UAV image semantic segmentation, provided in another embodiment of the present invention; Figure 4 A flowchart of step five in a photovoltaic array inspection method based on UAV image semantic segmentation, provided in another embodiment of the present invention; Figure 5 The diagram below shows a structural block diagram of a photovoltaic array inspection system based on UAV image semantic segmentation, which is provided as another embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details are presented in the embodiments of the present invention to facilitate a better understanding of the invention. However, the technical solutions claimed in the present invention can be implemented even without these technical details and various variations and modifications based on the following embodiments. The division of the following embodiments is for ease of description and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined with and referenced by each other without contradiction.

[0020] Existing photovoltaic array inspection schemes based on UAV image semantic segmentation face the challenge of effectively decoupling and co-utilizing two types of heterogeneous information: macroscopic geometric structure and microscopic defect texture. Perspective distortion introduced by flight attitude makes it difficult to stably extract the physical boundaries of the components, while defects such as cracks, hot spots, and dirt are easily confused with normal structural features such as grid lines and borders, ultimately leading to both false positives and false negatives. To address this, this invention proposes a photovoltaic array inspection method and system based on UAV image semantic segmentation. In implementation, this invention first performs perspective correction and normalization preprocessing on the visible light images of the photovoltaic array collected by the UAV, based on the pose data from synchronous telemetry, to establish a stable orthophoto input foundation. Then, a dual-branch parallel architecture processes two types of heterogeneous information: the structure perception branch extracts the gradient response of component boundaries using a differentiable edge operator, and then aggregates the geometric priors of the array arrangement at the topological level through a graph convolutional network; the texture discrimination branch responds to directional texture anomalies such as cracks, hot spots, and dirt using asymmetric convolutional kernels at multiple preset specific angles, and completes feature compression and reorganization through channel cascading and pointwise convolution. The two feature paths then complete structure-guided semantic fusion through a cross-attention mechanism, explicitly constraining the spatial propagation range of texture features with physical boundary priors, eliminating cross-boundary false detections at the mechanism level. Finally, Shannon information entropy is calculated pixel-by-pixel for the fused features to quantify classification uncertainty. The feature responses of highly ambiguous regions are adaptively amplified and then decoded at the pixel level using a fully convolutional integral classifier, outputting the final defect semantic mask, thereby achieving accurate identification of defects in the photovoltaic array.

[0021] Figure 1 A flowchart of a photovoltaic array inspection method based on UAV image semantic segmentation provided in an embodiment of the present invention. Figure 2 This is a data flow diagram illustrating a photovoltaic array inspection method based on UAV image semantic segmentation, provided as an embodiment of the present invention. (In conjunction with...) Figure 1 and Figure 2The photovoltaic array inspection method based on UAV image semantic segmentation provided in this embodiment of the invention includes: Step 1, performing perspective correction and channel normalization processing on the UAV visible light image of the photovoltaic array based on the inverse perspective mapping matrix constructed from the camera intrinsic parameter matrix and attitude extrinsic parameter information in the UAV telemetry pose data to obtain a normalized input tensor; Step 2, performing array topology encoding and geometric feature aggregation processing on the normalized input tensor to extract the spatial constraint relationship of the photovoltaic module boundary and grid line direction, and generating a priori feature map of the array structure; Step 3, capturing local pixel-level anomaly representations of crack direction, hot spot gradient, and dirt dispersion pattern from the normalized input tensor through multi-directional gradient sensitive convolution kernels with multiple preset specific angles to obtain a defect texture feature map; Step 4, performing structure-guided cross-attention semantic fusion on the array structure prior feature map and the defect texture feature map to obtain a cross-attention fusion feature map; Step 5, performing uncertainty-aware gating and defect semantic mask generation on the cross-attention fusion feature map to obtain a multi-class defect semantic mask.

[0022] Specifically, step one uses an inverse perspective mapping matrix constructed based on the camera intrinsic and attitude extrinsic parameters in the UAV telemetry pose data to perform perspective correction and channel normalization on the UAV visible light image of the photovoltaic array, in order to obtain a normalized input tensor. It should be understood that when the UAV performs a photovoltaic array inspection flight mission, its attitude is continuously affected by factors such as airflow disturbances, flight path turns, and altitude changes, causing the angle between the optical axis of the airborne camera and the photovoltaic array plane to be in a dynamic offset state. This results in varying degrees of perspective distortion in the physical boundaries, grid line orientation, and defect morphology of the photovoltaic modules in the acquired UAV visible light image of the photovoltaic array. If the original image containing perspective distortion is directly used as input for subsequent steps, the rectangular physical outline of the photovoltaic modules will appear as an irregular trapezoid or parallelogram, and the equidistant arrangement between modules will also be disrupted. This will severely interfere with the accurate extraction of the array topological boundaries by the structure perception branch, and at the same time, prevent the gradient-sensitive convolution kernel with the preset angle in the texture discrimination branch from aligning with the true orientation of the defects, ultimately leading to a significant decrease in segmentation accuracy. Therefore, this embodiment of the invention further constructs an inverse perspective mapping matrix based on the camera intrinsic parameter matrix and attitude extrinsic parameter information in the UAV telemetry pose data. This matrix performs perspective correction and channel normalization on the UAV visible light images of the photovoltaic array, thereby uniformly mapping photovoltaic array images acquired under different flight attitudes to an orthogonal top-down viewpoint and standardizing the pixel value distribution to the numerical range required by the deep learning network. This eliminates the interference of geometric distortion introduced by UAV attitude changes on subsequent bi-branch feature extraction, ensuring that the physical boundaries of the photovoltaic modules are restored to regular rectangular outlines in the image and that the array arrangement presents a true equidistant grid relationship. Simultaneously, it ensures that images acquired in different batches have consistent numerical distribution characteristics, laying a reliable data foundation for the stable extraction of subsequent array structure prior feature maps and defect texture feature maps.

[0023] For example, step one includes: step 1.1, constructing an extrinsic transformation matrix based on the Euler angles and translation vectors in the UAV telemetry pose data, and multiplying the extrinsic transformation matrix with the camera intrinsic matrix to obtain an inverse perspective mapping matrix; step 1.2, performing bilinear interpolation projection transformation on the pixel coordinates in the UAV visible light image based on the inverse matrix of the inverse perspective mapping matrix to obtain a perspective-corrected image; step 1.3, performing scale scaling and channel-wise mean-variance standardization on the perspective-corrected image to obtain a normalized input tensor.

[0024] Accordingly, step 1.1 is used to construct an extrinsic transformation matrix based on the Euler angles and translation vectors in the UAV telemetry pose data, and then multiply the extrinsic transformation matrix with the camera intrinsic parameter matrix to obtain the inverse perspective mapping matrix. It should be understood that when the UAV performs inspection flights above the photovoltaic array, its attitude continuously changes due to the combined effects of flight path planning, wind disturbances, and turning maneuvers. The spatial geometric relationship between the optical axis of the onboard camera and the plane where the photovoltaic array is located dynamically shifts accordingly. This shift is fully recorded in the Euler angles and translation vectors in the telemetry pose data output by the UAV flight control system. If the above attitude information is not jointly modeled with the camera's own optical imaging parameters, it is impossible to establish a precise mathematical mapping relationship between the image pixel coordinates and the physical coordinates of the photovoltaic array's ground plane, and subsequent perspective correction operations will lose their geometric transformation basis. Therefore, this embodiment of the invention constructs an extrinsic transformation matrix based on the Euler angles and translation vectors in the UAV telemetry pose data, and multiplies the extrinsic transformation matrix with the camera intrinsic parameter matrix to obtain an inverse perspective mapping matrix. This encodes the spatial rotation and translation information of the UAV's flight attitude, along with optical parameters such as camera focal length and principal point offset, into a single homography transformation matrix, establishing a complete projection mapping model from the image pixel plane to the photovoltaic array ground plane. This provides a precise and uniquely determined geometric transformation operator for subsequent perspective correction of the UAV's visible light images, ensuring that each frame acquired under different flight attitudes can be accurately projected to a unified orthographic top-down viewpoint according to its corresponding inverse perspective mapping matrix.

[0025] For example, the implementation process of step 1.1 is as follows: First, extract three Euler angle components that are strictly time-synchronized with the current frame of the UAV's visible light image from the telemetry pose data output by the UAV flight control system. These components are the roll angle describing the degree of rotation of the UAV around the front and rear axes, the pitch angle describing the degree of pitch of the UAV around the left and right axes, and the yaw angle describing the degree of yaw of the UAV around the vertical axis. At the same time, extract the translation vectors describing the displacement of the camera's optical center relative to the origin of the photovoltaic array's ground plane coordinates in the three spatial axis directions. Then, according to the standard transformation rules from Euler angles to rotation matrices, construct the basic rotation matrices around the three coordinate axes respectively and multiply them according to a predetermined rotation order to obtain a third-order rotation matrix describing the rotation relationship between the camera coordinate system and the photovoltaic array's ground plane world coordinate system. Concatenate this rotation matrix and the translation vector column-wise to construct the extrinsic parameter transformation matrix. This extrinsic parameter transformation matrix completely encodes the spatial pose state of the UAV when capturing the current frame image. Next, the camera intrinsic parameter matrix is ​​read from the camera calibration file. This intrinsic parameter matrix includes the horizontal focal length factor, the vertical focal length factor, and the horizontal and vertical offsets of the image principal point in the pixel coordinate system. Furthermore, the camera intrinsic parameter matrix and the extrinsic transformation matrix are multiplied. Since the height coordinate of the plane where the photovoltaic array is located is assumed to be zero when constructing the mapping relationship, the influence of the third column of the rotation matrix is ​​eliminated. The result of the matrix multiplication is a 3x3 homography matrix, namely the inverse perspective mapping matrix. This inverse perspective mapping matrix integrates the spatial pose information of the UAV with the optical imaging parameters of the camera into a unified geometric transformation operator, establishing a complete homography transformation relationship between the pixel coordinates of the UAV's visible light image and the physical coordinates of the photovoltaic array's ground plane. This provides a precise geometric transformation basis for subsequent bilinear interpolation projection transformation of the UAV's visible light image based on the inverse matrix of this matrix to obtain a perspective-corrected image.

[0026] Accordingly, step 1.2 is used to perform bilinear interpolation projection transformation on the pixel coordinates in the UAV visible light image based on the inverse matrix of the inverse perspective mapping matrix to obtain a perspective-corrected image. It should be understood that the inverse perspective mapping matrix obtained in step 1.1 describes the forward projection relationship from the image pixel plane to the photovoltaic array ground plane. If this forward mapping is directly used to perform coordinate transformation pixel by pixel on the UAV visible light image, it will result in a large number of hole areas not covered by the source pixels in the target orthophoto image, resulting in severe pixel loss and image tearing, and failing to generate a complete and continuous correction result. However, the inverse mapping strategy can ensure that every pixel in the target image can find a corresponding sampling position in the source image. At the same time, since the source coordinates after inverse mapping often fall at non-integer positions, if the nearest neighbor method is used, obvious jagged effects and block artifacts will be introduced, causing distortion of the geometric shape of fine defects such as photovoltaic module boundaries and cracks. Therefore, this embodiment of the invention further performs bilinear interpolation projection transformation on the pixel coordinates in the visible light image of the UAV based on the inverse of the inverse perspective mapping matrix to obtain a perspective-corrected image. This ensures that every pixel position in the target orthophoto image is effectively filled through an inverse sampling strategy, and performs sub-pixel-level smooth estimation of pixel values ​​at non-integer source coordinates through a bilinear interpolation algorithm. In this way, high-quality perspective-corrected images without holes or jagged artifacts can be generated, so that the physical boundaries of the photovoltaic modules are restored to regular rectangular outlines after correction, the equidistant arrangement relationship between modules is realistically restored, and the geometric morphology of defects such as cracks, hot spots, and dirt also maintains consistency with the actual physical morphology, providing geometrically accurate and visually continuous input data for subsequent bi-branch feature extraction.

[0027] For example, the implementation process of step 1.2 is as follows: First, perform a standard matrix inversion operation on the inverse perspective mapping matrix obtained in step 1.1 to obtain the inverse matrix of the inverse perspective mapping matrix. This inverse matrix describes the reverse mapping relationship from the target orthographic top view image coordinate system to the UAV visible light image pixel coordinate system. That is, given any pixel coordinate in the target image, its corresponding sampling position in the source image can be calculated through this inverse matrix. Next, based on the actual physical coverage area of ​​the photovoltaic array and the required spatial resolution, the canvas size of the target orthographic top view image is predetermined and a blank canvas is established. Each integer pixel coordinate point on the canvas is systematically traversed, and each target pixel coordinate is multiplied by the inverse matrix of the inverse perspective mapping matrix in homogeneous coordinate form. After homogeneous coordinate normalization, the source coordinates corresponding to the target pixel in the UAV visible light image are obtained. Since the horizontal and vertical components of the source coordinates are floating-point values ​​rather than integers, their landing points are located inside the cell enclosed by four adjacent integer pixels in the UAV visible light image. Then, with the floating-point source coordinates as the center, the coordinates of the four nearest neighbor integer pixels are located. The system reads the red, green, and blue color values ​​of the four integer pixels in the visible light image of the UAV. Using the horizontal and vertical distance differences between the floating-point source coordinates and the four integer pixels as weighting coefficients, it first performs a linear weighted summation on the upper and lower pairs of pixels along the horizontal direction, and then performs a linear weighted summation on the two horizontal interpolation results along the vertical direction. This calculates the precise color value at the floating-point source coordinates after sub-pixel level smoothing estimation, and writes this color value to the corresponding position on the target canvas. Further, after all pixel coordinates on the target canvas have undergone the aforementioned inverse mapping and bilinear interpolation sampling, a perspective-corrected image is generated. In this perspective-corrected image, the physical boundaries of each component of the photovoltaic array have been restored from the irregular trapezoids or parallelograms presented in the original image due to flight attitude deviation to regular rectangular outlines. The spacing between adjacent components is consistent with the actual physical arrangement. The linear direction of cracks, the diffusion contour of hot spots, and the coverage of dirt all maintain geometric correspondence with the real physical form. This perspective-corrected image will serve as the input data for the next step of scaling and channel-by-channel mean-variance standardization.

[0028] Accordingly, step 1.3 is used to scale and standardize the mean and variance of each channel on the perspective correction image to obtain a normalized input tensor. It should be understood that the spatial resolution of the perspective correction image obtained in step 1.2 depends on the UAV's flight altitude, the physical pixel density of the camera sensor, and the preset parameters of the target orthophoto canvas. Perspective correction images generated from different inspection flights, or even different segments within the same flight, will differ in width and height. The subsequent array topology coding branch and the convolutional layers, graph convolutional layers, and attention computation layers in the multi-directional gradient-sensitive convolution branch all require the input data to have a fixed and uniform spatial dimension. Inputs with inconsistent dimensions will directly lead to a mismatch in the tensor operation dimensions, preventing execution. Furthermore, the pixel values ​​of each color channel in the perspective correction image are still within the integer range of 0 to 255. Images taken under different lighting conditions show significant differences in the mean brightness and variance of contrast. If the original pixel values ​​with inconsistent numerical distributions are directly input into the deep learning network, the activation values ​​of each layer will be in an unstable numerical range, leading to gradient explosion or vanishing during gradient propagation, severely affecting the effectiveness and stability of feature extraction. Therefore, this embodiment of the invention further performs scale scaling and channel-wise mean-variance standardization on the perspective correction image to obtain a normalized input tensor. This uniformly adjusts perspective correction images of different resolutions to the fixed spatial size required by the deep learning network, and corrects the pixel value distribution of each color channel to a standard numerical range with zero mean and unit variance. This eliminates the influence of different inspection runs and different lighting conditions on the spatial dimension and numerical distribution of the input data, ensuring that the subsequent extraction of prior feature maps of the array structure and defect texture feature maps is performed under numerically stable conditions. It also ensures that the gradient propagation of convolution and graph convolution operations in the dual-branch network is within a reasonable numerical range, laying a data-level foundation for reliable inference of the entire semantic segmentation model.

[0029] For example, the implementation process of step 1.3 is as follows: First, read the original spatial resolution of the perspective correction image generated in step 1.2, determine its current width and height pixel counts, and obtain the fixed input size preset by the deep learning network. Then, use a bilinear interpolation resampling algorithm to simultaneously perform scaling operations on the perspective correction image in both the horizontal and vertical directions, adjusting its spatial resolution to a fixed width and height that are completely consistent with the network's preset input size. During the scaling process, the bilinear interpolation algorithm performs a weighted average of adjacent pixels to maintain the visual continuity of the photovoltaic module's boundary and defect texture. Next, separate the scaled image according to the red, green, and blue color channels. Divide each pixel value in each channel by 255, mapping its integer value range from zero to 255 to a floating-point value range from zero to one, completing the basic numerical domain conversion. Then, for each floating-point converted pixel value... For each color channel, pixel-wise mean-variance standardization is performed using the preset global statistical mean and global standard deviation for that channel. This involves subtracting the preset mean from each floating-point pixel value in each channel and then dividing by the preset standard deviation. After this processing, the pixel value distribution of each channel is corrected to a standard normal distribution range centered at zero with a variance of one, eliminating the systematic shift in pixel value distribution caused by different lighting intensities and color temperatures. Furthermore, the standardized color channel data is rearranged according to a tensor arrangement format with the channel dimension first, the spatial height dimension in the middle, and the spatial width dimension last, ultimately generating a normalized input tensor. This normalized input tensor has a fixed spatial size and a standardized numerical distribution, and will simultaneously serve as the common input data for the array topology encoding and geometric feature aggregation processing in step two, as well as the two parallel branches of multi-directional gradient-sensitive convolution in step three.

[0030] Specifically, step two is used to perform array topology encoding and geometric feature aggregation on the normalized input tensor to extract the spatial constraint relationships of the photovoltaic module boundaries and grid line directions, generating a priori feature map of the array structure. It should be understood that the photovoltaic array exhibits a highly regular grid-like arrangement in physical space, with fixed spacing between adjacent modules. The grid lines inside the modules are arranged equidistantly and parallel in a specific direction. This macroscopic geometric information constitutes an important spatial constraint basis for distinguishing defective regions from normal regions. For example, cracks can only exist within the physical boundary of a single module and will not cross the physical gaps between modules. The diffusion range of hot spots is also limited by the physical contour of the module. If the subsequent semantic segmentation process lacks explicit perception of the above array arrangement rules and module boundary positions, the network will be unable to determine whether a certain abnormal texture response is within a reasonable physical spatial range, leading to false detections of defects crossing module boundaries or misjudging gaps between adjacent modules as cracks, resulting in incorrect segmentation results. Traditional convolutional neural networks only extract features within a local receptive field, making it difficult to capture the global topological arrangement relationships and long-range spatial constraints of the photovoltaic array spanning multiple modules. Therefore, this embodiment of the invention further performs array topology encoding and geometric feature aggregation on the normalized input tensor to extract the spatial constraint relationships of the photovoltaic module boundary and grid line direction, generating a priori feature map of the array structure. This explicitly encodes the macroscopic grid arrangement pattern of the photovoltaic array and the physical boundary positions of the modules into high-dimensional feature representations, providing structural-level spatial constraint information for the subsequent cross-attention fusion stage. This enables the subsequent semantic segmentation process to have the ability to perceive the global topology of the photovoltaic array when performing defect localization and classification, effectively constraining the spatial scope of defect features, fundamentally suppressing cross-module false detection problems caused by ignoring physical boundaries, and improving the spatial localization accuracy of multi-class defect segmentation.

[0031] Figure 3 This is a flowchart of step two of the photovoltaic array inspection method based on UAV image semantic segmentation according to an embodiment of the present invention. Figure 3 As shown, step two includes: Step 2.1, based on differentiable horizontal and vertical convolution kernels, differentiable edge features are extracted from the normalized input tensor to obtain the edge response feature tensor; Step 2.2, the edge response feature tensor is divided into graph patches and aggregated with center features, and the topological connection weights between nodes are calculated by combining the Gaussian radial basis function and the connectivity distance threshold to obtain the array graph node feature matrix and the array graph adjacency matrix; Step 2.3, the array graph node feature matrix and the array graph adjacency matrix are aggregated with graph convolutional geometric features and back-projected to obtain the array structure prior feature map.

[0032] Accordingly, step 2.1 is used to extract differentiable edge features from the normalized input tensor based on differentiable horizontal and vertical convolutional kernels to obtain the edge response feature tensor. It should be understood that the physical boundaries of the photovoltaic array components, the installation gaps between components, and the internal grid lines of the components all appear as areas in the image where local brightness changes drastically. This structural edge information is the basic data source for subsequently constructing the array topology. However, the aforementioned edge information in the normalized input tensor is mixed with the uniform background area and defect texture area on the component surface. If a dedicated edge response extraction is not performed, subsequent patch segmentation and topology encoding will not be able to accurately locate the spatial position of the component boundaries and the extension direction of the grid lines. Furthermore, considering that the entire semantic segmentation network needs to perform end-to-end gradient optimization of each layer's parameters through backpropagation during the training phase, using a traditional non-differentiable edge detection operator would truncate the gradient propagation link at this point, preventing the structure-aware branch from jointly optimizing with the subsequent graph convolutional layers and cross-attention fusion layers. Therefore, this embodiment of the invention further performs differentiable edge feature extraction on the normalized input tensor based on differentiable horizontal and vertical convolution kernels to obtain an edge response feature tensor. This allows for the explicit extraction of directional gradient response information at the physical boundaries and grid lines of the photovoltaic module from the normalized input tensor, while ensuring that the gradients in the entire edge extraction process can propagate backward along the computation graph. This transforms the structural edge information implicit in the normalized input tensor into an explicit high-gradient response representation, making the photovoltaic module boundaries and grid lines appear as clear high-response regions in the edge response feature tensor. Simultaneously, it maintains the integrity of the gradient propagation link between the structure-aware branch and the entire network, providing accurate and optimizable edge feature input for subsequent tile partitioning and topology connection weight calculation based on the edge response feature tensor.

[0033] For example, the implementation process of step 2.1 is as follows: First, two sets of directional convolution kernels with fixed parameters are preset in the structure-aware branch, namely the horizontal convolution kernel and the vertical convolution kernel. The weight coefficients of the horizontal convolution kernel exhibit a symmetrical distribution pattern of negative on the left and positive on the right along the horizontal direction, which is used to detect edge structures in the image that have abrupt changes in brightness along the horizontal direction. The weight coefficients of the vertical convolution kernel exhibit a symmetrical distribution pattern of negative on the top and positive on the bottom along the vertical direction, which is used to detect edge structures in the image that have abrupt changes in brightness along the vertical direction. Both sets of convolution kernels are implemented in the form of a standard two-dimensional convolutional layer. Their weight parameters are assigned according to the preset edge detection template during network initialization and remain fixed during the inference phase. Since the convolution operation itself is a differentiable linear operation, the gradient can propagate backward through this layer without hindrance during the training phase. Next, the normalized input tensor is subjected to independent two-dimensional spatial convolution operations with the horizontal convolution kernel and the vertical convolution kernel, respectively. The horizontal convolution kernel performs independent two-dimensional spatial convolution operations with the normalized input tensor. During the upward sliding calculation, the pixel values ​​within each local receptive field are weighted and summed to output the horizontal brightness gradient response value at that location. The vertical convolution kernel outputs the vertical brightness gradient response value at each location in the same way, thus obtaining the horizontal gradient response tensor and the vertical gradient response tensor respectively. Then, the horizontal gradient response tensor and the vertical gradient response tensor are squared at the corresponding pixel positions and added pixel by pixel. The sum is then squared pixel by pixel to calculate the overall edge response intensity at each pixel position, which integrates the gradient information from the two orthogonal directions. Finally, the edge response feature tensor is obtained. In this edge response feature tensor, the physical boundary line of the photovoltaic module, the installation gap between adjacent modules, and the grid lines arranged along a specific direction inside the module all exhibit high gradient response values, while the uniform area on the module surface and the background areas such as the sky and the ground exhibit low response values ​​close to zero. This edge response feature tensor will be used as the input data for the next step of patch segmentation and center feature aggregation.

[0034] Accordingly, step 2.2 is used to perform patch partitioning and center feature aggregation on the edge response feature tensor, and to calculate the topological connection weights between nodes by combining the Gaussian radial basis function and the connectivity distance threshold, so as to obtain the array graph node feature matrix and the array graph adjacency matrix. It should be understood that the edge response feature tensor obtained in step 2.1 is a dense pixel-level feature representation with the same spatial resolution as the normalized input tensor. Each pixel independently records the edge gradient intensity at that location. However, the topological arrangement of a photovoltaic array is a global spatial structure information spanning multiple components. The local gradient value of a single pixel cannot express the macroscopic topological features such as the adjacency relationship between components, the spacing between components, and the grid orientation. If the dense pixel-level edge features are directly input into the subsequent graph convolutional network, on the one hand, the computational complexity will increase exponentially with the square of the total number of pixels, and on the other hand, the fragmented noise at the pixel level will interfere with the modeling of the global topological relationship. Therefore, it is necessary to convert the dense pixel-level representation into a sparse graph structure representation. At the same time, when constructing the graph structure, the connection relationship between nodes must be reasonably defined so that it can accurately reflect the physical fact that there is a physical adjacency between adjacent components in the photovoltaic array, while there is no direct topological association between distant components. If the connection weight between nodes does not decay reasonably with spatial distance or if no truncation threshold is set, the graph structure will degenerate into a fully connected graph, losing the ability to selectively model the local adjacency topology of the photovoltaic array. Therefore, this embodiment of the invention further performs patch partitioning and center feature aggregation on the edge response feature tensor, and calculates the topological connection weights between nodes by combining the Gaussian radial basis function and the connectivity distance threshold to obtain the array graph node feature matrix and the array graph adjacency matrix. This compresses the dense pixel-level edge features into a sparse node representation in units of patches, and explicitly models the physical adjacency topology between adjacent components in the photovoltaic array through a distance-sensitive connection weight mechanism. In this way, while significantly reducing the computational complexity of subsequent graph convolution operations, the distribution of non-zero connection weights in the array graph adjacency matrix accurately corresponds to physically adjacent component pairs in the photovoltaic array, while the connection weights between physically unrelated, distant component pairs are truncated to zero, providing a compact, efficient, and topologically accurate graph structure input for subsequent graph convolution geometric feature aggregation.

[0035] For example, the implementation process of step 2.2 is as follows: First, a uniform tile partitioning operation is performed on the edge response feature tensor in the spatial dimension. The spatial region of the edge response feature tensor is divided into multiple non-overlapping rectangular local receptive field regions according to the preset tile size. The spatial range covered by each tile is adapted to the physical size of one or more photovoltaic modules. The central region aggregation calculation is performed on the edge response feature values ​​inside each tile, that is, the feature values ​​in the central neighborhood of the tile are extracted and mean pooling is performed to obtain a representative feature vector of fixed dimension as the tile segment. Point features are generated by arranging the representative feature vectors of all graph tiles into an array of graph node feature matrices. The number of rows in this matrix equals the total number of graph tiles (i.e., the total number of graph nodes), and the number of columns equals the dimension of the feature vector of each node. Next, the two-dimensional center coordinates of each graph node in the original spatial coordinate system of the edge response feature tensor are extracted. The Euclidean spatial distance between any two graph nodes is calculated, and the Gaussian radial basis function is used to map the Euclidean spatial distance between nodes to a continuous weight value between zero and one. Nodes with closer spatial distances have weight values ​​closer to one, and nodes with greater spatial distances have weight values ​​closer to one. Approaching zero, the decay rate of this mapping relationship is controlled by the bandwidth parameter of the Gaussian kernel function; then, a preset connectivity distance threshold is introduced to hard truncate the above continuous weight values, forcing the weight values ​​between node pairs whose Euclidean spatial distance exceeds the connectivity distance threshold to zero. This connectivity distance threshold is set based on the actual physical spacing between adjacent components in the photovoltaic array, ensuring that only node pairs corresponding to physically directly adjacent components retain non-zero connection weights, while connections between distant node pairs spanning multiple component distances are completely severed. This generates an array graph adjacency matrix. The adjacency matrix is ​​a symmetric sparse square matrix with the number of rows and columns equal to the total number of graph nodes. The spatial distribution pattern of non-zero elements in the matrix accurately reflects the physical adjacency topology between adjacent components in the photovoltaic array. Furthermore, the array graph node feature matrix and the array graph adjacency matrix together constitute a complete graph representation of the macroscopic structure of the photovoltaic array. The array graph node feature matrix encodes the edge gradient feature information of each local region, and the array graph adjacency matrix encodes the topological connection relationship and connection strength between each region. Both will serve as input data for the next step of graph convolution geometric feature aggregation and backprojection.

[0036] Accordingly, step 2.3 is used to perform graph convolution geometric feature aggregation and back projection on the array graph node feature matrix and the array graph adjacency matrix to obtain the array structure prior feature map. It should be understood that the feature vector of each node in the array graph node feature matrix obtained in step 2.2 only encodes the local edge gradient information of its own plot, and has not yet fused the geometric arrangement features from neighboring nodes. However, the global topological structure information of the photovoltaic array is precisely contained in the spatial arrangement relationship and spacing rules between adjacent components. The isolated local features of a single node cannot express the relative positional relationship, arrangement direction consistency and spacing uniformity of the component where the node is located and its surrounding components. At the same time, the array graph node feature matrix and the array graph adjacency matrix are both sparse graph structure data representations, and their spatial resolution is much lower than the pixel-level resolution of the normalized input tensor. The cross-attention semantic fusion in the subsequent step four requires the structural prior features and defect texture features to be aligned pixel by pixel at the same spatial resolution. If the graph structure features are not restored to a dense pixel-level spatial resolution, the two features will not be able to perform cross-attention operations due to dimension mismatch. Therefore, this embodiment of the invention further performs graph convolution geometric feature aggregation and backprojection on the array graph node feature matrix and the array graph adjacency matrix to obtain the array structure prior feature map. This allows the geometric features of adjacent nodes to be aggregated along the topological connection relationship through graph convolution operations, thereby obtaining global array arrangement perception capability. Furthermore, spatial backprojection restores the sparse graph node features to a dense pixel-level feature representation with the same resolution as the normalized input tensor. In this way, each pixel position in the array structure prior feature map encodes the geometric arrangement information and physical boundary constraints of the component to which that position belongs in the entire photovoltaic array global topology, and its spatial resolution is completely aligned with the subsequent defect texture feature map, providing dimension-matched and semantically rich structural constraint input for the structure-guided cross-attention semantic fusion in step four.

[0037] For example, the implementation process of step 2.3 is as follows: First, a self-loop addition operation is performed on the adjacency matrix of the array graph, that is, a unit value is added to the diagonal position of the matrix, so that each graph node can retain its original feature information in the subsequent feature aggregation process without being completely covered by the features of adjacent nodes. Then, the diagonal degree matrix of the array graph adjacency matrix after adding the self-loop is calculated. Each diagonal element of the degree matrix is ​​the sum of all connection weights of the corresponding node. Then, the array graph adjacency matrix after adding the self-loop is symmetrically normalized using the negative half power of the degree matrix to obtain the normalized adjacency matrix. This normalization operation ensures that the feature contributions between nodes with different numbers of connections are reasonably balanced during the feature aggregation process, avoiding To prevent high-connectivity nodes from excessively dominating features for low-connectivity nodes, a matrix multiplication operation is performed between the normalized adjacency matrix and the array graph node feature matrix. This operation physically means that each graph node, based on the topological connectivity weights defined by the normalized adjacency matrix, performs a weighted sum of its own features and the features of all its neighboring nodes, achieving local geometric feature aggregation along the physical adjacency relationships of the photovoltaic array. The aggregated result is then multiplied with the learnable weight parameter matrix in the graph convolutional layer to perform a linear transformation of the feature space. Finally, a nonlinear activation function is applied to introduce nonlinear expressive power, resulting in an updated high-dimensional node feature matrix. In this matrix, the feature vector of each node has been fused with its features within the photovoltaic array. The edge gradients and geometric arrangement information of directly adjacent component nodes in the topology enable the perception of local array structures. Then, the two-dimensional center coordinates of each graph node in the original spatial coordinate system of the edge response feature tensor are recorded during the previous step of tile partitioning. Using these discrete center coordinates as anchor points, a bilinear interpolation backprojection algorithm is employed to map the feature vectors of each node in the updated high-dimensional node feature matrix back to the same two-dimensional spatial resolution as the normalized input tensor. That is, for each pixel coordinate in the target dense feature map, bilinear weighted interpolation is performed based on the spatial distance relationship between it and the center coordinates of its four nearest surrounding graph nodes to calculate the feature vector value at that pixel location. This process transforms sparse graph nodes... The high-level features are expanded into dense pixel-level feature representations. Further, after the above back-projection operation, an array structure prior feature map is generated. This array structure prior feature map has the same spatial height and spatial width as the normalized input tensor. The high-dimensional feature vector at each pixel location not only contains the edge gradient information of that location itself, but also integrates the geometric arrangement features of adjacent components propagating along the topological connection relationship of the photovoltaic array. It explicitly encodes the physical boundary position of the component to which the pixel belongs, the grid line direction constraint, and the relative topological position of the component in the entire array grid. This array structure prior feature map will be used as the structural constraint input for the structure-guided cross-attention semantic fusion with the defect texture feature map in step four.

[0038] Specifically, step three is used to capture local pixel-level anomaly representations of crack orientation, hot spot gradient, and contamination dispersion patterns from the normalized input tensor using preset multi-directional gradient-sensitive convolutional kernels at multiple specific angles, in order to obtain a defect texture feature map. It is worth mentioning that the preset multi-directional gradient-sensitive convolutional kernels at multiple specific angles include asymmetric convolutional kernels at four specific angles: 0 degrees, 45 degrees, 90 degrees, and 135 degrees. It is understandable that various types of defects on the surface of photovoltaic modules exhibit significant directional texture differences in images. Cracks appear as narrow, linear structures extending in a specific direction with random orientation. Hot spots appear as radial gradient distributions that gradually diffuse from the high-temperature center outwards. Dirt appears as irregularly covered areas with blurred boundaries and diffused in multiple directions. The texture features of these defects differ fundamentally from the normal surface texture of photovoltaic modules in terms of spatial frequency, directional distribution, and morphological patterns. Traditional square symmetric convolution kernels have equal feature responses in all directions and cannot selectively enhance subtle texture changes in specific directions. This results in the narrow crack signal extending at a specific angle being diluted by the isotropic convolution response. The radial gradient features of hot spots and the diffusion patterns of dirt are also difficult to distinguish effectively. Meanwhile, the structure-aware branch in step two is already specifically responsible for extracting the macroscopic geometric structure information of the photovoltaic array. This step needs to complement it functionally, focusing on extracting pure local pixel-level abnormal texture features stripped of macroscopic structural interference. Therefore, this embodiment of the invention further captures local pixel-level anomaly representations of crack direction, hot spot gradient, and contamination diffusion patterns from the normalized input tensor using multiple pre-set multi-directional gradient-sensitive convolutional kernels at specific angles to obtain a defect texture feature map. This allows for targeted gradient response extraction of texture anomalies in different directions using multiple asymmetric convolutional kernels at specific angles. Furthermore, channel fusion and dimensionality reduction are used to integrate the multi-directional anomaly texture information into a unified defect texture feature representation. This enables the defect texture feature map to possess highly sensitive pixel-level anomaly responses for cracks with different directions, hot spots with different diffusion patterns, and contamination with different coverage morphologies. This provides direction-sensitive and semantically rich texture discrimination input for the subsequent cross-attention semantic fusion with the array structure prior feature map in step four.

[0039] For example, step three includes: step 3.1, using multi-directional gradient-sensitive convolution kernels at multiple specific angles to perform multi-directional asymmetric convolution calculation on the normalized input tensor to obtain a set of multi-directional intermediate feature tensors; step 3.2, performing multi-directional feature channel concatenation on the set of multi-directional intermediate feature tensors to obtain a concatenated high-dimensional texture tensor; step 3.3, using pointwise convolution kernels to perform cross-channel dimensionality reduction projection on the concatenated high-dimensional texture tensor, and sequentially passing it through batch normalization layers and nonlinear activation functions for feature distribution correction and nonlinear recombination to obtain a defect texture feature map.

[0040] Accordingly, step 3.1 is used to perform multi-directional asymmetric convolution calculation on the normalized input tensor using multi-directional gradient-sensitive convolution kernels at multiple specific angles to obtain a set of multi-directional intermediate feature tensors. It should be understood that the texture direction of defects such as cracks, hot spots, and dirt on the surface of photovoltaic modules exhibits significant directional randomness in the image. Cracks may extend horizontally, vertically, or diagonally; the temperature gradient of hot spots decays from the center outwards along different radial directions; and the coverage boundary of dirt also presents an irregular diffuse transition along multiple directions. The aforementioned multi-directional texture anomaly information in the normalized input tensor exists in the numerical distribution of each pixel in a mixed and superimposed form. If a single-directional convolution kernel or a traditional square symmetric convolution kernel is used for feature extraction, the single-directional convolution kernel can only respond to texture changes consistent with its preset direction and completely ignores defect information in other directions. The square symmetric convolution kernel equalizes the response in all directions, thus submerging weak defect signals in specific directions in the isotropic average response. Neither can achieve comprehensive and targeted capture of multi-directional texture anomalies. Therefore, this embodiment of the invention further performs multi-directional asymmetric convolution calculations on the normalized input tensor to obtain a set of multi-directional intermediate feature tensors. Specifically, the preset multi-directional gradient-sensitive convolution kernels at multiple specific angles include asymmetric convolution kernels at four specific angles: 0 degrees, 45 degrees, 90 degrees, and 135 degrees. These four asymmetric convolution kernels are used to independently extract the direction-selective responses of texture gradient changes along the four directions of 0 degrees, 45 degrees, 90 degrees, and 135 degrees in the normalized input tensor. This decouples the mixed and superimposed multi-directional texture anomaly information in the normalized input tensor into gradient response representations of four independent directional channels. This allows crack orientations extending in any direction, hot spot gradients spreading along different radial directions, and contamination boundaries diffused along multiple directions to be captured with high sensitivity by at least one asymmetric convolution kernel, providing directionally complete and complementary intermediate feature inputs for subsequent multi-directional feature channel cascading.

[0041] For example, the implementation process of step 3.1 is as follows: First, four asymmetric convolution kernels are initialized in the texture discrimination branch. The zero-degree asymmetric convolution kernel has a horizontally elongated strip structure as its filter shape, with its weight coefficients arranged along the horizontal axis while occupying only a very small number of pixel rows in the vertical direction. This shape design allows its receptive field to have a large spatial span in the horizontal direction while maintaining a very narrow coverage range in the vertical direction, thereby generating a strong gradient response to the direction of cracks extending in the horizontal direction and the boundary of dirt coverage in the horizontal direction. The ninety-degree asymmetric convolution kernel has a vertically elongated strip structure as its filter shape, with its weight coefficients arranged along the vertical axis, which generates a strong gradient response to the direction of cracks extending in the vertical direction and the boundary of dirt coverage in the horizontal direction. The gradient changes in the orientation and vertical direction of the hotspot generate a strong gradient response. The filter shape of the 45-degree asymmetric convolution kernel is a stepped structure arranged along the upper left to lower right diagonal, which selectively responds to the defect texture extending along this diagonal direction. The filter shape of the 135-degree asymmetric convolution kernel is a stepped structure arranged along the upper right to lower left diagonal, which also selectively responds to the defect texture extending along this diagonal direction. Next, a two-dimensional spatial convolution operation is performed on the normalized input tensor and the zero-degree asymmetric convolution kernel. This convolution kernel slides along the spatial dimension position by position on the normalized input tensor, and at each position, the pixel values ​​within the coverage area of ​​its horizontal strip receptive field are weighted and summed. Adding a corresponding bias term, the texture gradient response value along the horizontal direction at that location is output. After traversing all spatial locations, the zero-degree intermediate feature tensor is obtained. Then, using the same convolution operation method, the normalized input tensor is subjected to independent two-dimensional spatial convolution operations with 45-degree, 90-degree, and 135-degree asymmetric convolution kernels, respectively. Each convolution kernel, based on its own asymmetric filter shape, only produces a strong response to texture gradient changes in a preset direction, resulting in the 45-degree, 90-degree, and 135-degree intermediate feature tensors, respectively. Further, the intermediate feature tensors of the above four directions are combined and... The package forms a set of multi-directional intermediate feature tensors. The four directional tensors in this set have the same spatial resolution as the normalized input tensor. Each directional tensor encodes the texture gradient anomaly response information in the normalized input tensor along the corresponding specific angular direction. The zero-degree directional intermediate feature tensor has a high response to horizontally oriented cracks and horizontally diffused dirt boundaries. The ninety-degree directional intermediate feature tensor has a high response to vertically oriented cracks and vertically oriented hot spot gradients. The forty-five-degree and one-thirty-five-degree directional intermediate feature tensors have high responses to defect textures extending along two diagonal directions, respectively. This set of multi-directional intermediate feature tensors will serve as the input data for the next step of multi-directional feature channel cascading.

[0042] Accordingly, step 3.2 is used to concatenate multi-directional feature channels of the multi-directional intermediate feature tensor set to obtain a concatenated high-dimensional texture tensor. It should be understood that the four directional intermediate feature tensors contained in the multi-directional intermediate feature tensor set obtained in step 3.1 independently encode the texture gradient anomaly response information along four specific angular directions of zero, forty-five, ninety, and one hundred and thirty-five degrees in the normalized input tensor. These four directional tensors are separated from each other in data structure and exist as independent tensor entities. However, the subsequent pointwise convolutional dimensionality reduction operation requires the input to be a unified single tensor so as to perform cross-directional linear projection and information interaction in the channel dimension. If the four directional tensors are input to the subsequent processing layer independently, the texture anomaly information in each direction will always be isolated, and it will be impossible to achieve joint expression and collaborative discrimination between gradient responses in different directions at the same pixel position. For example, when a crack extending along the forty-five-degree direction and a hot spot gradient spreading along the ninety-degree direction exist at a certain pixel position, the separated directional tensors cannot simultaneously present the composite feature pattern of these two anomalies at that position. Therefore, this embodiment of the invention further concatenates multi-directional feature channels of the multi-directional intermediate feature tensor set to obtain a concatenated high-dimensional texture tensor. This integrates the four directional tensors into a unified high-dimensional tensor along the channel dimension, enabling texture gradient response information from different directions at the same pixel location to form a complete multi-directional joint representation along the channel dimension. In this way, without losing any directional information or changing the spatial resolution of each directional tensor, the scattered four-directional texture anomaly responses can be integrated into the continuous channel space of the same tensor. This allows subsequent pointwise convolutional dimensionality reduction operations to simultaneously access the gradient response information of all four directions along the channel dimension and perform cross-directional feature interaction and fusion recombination.

[0043] For example, the implementation process of step 3.2 is as follows: First, extract the zero-degree, forty-five-degree, ninety-degree, and one-thirty-five-degree intermediate feature tensors sequentially from the multi-directional intermediate feature tensor set. Verify that the dimensions of the above four directional tensors are completely consistent in the spatial height and spatial width dimensions to ensure that the subsequent channel concatenation operation is strictly aligned in the spatial dimensions and will not produce dimension mismatch errors. Next, in a fixed order of zero, forty-five, ninety, and one-thirty-five degrees, perform a sequential concatenation operation along the channel dimensions of the tensors, that is, arrange the channel of the forty-five-degree intermediate feature tensor after the channel of the zero-degree intermediate feature tensor, arrange the channel of the ninety-degree intermediate feature tensor after the channel of the forty-five-degree intermediate feature tensor, and arrange the channel of the one-thirty-five-degree intermediate feature tensor after the channel of the ninety-degree intermediate feature tensor. Afterwards, the total number of channels in the resulting tensor after concatenation is equal to the sum of the number of channels in each of the four directional tensors, while the spatial height and spatial width remain unchanged. Then, the concatenated tensor is output as a cascaded high-dimensional texture tensor. In this cascaded high-dimensional texture tensor, the channel vectors at the same pixel position sequentially contain the horizontal texture gradient response along the zero-degree direction, the diagonal texture gradient response from the upper left to the lower right along the forty-five-degree direction, the vertical texture gradient response along the ninety-degree direction, and the diagonal texture gradient response from the upper right to the lower left along the one hundred and thirty-five-degree direction, forming a complete channel cascaded expression of multi-directional texture anomalies at that pixel position. Furthermore, this cascaded high-dimensional texture tensor will be used as input data for cross-channel dimensionality reduction projection using pointwise convolution kernels in the next step. Subsequent pointwise convolutions will perform linear combination operations on the channel dimensions of this tensor, enabling the texture gradient response information originally belonging to different directional channels to undergo cross-directional information interaction and feature recombination during the dimensionality reduction process.

[0044] Accordingly, step 3.3 is used to perform cross-channel dimensionality reduction projection on the cascaded high-dimensional texture tensor using pointwise convolution kernels, and then perform feature distribution correction and nonlinear recombination through batch normalization layers and nonlinear activation functions to obtain the defect texture feature map. It should be understood that the number of channels in the cascaded high-dimensional texture tensor obtained in step 3.2 is the sum of the number of channels in the four intermediate feature tensors. This channel dimension is significantly expanded due to the four-directional concatenation operation, and there is a lot of redundant information and repetitive expressions across directions. For example, the gradient response along the zero-degree direction and along the 135-degree direction at a certain pixel position may produce partially overlapping feature encodings for the same diagonal crack. If the cascaded high-dimensional texture tensor with expanded channel number is directly input into the subsequent cross-attention fusion layer, on the one hand, the redundant channel dimension will cause the computational complexity of attention weight calculation to increase exponentially, and on the other hand, the independent feature expressions between the channels in each direction without information interaction will make it difficult for the attention mechanism to obtain the texture semantics that integrate multi-directional comprehensive discrimination information. At the same time, due to the different filter shapes and weight distributions of the four convolutional kernels, the output gradient response values ​​have significant differences in numerical range and statistical distribution. If this inconsistency in the distribution between channels is not corrected, the gradient propagation of the subsequent network layers will be in an unstable state, and the features after linear dimensionality reduction only have the expressive ability of linear combination, which cannot capture the complex nonlinear interaction patterns between texture anomalies in different directions. Therefore, this embodiment of the invention further utilizes pointwise convolutional kernels to perform cross-channel dimensionality reduction projection on the cascaded high-dimensional texture tensor. The feature distribution is then corrected and nonlinearly recombined using batch normalization layers and nonlinear activation functions to obtain a defect texture feature map. This compresses redundant information in the channel dimension and promotes cross-directional information interaction among texture features in different directions. Simultaneously, it corrects numerical distribution differences between channels and introduces nonlinear expressive power to enhance the discriminative power between defective features and normal background features. In this way, the inflated and redundant multi-directional independent feature representation in the cascaded high-dimensional texture tensor can be transformed into a compact unified texture feature representation that integrates cross-directional comprehensive discriminative information. This ensures that the feature vector at each pixel position in the defect texture feature map simultaneously contains nonlinear fusion semantics of multi-directional texture anomalies, and the numerical distribution of each channel is within a stable and consistent range. This provides a compact, stable, and highly discriminative texture input for the subsequent cross-attention semantic fusion with the array structure prior feature map in step four.

[0045] For example, the implementation process of step 3.3 is as follows: First, a pointwise convolutional kernel is constructed. The spatial size of this convolutional kernel is one-to-one. Its input channel number is equal to the total number of channels of the cascaded high-dimensional texture tensor, that is, the sum of the tensor channels in the four directions. Its output channel number is set to the preset target channel number. The cascaded high-dimensional texture tensor is input into the pointwise convolutional kernel to perform a cross-channel linear projection operation. At each spatial pixel position, the pointwise convolutional kernel performs a weighted linear combination of all gradient response components from the four directions of zero, forty-five, ninety, and one hundred and thirty-five degrees in the channel vector at that position and adds a bias term. Since the spatial size of the pointwise convolutional kernel is one-to-one, this operation does not change the spatial resolution of the tensor but only performs dimensionality reduction and cross-channel compression in the channel dimension. Directional information interaction reduces the number of output channels from the inflated total number of four-directional stitching to a preset target number. During dimensionality reduction, texture gradient response information originally belonging to different directional channels is forcibly linearly fused, so that each output channel after dimensionality reduction no longer corresponds to only a single direction but encodes texture discrimination information after multi-directional comprehensive weighting. Next, the tensor after pointwise convolutional dimensionality reduction is input into a batch normalization layer for feature distribution correction. The batch normalization layer calculates the mean and variance of each output channel on the sample set of the current batch, performs a standardization operation of subtracting the mean and dividing the variance on all pixel values ​​in each channel, and then performs an affine transformation on the standardized values ​​through the learnable scaling and translation parameters built into the layer to eliminate the output of the four-directional convolution kernel. Inconsistent numerical ranges and potential channel-to-channel distribution shifts resulting from pointwise convolutional linear combinations correct the feature value distribution of each channel to a uniform and stable numerical range. Then, the batch-normalized tensors are input into a nonlinear activation function for nonlinear recombination. This function independently performs a nonlinear transformation on each element value in the tensor, suppressing negative response values ​​less than zero while retaining positive response values ​​greater than zero. This operation mathematically introduces piecewise linear nonlinearity, enabling the features after linear dimensionality reduction and distribution correction to capture complex nonlinear interaction patterns between texture anomalies in different directions. Simultaneously, by suppressing negative response values, background noise features unrelated to defects are filtered out, enhancing the distinction between defect areas and normal areas. The discriminative features between regions; furthermore, after the sequential processing of the above three steps of pointwise convolutional dimensionality reduction, batch normalization correction and nonlinear activation recombination, the final defect texture feature map is obtained. The spatial resolution of the defect texture feature map is consistent with the normalized input tensor, and its number of channels is the preset compact target number of channels. The feature vector at each pixel position is fused with the texture gradient anomaly information along four specific angular directions at that position and enhanced by nonlinear recombination. It has a highly sensitive comprehensive discriminative response to cracks with different orientations, hot spots with different diffusion modes and dirt with different coverage forms. This defect texture feature map will be used as the texture discriminative input for the structure-guided cross-attention semantic fusion with the array structure prior feature map in step four.

[0046] Specifically, step four is used to perform structure-guided cross-attention semantic fusion on the array structure prior feature map and the defect texture feature map to obtain a cross-attention fusion feature map. It is understandable that the array structure prior feature map generated in step two and the defect texture feature map generated in step three are extracted from two independent parallel branches. The array structure prior feature map encodes the macroscopic grid arrangement law of the photovoltaic array and the spatial constraint information of the physical boundary of the component. The defect texture feature map encodes the local pixel-level anomaly representation information of crack direction, hot spot gradient and pollution dispersion mode. At the semantic level, the two carry two complementary but isolated feature information: structural constraint and anomaly discrimination. If the two features are simply added pixel by pixel or spliced ​​through channels, this fixed-weight fusion method cannot dynamically adjust the constraint strength of structural information on texture information according to the semantic needs of different spatial locations. For example, strong structural constraints are needed near the physical boundary of the component to suppress false crack responses across the boundary, while structural constraints need to be weakened at the obvious defect location in the central region of the component to retain the original discriminative power of the texture features. The fixed-weight fusion method cannot achieve this spatially adaptive differentiated constraint, which will lead to the problem that the structural constraints in some areas are too strong and suppress the real defect features, while the structural constraints in other areas are insufficient and cannot suppress false detections. Therefore, this embodiment of the invention further performs structure-guided cross-attention semantic fusion on the array structure prior feature map and the defect texture feature map to obtain a cross-attention fusion feature map. This allows for the dynamic calculation of the semantic relevance weight of each position in the structural features to all positions in the texture features within the global space via a cross-attention mechanism. This enables the structural prior information to adaptively adjust the constraint strength and scope of the texture features according to the actual semantic needs of each spatial position. In this way, the cross-attention fusion feature map obtains strong structural constraints at pixel positions near the physical boundaries of the components, effectively suppressing false defect responses across boundaries. Sufficient texture anomaly discrimination information is retained in clearly defined defect regions within the components. This achieves a dynamic balance between macroscopic structural constraints and microscopic texture discrimination within the global space, providing a fusion feature input with both structural localization accuracy and texture discrimination sensitivity for the uncertainty-aware gating and defect semantic mask generation in the subsequent step five.

[0047] For example, step four includes: step 4.1, performing linear mapping and spatial dimension flattening operations on the array structure prior feature map and the defect texture feature map to obtain the query matrix, key matrix and value matrix; step 4.2, calculating the cross-attention weight map between the query matrix and the key matrix; step 4.3, performing weighted aggregation multiplication operations on the value matrix based on the cross-attention weight map and restoring the two-dimensional spatial structure to obtain the structure-guided enhanced texture tensor, and adding the structure-guided enhanced texture tensor and the defect texture feature map pixel by pixel residual to obtain the cross-attention fusion feature map.

[0048] Accordingly, step 4.1 involves linear mapping and spatial dimension flattening operations on the array structure prior feature map and the defect texture feature map to obtain the query matrix, key matrix, and value matrix. It should be understood that both the array structure prior feature map and the defect texture feature map are three-dimensional tensors with three dimensions: spatial height, spatial width, and channel depth. The subsequent cross-attention weight calculation essentially calculates the semantic relevance score between any two pixel positions within the global space. This calculation requires the input data to be presented in a two-dimensional matrix form, with one dimension being the spatial position sequence and the other being the feature vector. The two-dimensional spatial structure of the three-dimensional tensor is incompatible with the two-dimensional sequence structure required by matrix multiplication in terms of dimensionality. Furthermore, the array structure prior feature map and the defect texture feature map originate from the structure perception branch and the texture discrimination branch, respectively. The semantic spaces encoded by the channel dimensions of the features are fundamentally different. The channel space of structural features focuses on geometric arrangement and boundary constraint information, while the channel space of texture features focuses on directional gradient anomaly information. If the two are not projected separately into a unified linear subspace suitable for performing correlation calculation, directly calculating the dot product in the original heterogeneous channel space will not yield a meaningful semantic correlation metric. Furthermore, the cross-attention mechanism requires the query signal to originate from structural features to guide and constrain the texture, while the key and value signals originate from texture features as the objects to be constrained and aggregated. This heterogeneous mapping relationship needs to be established through a parameter-independent linear projection layer. Therefore, this embodiment of the invention further performs linear mapping and spatial dimension flattening operations on the array structure prior feature map and the defect texture feature map to obtain the query matrix, key matrix, and value matrix. This projects the two heterogeneous features separately into a linear subspace suitable for performing cross-attention correlation calculation, and flattens the spatial dimension of the three-dimensional tensor into a one-dimensional sequence to meet the dimensional requirements of subsequent matrix dot product operations. In this way, each row of the query matrix encodes the structural query vector after linear transformation of the corresponding spatial position in the prior feature map of the array structure, and each row of the key matrix and value matrix encodes the texture key vector and texture value vector after different linear transformations of the corresponding spatial position in the defect texture feature map. The three of them fully meet the input requirements of subsequent cross-attention weight map calculation and weighted aggregation operation in terms of dimensionality and form. Moreover, the heterogeneous mapping relationship of query originating from structure and key value originating from texture ensures the structural guidance semantics of subsequent attention calculation.

[0049] For example, the implementation process of step 4.1 is as follows: First, three sets of pointwise convolutional kernels with independent parameters are constructed, denoted as query projection layer, key projection layer, and value projection layer, respectively. The spatial size of the three sets of pointwise convolutional kernels is one-to-one, and each has independent learnable weight parameters and bias parameters. The number of input channels of the query projection layer is equal to the number of channels of the array structure prior feature map, and the number of input channels of the key projection layer and value projection layer is equal to the number of channels of the defect texture feature map. The number of output channels of the three projection layers is set to the preset attention feature dimension. The array structure prior feature map is input into the query projection layer to perform pointwise convolution operation. This operation performs a linear transformation on the channel vector at each spatial pixel position of the array structure prior feature map. The calculation process is expressed as follows: ;in, This is the 3D query tensor after query projection; To query the pointwise convolution weight parameter matrix of the projection layer; This is the spatial convolution operator; The prior feature map of the input array structure; To query the bias parameter vector of the projection layer. In this formula, It is responsible for mapping the structural feature channel space, which encodes the photovoltaic array grid arrangement rules and component physical boundary constraints in the prior feature map of the array structure, to a linear subspace suitable for performing correlation queries. Macroscopic structural semantic information at each spatial location is provided as the source of the query signal, ensuring that the query direction in subsequent attention calculations is dominated by structural priors. Then, the defect texture feature map is input into the key projection layer and value projection layer respectively to perform pointwise convolution operations, the calculation processes of which are expressed as follows: ; ; in, This is the three-dimensional bond tensor after bond projection; It is the three-dimensional value tensor after longitude projection; and These are the pointwise convolutional weight parameter matrices for the key projection layer and the value projection layer, respectively; The input is the defect texture feature map; and These are the bias parameter vectors for the bond projection layer and the value projection layer, respectively. In the two formulas above, and Using different learnable weight parameters, It is responsible for mapping multi-directional texture anomaly information in the defect texture feature map to a key subspace suitable for correlation matching with the query vector. The task is to map the texture anomaly information in the defect texture feature map to a value subspace suitable for weighted aggregation by attention weights. Although both are based on the defect texture feature map... The input is used, but the same texture information is projected into two linear subspaces with different functions through different weight parameters. The key subspace is used to measure the semantic matching degree between texture positions and structural positions, and the value subspace is used to carry the texture feature content that is finally aggregated and transmitted. Then, the three-dimensional query tensor, three-dimensional key tensor, and three-dimensional value tensor are flattened along the spatial height and spatial width dimensions, respectively. That is, the feature vectors of the spatial height multiplied by the spatial width of each tensor are arranged into a one-dimensional sequence in the order of row scanning, so that the three-dimensional tensor is transformed from the form of channel multiplied by height multiplied by width into a two-dimensional matrix form of the total number of spatial positions multiplied by the attention feature dimension, respectively, to obtain the query matrix, key matrix, and value matrix. The number of rows of the query matrix is ​​equal to the array structure. The total number of spatial pixels in the prior feature map, the number of rows in the key matrix and value matrix are equal to the total number of spatial pixels in the defect texture feature map, and the number of columns in all three are equal to the preset attention feature dimension. Furthermore, after the above linear mapping and spatial dimension flattening operations, each row in the query matrix encodes the structure query vector at the corresponding spatial position in the array structure prior feature map, each row in the key matrix encodes the texture key vector at the corresponding spatial position in the defect texture feature map, and each row in the value matrix encodes the texture value vector at the corresponding spatial position in the defect texture feature map. All three are presented in two-dimensional matrix form and the dimensions fully meet the requirements of the subsequent matrix dot product operation. The query matrix, key matrix, and value matrix will be used as input data for the next step of calculating the cross-attention weight map.

[0050] Accordingly, step 4.2 is used to calculate the cross-attention weight graph between the query matrix and the key matrix. It should be understood that the query matrix obtained in step 4.1 encodes the structural query vectors of each spatial location in the prior feature map of the array structure, and the key matrix encodes the texture key vectors of each spatial location in the defect texture feature map. The two are respectively located in the structural query subspace and the texture key subspace. The subsequent weighted aggregation operation needs a clear weight allocation basis to determine which texture locations in the defect texture feature map each structural location in the prior feature map of the array structure should pay attention to and with what intensity feature aggregation. If this weight allocation basis is lacking, the weighted aggregation operation will not be able to distinguish the differences in structural constraints between texture locations within the same component and texture locations in adjacent components, resulting in the loss of spatial selectivity in structural guidance. The vector dot product between the query matrix and the key matrix can precisely measure the degree of matching between the structural query vector and the texture key vector in the semantic space. The larger the dot product value, the closer the semantic association between the corresponding structural location and the texture location. However, the numerical range of the original dot product value increases linearly with the increase of the feature dimension. If no scaling processing is performed, the input value of the subsequent normalization operation will be in an extreme range, causing the normalized probability distribution to tend to a sharp one-hot distribution and lose the ability to flexibly pay attention to multiple related texture locations. Therefore, this embodiment of the invention further calculates a cross-attention weight map between the query matrix and the key matrix to quantify the semantic relevance strength between each structural position in the prior feature map of the array structure and all texture positions in the defect texture feature map in the global space, and normalizes the relevance score into an attention weight in the form of a probability distribution. This allows each row of the probability distribution in the cross-attention weight map to accurately describe the attention allocation pattern of the corresponding structural position to all texture positions. At structural positions near the physical boundary of the component, attention weights are concentrated on texture positions within the same component, while extremely low weights are assigned to texture positions of adjacent components. This provides a spatially adaptive structural constraint basis for subsequent weighted aggregation of the value matrix based on this weight map.

[0051] For example, the implementation process of step 4.2 is as follows: First, perform a matrix transpose operation on the key matrix, which swaps its rows and columns so that the number of rows of the transposed key matrix is ​​equal to the attention feature dimension and the number of columns is equal to the total number of spatial pixels of the defect texture feature map. This transpose operation is to make the query matrix and the transposed key matrix meet the inner dimension matching requirements of matrix multiplication in terms of dimension. Next, the matrix dot product between the query matrix and the transposed key matrix is ​​calculated to obtain the original relevance score matrix. The number of rows in this matrix equals the total number of spatial pixels in the array structure prior feature map, and the number of columns equals the total number of spatial pixels in the defect texture feature map. The element in the i-th row and j-th column represents the relevance score of the inner product between the structure query vector at the i-th spatial position in the array structure prior feature map and the texture key vector at the j-th spatial position in the defect texture feature map. To prevent this score from excessively expanding with the increase of the attention feature dimension, the square root of the attention feature dimension is used as a scaling factor to perform a division scaling operation on each element in the original relevance score matrix. Then, a normalized exponential function operation is performed along the row dimension of the scaled relevance score matrix, mapping the real-valued values ​​in each row to a probability distribution whose sum is strictly equal to one, resulting in the cross-attention weight map. Its calculation process is expressed as follows: ;in, The attention weight value in the i-th row and j-th column of the cross-attention weight map represents the intensity of attention allocation from the i-th spatial location in the prior feature map of the array structure to the j-th spatial location in the defect texture feature map. For the structure query vector in the i-th row of the query matrix, the component boundary and topological arrangement information at the i-th pixel position in the prior feature map of the array structure are encoded. The transpose of the texture key vector in the j-th row of the key matrix encodes the multi-directional texture anomaly information at the j-th pixel position in the defect texture feature map; The transpose of the texture key vector in the nth row of the key matrix encodes the multi-directional texture anomaly information at the nth pixel position in the defect texture feature map; The vector inner product operation between the two measures the degree of matching between the i-th structural position and the j-th texture position in the semantic space. The attention feature dimension, i.e., the number of columns in the query matrix and the key matrix, is used as a scaling factor to suppress the numerical effect of excessive expansion of the inner product value as the feature dimension increases, and to keep the input value of the normalization operation within a reasonable numerical range. The function is an exponential function; N is the total number of spatial pixels in the defect texture feature map; the denominator is the sum of the scaling inner product exponents of all N spatial locations in the defect texture feature map, ensuring that the sum of all attention weights in the i-th row is strictly equal to one. In this formula, the numerator is... The scaling semantic relevance score between the i-th structural location and the j-th texture location is converted into a positive exponential response; the stronger the relevance, the larger the exponential value. The denominator is normalized by summing the exponential responses of all texture locations, so that texture locations with strong semantic association with the i-th structural location receive higher attention weights, while texture locations with weak semantic association receive near-zero attention weights. When the i-th structural location is near the physical boundary of a photovoltaic module, the inner product between the boundary constraint information encoded in its structural query vector and the texture key vector of the texture location within the same module is significantly higher than the product between the i-th structural location and the texture key vector of the texture location within the same module. The inner product value, after normalization, naturally concentrates the attention weights on the texture positions within the same component, thus realizing the dynamic constraint of the structural boundary on the propagation range of the texture feature space at the weight level. Furthermore, the cross-attention weight map obtained after the above scaling dot product and normalization operations is a probability distribution matrix with the number of rows equal to the total number of pixels in the array structure prior feature map space and the number of columns equal to the total number of pixels in the defect texture feature map space. This matrix completely encodes the attention allocation pattern of each spatial position in the structural feature to all spatial positions in the texture feature, and will serve as the weight basis for the weighted aggregation multiplication operation of the value matrix in the next step.

[0052] Accordingly, step 4.3 is used to perform weighted aggregation multiplication on the value matrix based on the cross-attention weight map and restore the two-dimensional spatial structure to obtain the structure-guided enhanced texture tensor, and to add the structure-guided enhanced texture tensor to the defect texture feature map pixel by pixel residual to obtain the cross-attention fusion feature map. It should be understood that the cross-attention weight map obtained in step 4.2 has quantified the attention allocation intensity of each structural position in the array structure prior feature map to all texture positions in the defect texture feature map in the global space. However, the weight map itself is only a probability distribution matrix and has not yet applied the structural constraint information it encodes to the actual numerical content of the texture features. It is necessary to use the weight map to perform weighted aggregation operation on the value matrix that carries the actual numerical content of the texture features in order to truly inject the spatial constraints guided by the structure into the texture feature representation. At the same time, the result after weighted aggregation is still in the form of a two-dimensional matrix of a one-dimensional sequence. The uncertainty perception gating operation in the subsequent step five requires the input data to have a two-dimensional spatial structure with spatial height and spatial width. Therefore, it is necessary to restore the aggregation result to the form of a three-dimensional tensor. In addition, although the global weighted aggregation operation of cross-attention introduces structural constraint information, the low-level texture details in the original defect texture feature map may be partially smoothed or diluted due to the global weighted averaging during the aggregation process. If only the aggregated result is used as the output, it will lead to the loss of key low-level information such as the sharp texture features of the crack edge and the high gradient response of the hot spot center. Therefore, this embodiment of the invention further performs weighted aggregation multiplication on the value matrix based on the cross-attention weight map to restore the two-dimensional spatial structure and obtain the structure-guided enhanced texture tensor. The structure-guided enhanced texture tensor is then added pixel-by-pixel residually to the defect texture feature map to obtain a cross-attention fusion feature map. This allows the structural constraint information encoded by the cross-attention weight map to be actually injected into the numerical representation of the texture features through weighted aggregation. Furthermore, the residual connection mechanism preserves the underlying texture discrimination details in the original defect texture feature map while superimposing the structural constraint enhancement information. In this way, the cross-attention fusion feature map simultaneously contains global texture aggregation information constrained by structural boundaries and the original underlying texture anomaly discrimination information at each pixel location. This effectively suppresses false defect responses across boundaries near the component's physical boundaries while preserving complete texture discrimination sensitivity in the defect region within the component. This provides the feature input for the deep fusion of structural constraints and texture discrimination for uncertainty-aware gating and defect semantic mask generation in the subsequent step five.

[0053] For example, the implementation process of step 4.3 is as follows: First, perform matrix multiplication on the cross-attention weight map and the value matrix. The number of rows in the cross-attention weight map is equal to the total number of spatial pixels in the array structure prior feature map, and the number of columns is equal to the total number of spatial pixels in the defect texture feature map. The number of rows in the value matrix is ​​equal to the total number of spatial pixels in the defect texture feature map, and the number of columns is equal to the attention feature dimension. After multiplying the two, an aggregated feature matrix is ​​obtained. The number of rows in this matrix is ​​equal to the total number of spatial pixels in the array structure prior feature map, and the number of columns is equal to the attention feature dimension. The feature vector in the i-th row of the matrix is ​​equal to the result of weighted summation of the texture value vectors of all spatial positions in the defect texture feature map according to the probability distribution of the i-th row of the cross-attention weight map. That is, the value vectors of texture positions that are semantically closely related to the i-th structural position receive higher aggregate weights, while the value vectors of texture positions that are semantically weakly related receive lower aggregate weights. A near-zero aggregation weight is obtained, thereby achieving selective aggregation and constrained redistribution of texture features in the global space by the structural prior. Next, the aggregated feature matrix undergoes a dimension reshaping operation according to its original spatial height and width, unfolding the row dimension of the one-dimensional sequence into a two-dimensional spatial grid structure. This restores the aggregated feature matrix from a two-dimensional matrix of total spatial pixels multiplied by the attention feature dimension to a three-dimensional tensor of the attention feature dimension multiplied by the spatial height and spatial width, resulting in the structure-guided enhanced texture tensor. This tensor has the same spatial resolution as the defect texture feature map, where the feature vector at each pixel location has undergone global constraints and selective aggregation by the array structure prior. Then, the structure-guided enhanced texture tensor and the defect texture feature map are summed pixel-by-pixel residuals at corresponding spatial locations. The calculation process is expressed as follows: ; in, For cross-attention fusion feature maps in spatial coordinates The eigenvector at that location; This is the original texture feature vector of the defect texture feature map at spatial coordinates (i,j); This is a dimension reshaping operation function that restores a two-dimensional matrix to a three-dimensional spatial tensor form. This is a cross-attention weight graph; It is a value matrix; To perform matrix multiplication between the cross-attention weight map and the value matrix, a global weighted aggregation of texture features based on structural constraints is implemented. Let be the eigenvector of the structure-guided enhanced texture tensor at spatial coordinates (i,j); in this formula, As a residual base, it retains the original low-level texture discrimination information at this location in the defect texture feature map, including the sharp gradient response of the crack edge, the high-intensity abnormal response of the hot spot center, and the diffuse texture pattern of the dirt-covered area, as well as other original detailed features that have not been globally aggregated and smoothed. Partially, structural guidance enhancement information, aggregated from global texture locations and constrained by the cross-attention weight map, is superimposed at this position. The pixel-by-pixel addition of these two elements ensures that each pixel location in the cross-attention fusion feature map simultaneously possesses the local sensitivity of the original texture discrimination and the global consistency of structural constraint aggregation. When a pixel location is near the component's physical boundary... The aggregated texture information has been confined to the same component by the cross-attention weight map and superimposed onto it. This effectively strengthens the structural constraints of the boundary region, especially when a pixel is located within a clearly defined defect area inside the component. The original high-intensity anomaly response is fully preserved through residual connections, ensuring that the defect discrimination sensitivity is not attenuated by the global aggregation operation. Furthermore, after the above weighted aggregation, dimension reshaping and residual addition operations, a cross-attention fusion feature map is finally obtained. This cross-attention fusion feature map deeply fuses the global texture aggregation information constrained by the array structure boundary and the original low-level texture anomaly discrimination information at each pixel position, realizing a dynamic balance between macroscopic structural constraints and microscopic texture discrimination at the pixel level. This cross-attention fusion feature map will be used as the input data for uncertainty perception gating and defect semantic mask generation in step five.

[0054] Specifically, step five is used to perform uncertainty-aware gating and defect semantic mask generation on the cross-attention fusion feature map to obtain multi-class defect semantic masks. It is understandable that although the cross-attention fusion feature map obtained in step four has deeply fused the prior spatial constraint information of the array structure and the anomaly discrimination information of defect texture, there are still a large number of easily confused regions with high classification ambiguity in the actual inspection scenario of photovoltaic arrays. For example, cracks and module grid lines both appear as narrow linear structures in terms of visual morphology. The edge gradient region of hot spots and the coverage boundary of dirt are highly similar in color gradient. The shadow region of module gaps and the local dirt on the module surface are difficult to distinguish in terms of grayscale distribution. The feature vectors of the above-mentioned easily confused regions in the cross-attention fusion feature map are located at the intersection of multiple class decision boundaries in the class discrimination space. If classification decoding is performed directly on all pixel positions without distinction, the network will process high-deterministic regions and high-ambiguity regions with the same confidence. This makes the classification results of easily confused regions extremely susceptible to the influence of small feature perturbations and produce incorrect judgments. These easily confused regions are precisely the key bottlenecks that determine the overall segmentation accuracy. At the same time, as a high-dimensional continuous feature representation, the cross-attention fusion feature map needs to be converted into a defect segmentation result in the form of discrete category labels that can be directly used by the photovoltaic power plant operation and maintenance system after classification decoding operation. Therefore, this embodiment of the invention further applies uncertainty-aware gating and defect semantic mask generation to the cross-attention fusion feature map to obtain a multi-class defect semantic mask. This allows for the automatic identification of easily confused regions with high classification ambiguity in the cross-attention fusion feature map through pixel-by-pixel information entropy quantization. Uncertainty weights are used as a spatial gating mechanism to adaptively scale and enhance the feature responses of these regions. Finally, the enhanced features are decoded into discrete pixel-level multi-class defect labels via a fully convolutional integral classifier. This enables the network to specifically strengthen the feature discrimination power of easily confused regions before classification decoding, reducing the probability of misclassification between easily confused category pairs such as cracks and grid lines, hot spots and dirt, and gap shadows and surface dirt. The final output multi-class defect semantic mask labels each pixel with a clear defect category label, which can be directly used for defect quantification statistics and alarm decision-making in photovoltaic power plant operation and maintenance systems.

[0055] For example, step five includes: performing uncertainty quantization on the cross-attention fusion feature map by calculating Shannon entropy pixel by pixel to obtain an uncertainty weight map; using the uncertainty weight map as a spatial gate to perform feature scaling enhancement on easily confused regions in the cross-attention fusion feature map, and decoding pixel by pixel using a full-convolution integral class head to obtain a multi-class defect semantic mask.

[0056] Specifically, for example, the process in step five is as follows: Figure 4As shown, the process includes steps 5.1 to 5.3. Step 5.1 involves estimating the initial pixel-level probability distribution of the cross-attention fusion feature map to obtain an initial segmentation probability tensor. Step 5.2 involves quantizing the initial segmentation probability tensor with information entropy to obtain an uncertainty weight map, and then scaling the cross-attention fusion feature map based on the uncertainty weight map to obtain a gated enhanced fusion feature map. Step 5.3 involves performing full-convolution integral class decoding on the gated enhanced fusion feature map and performing maximum value index decoding operation pixel by pixel along the channel dimension to obtain a multi-class defect semantic mask.

[0057] Accordingly, step 5.1 is used to estimate the initial pixel-level probability distribution of the cross-attention fusion feature map to obtain the initial segmentation probability tensor. It should be understood that the feature vectors at each pixel position in the cross-attention fusion feature map obtained in step four are in a high-dimensional continuous feature space. Their values ​​represent abstract feature codes that integrate structural constraints and texture discrimination information, and are not directly applicable to uncertainty quantification. The subsequent information entropy quantization operation requires the input data to have a clear class probability distribution at each pixel position, i.e., the predicted probability values ​​for each class are non-negative and their sum is strictly equal to one. If the high-dimensional feature vector is not first converted into a class probability distribution that conforms to probability axioms, the calculation of information entropy will lose statistical significance and cannot accurately measure the degree of classification uncertainty at each pixel position. Furthermore, the number of channels in the cross-attention fusion feature map is the hidden layer feature dimension set in the cross-attention fusion stage, which is inconsistent with the actual total number of classes in the photovoltaic array semantic segmentation. It is necessary to convert the hidden layer feature dimension into a classification response dimension corresponding to the total number of classes through a linear mapping of the channel dimension in order to establish independent response channels for each class in the channel dimension. Therefore, this embodiment of the invention further performs initial pixel-level probability distribution estimation on the cross-attention fusion feature map to obtain an initial segmentation probability tensor. This transforms the high-dimensional continuous feature vectors at each pixel location in the cross-attention fusion feature map into a multi-class prediction probability distribution that conforms to probability axioms, providing a legitimate probability input for subsequent uncertainty quantification based on Shannon information entropy. In this way, the initial segmentation probability tensor can initially reflect the prediction confidence of each pixel location belonging to categories such as background, normal components, cracks, hot spots, and dirt in the form of a probability distribution. Pixel locations with probability distributions concentrated in a single category indicate high classification certainty, while pixel locations with probability distributions dispersed among multiple categories indicate classification ambiguity, providing a numerical basis for subsequent information entropy quantification to accurately identify easily confused regions.

[0058] For example, the implementation process of step 5.1 is as follows: First, a lightweight pointwise convolutional layer is constructed. The spatial size of this pointwise convolutional layer is one-to-one. The number of its input channels is equal to the hidden layer feature dimension of the cross-attention fusion feature map, and the number of its output channels is equal to the total number of preset photovoltaic array semantic segmentation categories, including independent categories such as background, normal components, cracks, hot spots, and dirt. The cross-attention fusion feature map is input into this pointwise convolutional layer to perform a pixel-wise linear projection operation. At each spatial pixel position, a matrix multiplication is performed between its high-dimensional feature vector and the weight parameter matrix of the pointwise convolutional layer, and a bias parameter is added. The process involves mapping the hidden layer feature dimension to real-domain classification response values ​​in the dimension of the total number of categories, resulting in the original classification response tensor. This tensor contains a number of real-domain response values ​​at each pixel location equal to the total number of categories. The absolute magnitude of each response value reflects the initial tendency for that location to belong to the corresponding category. However, at this point, the numerical range of each response value is unrestricted; it may be positive or negative, and the sum of the response values ​​for each category is not equal to one, thus not yet satisfying the basic axioms of probability distribution. Next, a normalized exponential function operation is performed pixel-by-pixel on the channel dimension of the original classification response tensor. This operation applies to each pixel... For each category's real-domain response value at a given location, an exponential function value is calculated. Then, each category's exponential value is divided by the sum of all category exponential values ​​at that pixel location. This maps the real-domain classification response value to a non-negative probability value between zero and one, with the sum of all category probabilities strictly equal to one, thus obtaining the initial segmentation probability tensor. Next, the data dimension and numerical properties of the initial segmentation probability tensor are verified to ensure its spatial resolution is consistent with the cross-attention fusion feature map, the number of channels equals the total number of categories, and the probability value of each channel at each pixel location is non-negative and the sum along the channel dimension is strictly equal to one. Further, in In this initial segmentation probability tensor, the probability distribution at pixel locations in clearly defective areas inside the photovoltaic module is highly concentrated in the corresponding defect category channel, while the probability distribution at pixel locations on the surface of normal modules is highly concentrated in the normal module category channel. However, the probability distribution at easily confused locations such as the junction of cracks and grid lines, areas with similar colors of hot spots and dirt, and areas where the gray levels of module gap shadows and surface dirt are similar, are dispersed across multiple category channels. This initial segmentation probability tensor will serve as the input data for the next step of information entropy quantization, used to calculate the classification uncertainty at each location pixel by pixel.

[0059] Accordingly, step 5.2 is used to quantize the initial segmentation probability tensor with information entropy to obtain an uncertainty weight map, and to scale the cross-attention fusion feature map based on the uncertainty weight map to obtain a gated enhanced fusion feature map. It should be understood that the initial segmentation probability tensor obtained in step 5.1 has already initially reflected the prediction confidence of each pixel location belonging to a specific category in the form of a probability distribution. Pixel locations where the probability distribution is dispersed across multiple categories are easily confused locations such as the intersection of cracks and grid lines, areas with similar colors of hot spots and dirt, and areas where the gray levels of component gap shadows and surface dirt are similar. However, the initial segmentation probability tensor itself is only a set of probability values ​​and has not yet transformed the degree of classification ambiguity at each pixel location into a scalar measure that can be directly used for feature manipulation. An information-theoretic quantification method is needed to condense the dispersion of the probability distribution into a single, undefined value. While the feature vectors of the aforementioned easily confused regions in the cross-attention fusion feature map have incorporated structural constraints and texture discrimination information, their feature response amplitudes are at similar levels to those of the highly deterministic regions. Subsequent full-convolution integral classifiers will treat both highly deterministic and highly ambiguous regions with the same discrimination strength during pixel-by-pixel classification decoding, failing to allocate additional discrimination resources to the critical easily confused regions that determine the overall segmentation accuracy. Without targeted amplitude enhancement of the feature responses in these easily confused regions before classification decoding, the classification results in these regions will be highly susceptible to minor feature perturbations, leading to incorrect judgments. Therefore, this embodiment of the invention further quantizes the initial segmentation probability tensor with information entropy to obtain an uncertainty weight map, and performs uncertainty feature scaling on the cross-attention fusion feature map based on the uncertainty weight map to obtain a gated enhanced fusion feature map. This utilizes Shannon information entropy to quantify the dispersion of the probability distribution at each pixel location into a scalar uncertainty measure, and uses this measure as a spatial gating mechanism to adaptively scale and enhance the feature responses of highly ambiguous regions in the cross-attention fusion feature map. In this way, the feature response amplitude of easily confused regions in the gated enhanced fusion feature map can be significantly improved, prompting the subsequent full-convolution integral classifier to exert stronger discrimination power on key ambiguous positions such as the intersection of cracks and grid lines, and similar regions of hot spots and dirt. At the same time, the feature response of high-determinism regions remains basically unchanged to avoid unnecessary interference to existing high-confidence discrimination results, thus realizing differentiated feature control of pixel positions with different degrees of determination before classification and decoding.

[0060] For example, the implementation process of step 5.2 is as follows: First, based on the Shannon information entropy principle, the initial segmentation probability tensor is subjected to pixel-by-pixel information entropy quantization calculation. At each spatial pixel location, the information entropy value of the probability distribution at that location is calculated along the category channel dimension. The calculation process is expressed as follows: ; in, For the uncertainty weighting graph in spatial coordinates The information entropy value at that location; The total number of preset semantic segmentation categories for photovoltaic arrays, including independent categories such as background, normal components, cracks, hot spots, and dirt; The initial segmentation probability tensor in spatial coordinates The predicted probability value that the location belongs to class c; It is the natural logarithm function; A minimal positive floating-point constant is introduced to prevent the input of the natural logarithm function from being zero. In this formula, the summation symbol... The contribution of the c-th class probability value to the overall uncertainty of the pixel location was partially calculated. The leading negative sign converts the summation result into a non-negative entropy value. When a pixel location is within a clearly defined crack region inside the photovoltaic module, the probability distribution of that location in the initial segmentation probability tensor is highly concentrated in the crack category channel. The probability value of the crack category approaches one, while the probability values ​​of other categories approach zero. The absolute values ​​of all values ​​approach zero, and the information entropy value... Approaching the minimum value indicates high classification certainty at that location. When a pixel is located at the boundary between a crack and a gate line, the probability distribution of that location in the initial segmentation probability tensor shows an approximately uniform dispersion between the crack category and the normal component category, with each component contributing significantly. The information entropy value is also high. Approaching the maximum value indicates high classification ambiguity at that location. After traversing all spatial pixel locations, an uncertainty weight map is generated. This uncertainty weight map accurately identifies the classification uncertainty of each pixel location in the cross-attention fusion feature map through the spatial distribution of information entropy values. Next, the uncertainty weight map is input into a normalized activation function for numerical domain mapping, mapping the information entropy values ​​from their original non-negative real number domain to a closed interval of zero to one, resulting in a normalized gating coefficient map. Pixel locations with higher information entropy that are easily confused receive gating coefficients closer to one, while pixel locations with lower information entropy and higher certainty receive gating coefficients closer to zero. Subsequently, the normalized gating coefficient map is spatially broadcast along the channel dimension, expanding it from a single-channel two-dimensional spatial map into a three-dimensional tensor with the same number of channels as the cross-attention fusion feature map, to meet the dimension matching requirements of subsequent element-wise multiplication operations. Then, element-wise multiplication operations are performed on the cross-attention fusion feature map using the broadcast gating coefficient tensor, and the multiplication result is added to the cross-attention fusion feature map using pixel-wise residuals. The calculation process is expressed as follows: ; in, To enhance the fusion of gating feature maps in spatial coordinates The eigenvector at that location; is the feature vector of the cross-attention fusion feature map at spatial coordinates (i,j); ⊙ is the element-wise multiplication operator; The normalized activation function maps the information entropy value to the gating coefficient range of zero to one. This represents the information entropy value of the uncertainty weighting graph at spatial coordinates (i,j); in this formula, the left side of the plus sign... As a residual basis, it fully preserves the original feature vector at this location in the cross-attention fusion feature map, which incorporates structural constraints and texture discrimination information, ensuring that gating operations do not lead to the loss of original feature information. (The right side of the plus sign...) Some of the feature increments are driven by uncertainty. Convert the information entropy value at this location into a gating scaling factor between zero and one. This applies when the location is in an ambiguity region at the intersection of the crack and the grid line. Approaching one Approximately equal to The characteristic response amplitude at this location is approximately doubled after being superimposed onto the residual basis, significantly enhancing the feature discrimination power of this easily confused location. This is especially true when the location is within a highly deterministic region, such as a clearly defined defect area within the component. Approaching zero Approximately zero vectors are superimposed, and the feature response amplitude at this position remains essentially unchanged, avoiding unnecessary perturbation to the existing high-confidence feature representation. Furthermore, after the above information entropy quantization and uncertainty feature scaling operations, a gated enhanced fusion feature map is finally obtained. This gated enhanced fusion feature map has the same spatial resolution and channel dimension as the cross-attention fusion feature map. The feature response amplitude of easily confused regions has been adaptively enhanced, while the feature response amplitude of high-determinism regions remains stable. This gated enhanced fusion feature map will serve as the input data for the next step of full-convolution integral class decoding and maximum value index decoding operations.

[0061] Accordingly, step 5.3 is used to perform full-convolution integral decoding on the gated enhanced fusion feature map and perform maximum value index decoding operation pixel by pixel along the channel dimension to obtain multi-class defect semantic mask. It should be understood that although the gated enhanced fusion feature map obtained in step 5.2 has completed the adaptive enhancement of the feature response amplitude in the easily confused region, its data form is still a high-dimensional continuous feature tensor. The feature vector at each pixel position is in the hidden feature space, which is not a discrete category label form that the photovoltaic power plant operation and maintenance system can directly interpret and use. The defect area quantification and statistics module of the operation and maintenance system requires the input data to be labeled with a unique integer label at each pixel position to indicate the specific defect category to which the position belongs. The defect severity rating module requires quantitative calculation based on the pixel coverage area of ​​each category of defects. The operation and maintenance alarm decision module requires to determine whether to trigger the corresponding alarm threshold based on the discrete category label. None of the above downstream operation and maintenance modules can directly process continuous high-dimensional feature vectors. At the same time, the number of channels of the gated enhanced fusion feature map is the hidden feature dimension, which is inconsistent with the actual total number of categories in the semantic segmentation of the photovoltaic array. It is necessary to map the hidden feature dimension to the classification response dimension corresponding to the total number of categories through the full-convolution integral class head, and on this basis, convert the continuous classification response value into an exclusive single category label through the discretization decision operation. Therefore, this embodiment of the invention further performs full-convolution integral class decoding and pixel-by-pixel maximum value index decoding along the channel dimension on the gated enhanced fusion feature map to obtain a multi-class defect semantic mask. This maps the high-dimensional continuous feature vectors at each pixel position in the gated enhanced fusion feature map to classification response values ​​in the category dimension via a full-convolution integral class header, and then completely decodes the continuous response values ​​into discrete, exclusive category integer labels through the maximum value index operation. In this way, the multi-class defect semantic mask can uniquely label the specific category to which each pixel belongs with a discrete integer label, including background, normal components, cracks, hot spots, and dirt, forming a final pixel-level segmentation result that can be directly used for defect area quantification statistics, defect severity rating, and operation and maintenance alarm decision-making in photovoltaic power plant operation and maintenance systems.

[0062] For example, the implementation process of step 5.3 is as follows: First, the gated enhanced fusion feature map is input into the fully convolutional integral class head. This fully convolutional integral class head consists of one or more cascaded convolutional layers. The number of output channels of the convolutional kernel of its final output layer is equal to the preset total number of semantic segmentation categories for photovoltaic arrays, including independent categories such as background, normal components, cracks, hot spots, and dirt. The fully convolutional integral class head performs layer-by-layer convolution and nonlinear transformation operations on its feature vector at each spatial pixel position of the gated enhanced fusion feature map, gradually converting the abstract feature encoding of the hidden layer feature dimension into a classification response value of the total number of categories. All operations of the fully convolutional integral class head are implemented based on convolutional operations without fully connected layers. It can handle input feature maps of arbitrary spatial resolution while maintaining the same spatial resolution as the input. After processing by the fully convolutional integral class head, the final classification response tensor is obtained. This tensor contains a number of real-domain classification response values ​​at each spatial pixel location, equal to the total number of categories. The relative magnitude of each response value reflects the final discriminative tendency for that location to belong to the corresponding category. Since the feature response amplitudes of easily confused regions in the gated enhancement fusion feature map have been adaptively enhanced in the previous step, these are mapped by the fully convolutional integral class head. The differences in response values ​​between different categories in the region are also amplified accordingly, which is beneficial for the subsequent maximum index operation to make more explicit category decisions in these key regions. Next, the maximum index decoding operation is performed pixel by pixel on the channel dimension of the final classification response tensor. That is, at each spatial pixel position, the classification response values ​​of all category channels at that position are traversed, and the category index number corresponding to the response value with the largest value is selected as the final classification label of that pixel position. This operation completely converts the continuous multi-channel real number domain response values ​​into single-channel discrete integer labels. Only one unique category integer index is retained at each pixel position, realizing an irreversible decision mapping from continuous feature space to discrete label space. Then, the discrete category integer labels of all spatial pixel positions are organized into a single-channel two-dimensional integer matrix with the same spatial height and spatial width as the gated enhancement fusion feature map. Combined with the preset photovoltaic array defect category index dictionary, a mapping relationship is established between each integer label and its corresponding semantic category name. Index 0 corresponds to the background category, index 1 corresponds to the normal component category, index 2 corresponds to the crack category, index 3 corresponds to the hot spot category, and index 4 corresponds to the dirt category, thereby generating a multi-category defect semantic mask.Furthermore, this multi-category defect semantic mask is used as the final output of the entire photovoltaic array inspection method based on UAV image semantic segmentation. At each pixel location, a discrete integer label clearly indicates the specific defect category. The photovoltaic power plant operation and maintenance system can directly use this mask to statistically analyze the pixel coverage area and spatial distribution of each type of defect, calculate quantitative indicators such as crack length and direction, hot spot area and temperature level, and pollution coverage ratio, and determine whether to trigger the corresponding level of operation and maintenance alarm and dispatch a repair work order based on preset alarm thresholds. This achieves a complete inspection closed loop from UAV aerial image acquisition to pixel-level accurate defect identification and operation and maintenance decision support.

[0063] In summary, this invention first utilizes UAV telemetry pose data to construct an inverse perspective mapping matrix, correcting perspective distortion in the UAV visible light images of the acquired photovoltaic array to eliminate interference from flight attitude changes on subsequent structural information extraction. Based on this, two parallel independent branches are designed: a structure perception branch explicitly extracts the grid topology and component physical boundaries of the photovoltaic array using differentiable edge operators and graph convolutional networks, forming an array structure prior. The texture discrimination branch specifically captures local pixel-level abnormal textures such as crack direction, hot spot gradients, and dirt dispersion using gradient-sensitive asymmetric convolutional kernels at multiple specific angles. Subsequently, a cross-attention mechanism is used to perform structure-guided semantic fusion of the two features, constraining the spatial scope of micro-texture features with macro-structural priors, fundamentally suppressing false detections across physical boundaries. Finally, an uncertainty-aware gating mechanism based on Shannon entropy is introduced to adaptively enhance highly ambiguous regions easily confused in the fused features. After decoding with a fully convolutional integral class head, a multi-class defect pixel-level semantic mask is output, thereby achieving accurate identification of photovoltaic array defects.

[0064] Another embodiment of the present invention provides a photovoltaic array inspection system based on UAV image semantic segmentation. Figure 5 This is a structural block diagram of a photovoltaic array inspection system based on UAV image semantic segmentation according to an embodiment of the present invention. Figure 5As shown, the photovoltaic array inspection system 100 based on UAV image semantic segmentation according to an embodiment of the present invention includes: a UAV image perspective correction module 110, used to perform perspective correction and channel normalization processing on the UAV visible light image of the photovoltaic array based on the inverse perspective mapping matrix constructed from the camera intrinsic parameter matrix and attitude extrinsic parameter information in the UAV telemetry pose data, so as to obtain a normalized input tensor; and an array topology feature aggregation module 120, used to perform array topology encoding and geometric feature aggregation processing on the normalized input tensor, so as to extract the spatial constraint relationship of the photovoltaic module boundary and the grid line direction, and generate a priori feature map of the array structure. The local pixel-level anomaly representation module 130 is used to capture local pixel-level anomaly representations of crack direction, hot spot gradient, and dirt dispersion pattern from the normalized input tensor by using multi-directional gradient-sensitive convolution kernels with multiple preset specific angles, so as to obtain a defect texture feature map; the structure-guided semantic fusion module 140 is used to perform structure-guided cross-attention semantic fusion on the array structure prior feature map and the defect texture feature map, so as to obtain a cross-attention fusion feature map; the defect semantic mask generation module 150 is used to perform uncertainty-aware gating and defect semantic mask generation on the cross-attention fusion feature map, so as to obtain a multi-class defect semantic mask.

[0065] For example, the defect semantic mask generation module 150 includes a pixel-level probability distribution estimation unit, an uncertainty feature scaling unit, and a maximum value index decoding unit.

[0066] The pixel-level probability distribution estimation unit is used to perform initial pixel-level probability distribution estimation on the cross-attention fusion feature map to obtain the initial segmentation probability tensor.

[0067] The uncertainty feature scaling unit is used to quantize the information entropy of the initial segmentation probability tensor to obtain the uncertainty weight map, and to scale the uncertainty feature map across attention fusion based on the uncertainty weight map to obtain the gated enhanced fusion feature map.

[0068] The maximum value index decoding unit is used to perform full-convolution integral-class decoding on the gated enhanced fusion feature map and perform maximum value index decoding operation pixel by pixel along the channel dimension to obtain multi-class defect semantic masks.

[0069] The specific implementation method of the photovoltaic array inspection system 100 based on UAV image semantic segmentation provided in this embodiment of the invention can be found in the photovoltaic array inspection method based on UAV image semantic segmentation provided in this embodiment of the invention, and will not be repeated here.

[0070] The photovoltaic array inspection system 100 based on UAV image semantic segmentation according to embodiments of the present invention can be implemented in various types of computing devices or control units. In one possible implementation, the photovoltaic array inspection system 100 based on UAV image semantic segmentation according to embodiments of the present invention can be integrated into the computing device or control unit as a software module and / or hardware module.

[0071] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing the present invention, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of the present invention.

Claims

1. A photovoltaic array inspection method based on UAV image semantic segmentation, characterized in that, include: Step 1: Based on the camera intrinsic parameter matrix and attitude extrinsic parameter information constructed from the UAV telemetry pose data, the UAV visible light image of the photovoltaic array is subjected to perspective correction and channel normalization processing to obtain the normalized input tensor. Step 2: Perform array topology encoding and geometric feature aggregation on the normalized input tensor to extract the spatial constraint relationship of the photovoltaic module boundary and grid line direction, and generate the array structure prior feature map; Step 3: By pre-setting multiple multi-directional gradient-sensitive convolution kernels with specific angles, local pixel-level anomaly representations of crack direction, hot spot gradient and dirt diffusion pattern are captured from the normalized input tensor to obtain defect texture feature map. Step 4: Perform structure-guided cross-attention semantic fusion on the array structure prior feature map and the defect texture feature map to obtain a cross-attention fusion feature map; Step 5: Perform uncertainty-aware gating and defect semantic mask generation on the cross-attention fusion feature map to obtain multi-class defect semantic masks.

2. The photovoltaic array inspection method based on UAV image semantic segmentation according to claim 1, characterized in that, Step one includes: Step 1.1: Based on the Euler angles and translation vectors in the UAV telemetry pose data, construct the extrinsic transformation matrix, and multiply the extrinsic transformation matrix with the camera intrinsic matrix to obtain the inverse perspective mapping matrix; Step 1.2: Based on the inverse of the inverse perspective mapping matrix, perform bilinear interpolation projection transformation on the pixel coordinates in the UAV visible light image to obtain a perspective-corrected image; Step 1.3: Scale the perspective-corrected image and standardize the mean and variance of each channel to obtain a normalized input tensor.

3. The photovoltaic array inspection method based on UAV image semantic segmentation according to claim 1, characterized in that, Step two includes: Step 2.1: Based on the differentiable horizontal and vertical convolution kernels, perform differentiable edge feature extraction on the normalized input tensor to obtain the edge response feature tensor. Step 2.2: Perform patch partitioning and center feature aggregation on the edge response feature tensor, and calculate the topological connection weights between nodes by combining the Gaussian radial basis function and the connectivity distance threshold to obtain the array graph node feature matrix and the array graph adjacency matrix. Step 2.3: Perform graph convolution geometric feature aggregation and back projection on the array graph node feature matrix and array graph adjacency matrix to obtain the array structure prior feature map.

4. The photovoltaic array inspection method based on UAV image semantic segmentation according to claim 1, characterized in that, Multi-directional gradient-sensitive convolutional kernels with multiple specific angles include asymmetric convolutional kernels at 0 degrees, 45 degrees, 90 degrees, and 135 degrees.

5. The photovoltaic array inspection method based on UAV image semantic segmentation according to claim 4, characterized in that, Step three includes: Step 3.1: Perform multi-directional asymmetric convolution calculation on the normalized input tensor using multi-directional gradient-sensitive convolution kernels at multiple specific angles to obtain a set of multi-directional intermediate feature tensors; Step 3.2: Concatenate multi-directional feature channels on the multi-directional intermediate feature tensor set to obtain a concatenated high-dimensional texture tensor; Step 3.3: The cascaded high-dimensional texture tensor is subjected to cross-channel dimensionality reduction projection using pointwise convolution kernels, and then subjected to feature distribution correction and nonlinear recombination through batch normalization layers and nonlinear activation functions to obtain the defect texture feature map.

6. The photovoltaic array inspection method based on UAV image semantic segmentation according to claim 1, characterized in that, Step four includes: Step 4.1: Perform linear mapping and spatial dimension flattening operations on the prior feature map of the array structure and the defect texture feature map to obtain the query matrix, key matrix and value matrix; The linear mapping between the prior feature map of the array structure and the defect texture feature map includes: A linear mapping is performed between the prior feature map of the array structure and the defect texture feature map according to the following formula: ； ； ； in, This is the 3D query tensor after query projection; To query the pointwise convolution weight parameter matrix of the projection layer; This is the spatial convolution operator; For the prior feature map of the array structure; To query the bias parameter vector of the projection layer; This is the three-dimensional bond tensor after bond projection; It is the three-dimensional value tensor after longitude projection; and These are the pointwise convolutional weight parameter matrices for the key projection layer and the value projection layer, respectively; For defect texture feature maps; and These are the bias parameter vectors for the key projection layer and the value projection layer, respectively. Step 4.2: Calculate the cross-attention weight graph between the query matrix and the key matrix; where the attention weight value in the i-th row and j-th column of the cross-attention weight graph is... Represented as: ； in, This is the structure query vector for the i-th row of the query matrix. Let be the transpose of the texture key vector in the j-th row of the key matrix. This is the transpose of the texture key vector in the nth row of the key matrix. For attention feature dimension, It is an exponential function, where N is the total number of spatial pixels in the defect texture feature map; Step 4.3: Based on the cross-attention weight map, perform weighted aggregation multiplication on the value matrix to recover the two-dimensional spatial structure, thereby obtaining the structure-guided enhanced texture tensor. Then, add the structure-guided enhanced texture tensor to the defect texture feature map pixel-by-pixel residual to obtain the cross-attention fusion feature map; wherein, the feature vector of the cross-attention fusion feature map at spatial coordinates (i,j) is... Represented as: ； in, For the defect texture feature map in spatial coordinates The original texture feature vector at that location, For dimension reshaping operation functions, This is a cross-attention weight graph; For value matrices, For the structure-guided enhanced texture tensor in spatial coordinates The eigenvector at that location.

7. The photovoltaic array inspection method based on UAV image semantic segmentation according to claim 1, characterized in that, Step five includes: Uncertainty quantization of cross-attention fusion feature maps is performed by pixel-wise Shannon entropy calculation to obtain an uncertainty weight map; Based on the uncertainty weight map as a spatial gate, feature scaling enhancement is performed on easily confused regions in the cross-attention fusion feature map, and then multi-class defect semantic masks are obtained by pixel-by-pixel decoding through the full-convolution integral class head.

8. The photovoltaic array inspection method based on UAV image semantic segmentation according to claim 1, characterized in that, Step five includes: Step 5.1: Perform initial pixel-level probability distribution estimation on the cross-attention fusion feature map to obtain the initial segmentation probability tensor; Step 5.2: Quantize the initial segmentation probability tensor with information entropy to obtain an uncertain weight map. Then, based on the uncertain weight map, scale the cross-attention fusion feature map to obtain a gated augmented fusion feature map. The gated augmented fusion feature map is located in spatial coordinates... eigenvectors at location Represented as: ； in, For cross-attention fusion feature maps in spatial coordinates The eigenvector at point , where ⊙ is the element-wise multiplication operator. For normalized activation functions, For the uncertainty weighting graph in spatial coordinates The information entropy value at that location, and , This represents the preset total number of semantic segmentation categories for photovoltaic arrays. The initial segmentation probability tensor in spatial coordinates The predicted probability value for belonging to class c. It is the natural logarithm function. A very small positive floating-point constant is introduced to prevent the input of the natural logarithm function from being zero; Step 5.3: Perform full-convolution integral decoding on the gated enhanced fusion feature map and perform maximum value index decoding operation pixel by pixel along the channel dimension to obtain multi-class defect semantic mask.

9. A photovoltaic array inspection system based on UAV image semantic segmentation, characterized in that, include: The UAV image perspective correction module is used to perform perspective correction and channel normalization processing on the UAV visible light image of the photovoltaic array based on the inverse perspective mapping matrix constructed from the camera intrinsic parameter matrix and attitude extrinsic parameter information in the UAV telemetry pose data, so as to obtain the normalized input tensor. The array topology feature aggregation module is used to perform array topology encoding and geometric feature aggregation on the normalized input tensor to extract the spatial constraint relationship of the photovoltaic module boundary and grid line direction, and generate the array structure prior feature map. The local pixel-level anomaly representation module is used to capture local pixel-level anomaly representations of crack direction, hot spot gradient and dirt dispersion pattern from the normalized input tensor by using multi-directional gradient sensitive convolution kernels with multiple preset specific angles, so as to obtain defect texture feature map. The structure-guided semantic fusion module is used to perform structure-guided cross-attention semantic fusion on the array structure prior feature map and the defect texture feature map to obtain a cross-attention fusion feature map. The defect semantic mask generation module is used to perform uncertainty-aware gating and defect semantic mask generation on cross-attention fusion feature maps to obtain multi-class defect semantic masks.

10. The photovoltaic array inspection system based on UAV image semantic segmentation according to claim 9, characterized in that, The defect semantic mask generation module includes: A pixel-level probability distribution estimation unit is used to perform initial pixel-level probability distribution estimation on the cross-attention fusion feature map to obtain the initial segmentation probability tensor; The uncertainty feature scaling unit is used to quantize the information entropy of the initial segmentation probability tensor to obtain the uncertainty weight map, and to scale the uncertainty feature of the cross-attention fusion feature map based on the uncertainty weight map to obtain the gated enhanced fusion feature map. The maximum value index decoding unit is used to perform full-convolution integral class decoding on the gated enhanced fusion feature map and perform maximum value index decoding operation pixel by pixel along the channel dimension to obtain multi-class defect semantic masks.

Images