Photovoltaic module infrared image segmentation method, device, equipment, medium and program product

Abstract

The present invention discloses a photovoltaic module infrared image segmentation method, device, equipment, medium and program product. The method comprises: collecting original infrared images of photovoltaic modules, preprocessing the original infrared images to obtain target photovoltaic images, calculating grayscale co-occurrence matrix entropy of the target photovoltaic images, obtaining texture feature images based on the grayscale co-occurrence matrix entropy, splicing the texture feature images with grayscale images of the target photovoltaic images to obtain input feature images, performing multi-scale feature analysis on the input feature images to obtain fused feature images, and inputting the fused feature images into an image segmentation model to perform image segmentation to obtain segmented feature images. Since the present invention effectively captures texture information in photovoltaic images through grayscale co-occurrence analysis and ensures that the model can capture rich detail information through multi-scale feature analysis, the problem of information loss caused by single-scale analysis is effectively avoided, thereby improving the model's performance in photovoltaic module infrared image segmentation.

Description

Technical Field

[0001] The present invention relates to the field of image segmentation technology, and in particular to a photovoltaic module infrared image segmentation method, device, equipment, medium and program product. Background Art

[0002] In today's digital age, image segmentation has become a critical branch of computer vision, demonstrating widespread application value across numerous industries. The core of image segmentation lies in accurately dividing an image into several blocks with specific requirements based on its intrinsic characteristics, such as texture, contrast, and topology. These blocks exhibit high internal similarities, while distinct differences exist between them. With the rapid development and increasing popularity of deep learning technology, image segmentation methods have gradually evolved from traditional methods based on manual feature extraction to more efficient, accurate, and highly generalizable deep learning-based approaches. This shift has not only significantly improved the accuracy and efficiency of image segmentation but also laid a solid foundation for the in-depth application of computer vision in a wider range of fields.

[0003] In the field of deep learning, the U-net (U-net) architecture has become a common framework for image segmentation. The basic U-net architecture consists of two parts: a contracting path on the left and an expanding path on the right. However, U-nets currently have limitations when segmenting complex images, such as infrared images of photovoltaic panels. Limited by a small sample size and a single feature analysis scale, U-nets are unable to effectively extract deep features from images, resulting in the inability to accurately segment the characteristic image of photovoltaic panels from infrared images. Summary of the Invention

[0004] The main purpose of the present invention is to provide a photovoltaic module infrared image segmentation method, device, equipment, medium and program product, aiming to solve the problem that the U-Net network in the prior art is limited by the small number of samples and the single feature analysis scale, which makes it impossible to effectively mine the deep features in the image, resulting in the inability to accurately segment the characteristic image of the photovoltaic module from the infrared image.

[0005] To achieve the above object, the present invention provides a photovoltaic module infrared image segmentation method, the method comprising the following steps:

[0006] Collecting original infrared images of photovoltaic modules, where the infrared images are collected by an infrared camera of a drone;

[0007] Preprocessing the original infrared image to obtain a target photovoltaic image, wherein the preprocessing includes geometric correction processing, filtering processing, and image enhancement processing;

[0008] Calculating the gray-level co-occurrence matrix entropy of the target photovoltaic image, and acquiring a texture feature image based on the gray-level co-occurrence matrix entropy;

[0009] splicing the texture feature image with the grayscale image of the target photovoltaic image to obtain an input feature image;

[0010] Performing multi-scale feature analysis on the input feature image to obtain a fused feature image;

[0011] Inputting the fused feature image into an image segmentation model to perform image segmentation to obtain a segmented feature image;

[0012] The image segmentation model is constructed based on the U-Net model, and the image segmentation model includes a multi-scale encoder and a multi-scale decoder;

[0013] The multi-scale encoder includes a plurality of convolutional layers, pooling layers and a multi-scale fusion layer of different scales. The convolutional layers are used to extract feature information of different scales in the fusion feature map. The pooling layers are used to reduce the feature dimension. The multi-scale fusion layer is used to fuse the feature images output by the convolutional layers of each scale.

[0014] The multi-scale decoder includes multiple deconvolution layers of different scales, skip connection layers, feature fusion layers and upsampling layers. The deconvolution layers are used to extract feature information of different scales. The skip connection layers are used to fuse the feature information of the multi-scale encoder with the feature information of the multi-scale decoder. The feature fusion layer is used to perform multi-scale feature integration on the feature information output by the multiple deconvolution layers of different scales and the feature information output by the skip connection layers. The upsampling layer is used to restore the feature image.

[0015] Optionally, calculating the gray level co-occurrence matrix entropy of the target photovoltaic image and acquiring the texture feature image based on the gray level co-occurrence matrix entropy includes:

[0016] Normalize the pixel grayscale value of each pixel in the target photovoltaic image:

[0017]

[0018] in, Representing coordinates The grayscale value of the pixel after normalization, Representing coordinates The original grayscale value of the pixel, represents the maximum grayscale value of the target photovoltaic image, Indicates the number of gray levels;

[0019] Counting the pixel spacing and pixel direction of each pixel pair based on the normalized pixels, where the pixel pair consists of two normalized pixels;

[0020] traversing each pixel pair in the target photovoltaic image according to the pixel spacing and the pixel direction to obtain grayscale symbiosis information;

[0021] A gray-level symbiosis matrix is ​​constructed based on the gray-level symbiosis information, and the gray-level symbiosis matrix entropy is calculated based on the gray-level symbiosis matrix:

[0022]

[0023] in, Indicates the target distance and target direction Next, grayscale value and The co-occurrence frequency of pixel pairs;

[0024] The target entropy feature vector is obtained by concatenating the gray-level co-occurrence matrix entropies at multiple directions and distances in the target photovoltaic image:

[0025]

[0026] in, represents the target entropy feature vector, Indicates direction and distance Gray-level co-occurrence matrix entropy under Represents a transpose operation;

[0027] The target entropy feature vector is mapped to the image space to generate a texture feature image.

[0028] Optionally, performing multi-scale feature analysis on the input feature image to obtain a fused feature image includes:

[0029] The input feature image is decomposed into multiple scales by wavelet decomposition to obtain feature images of multiple scales:

[0030]

[0031] in, represents the wavelet decomposition operation, represents the input feature image, represents a low-pass filter, represents a high-pass filter, represents the decomposition scale, Represents the decomposed feature image;

[0032] Arrange the feature images of the multiple scales according to the scale order to construct a feature pyramid matrix;

[0033] Multi-scale feature fusion is performed based on the pyramid matrix and the channel attention mechanism to obtain a fused feature image. The multi-scale fusion refers to the following formula:

[0034]

[0035]

[0036]

[0037] in, represents the fused feature image, represents the attention weight, 、 and Represent feature maps of different scales, 、 and Represent the weight coefficients, represents the feature concatenation operation, represents the weighted concatenated feature image, and Represents the fully connected layer, which is the Sigmoid activation function. represents a nonlinear function.

[0038] Optionally, preprocessing the original infrared image to obtain a target photovoltaic image includes:

[0039] The original infrared image is subjected to geometric correction processing, wherein the geometric correction processing includes translation transformation, mirror transformation, rotation transformation, similarity transformation, affine transformation and perspective transformation. The translation transformation refers to the following formula:

[0040]

[0041] in, and Represent the horizontal coordinate and vertical coordinate in the original infrared image, represents the horizontal translation amount, represents the vertical translation, and Indicates the horizontal and vertical coordinates after translation;

[0042] The mirror transformation includes horizontal mirror transformation and vertical mirror transformation, referring to the following formula:

[0043]

[0044]

[0045] in, and Respectively represent the horizontal coordinate and vertical coordinate after mirror transformation, Indicates the image width, Indicates the image length;

[0046] The rotation transformation refers to the following formula:

[0047]

[0048] in, and Respectively represent the horizontal coordinate and vertical coordinate after rotation transformation, Indicates the rotation angle;

[0049] The similarity transformation refers to the following formula:

[0050]

[0051] in, and Respectively represent the horizontal coordinate and vertical coordinate after similarity transformation, represents the scaling factor;

[0052] The affine transformation refers to the following formula:

[0053]

[0054] in, and Respectively represent the horizontal coordinate and vertical coordinate after affine transformation, 、 、 and are all linear transformation coefficients;

[0055] The perspective transformation refers to the following formula:

[0056]

[0057] in, and Respectively represent the horizontal coordinate and vertical coordinate after perspective transformation, Represents the transformation element of the perspective transformation matrix;

[0058] Performing filtering on the original infrared image after geometric correction to obtain a candidate infrared image;

[0059] Perform image enhancement processing on the candidate infrared image to obtain a target photovoltaic image.

[0060] Optionally, filtering the original infrared image after geometric correction to obtain a candidate infrared image includes:

[0061] The original infrared image after geometric correction is subjected to median filtering to obtain an initial filtered image. The median filtering process refers to the following formula:

[0062]

[0063]

[0064] in, represents the initial filtered image obtained after median filtering, Indicates that the filter window becomes longer, Represents the first pixel in the sorted pixel value sequence elements, Represents a sorted sequence of pixel values;

[0065] The initial filtered image is subjected to Gaussian filtering to obtain a candidate infrared image. The Gaussian filtering is performed according to the following formula:

[0066]

[0067]

[0068]

[0069] in, Represents the pixel value of the filtered image at coordinates, represents the Gaussian weight function, represents the standard deviation of the Gaussian distribution, and Represents the horizontal and vertical offsets respectively. Indicates the initial filtered image in The pixel value at position, Indicated in pixels Center the field window size.

[0070] Optionally, performing image enhancement processing on the candidate infrared image to obtain a target photovoltaic image includes:

[0071] Analyze the pixel grayscale distribution of the candidate infrared image to determine the number of pixels at each grayscale level, and generate an original histogram based on the number of pixels at each grayscale level:

[0072]

[0073] in, represents the number of pixels with gray level in the candidate infrared image, Indicates the Gray levels;

[0074] The original histograms are accumulated to generate a cumulative histogram:

[0075]

[0076] in, Indicates the cumulative distribution function, which is used to accumulate gray levels from 0 to The number of pixels;

[0077] Grayscale mapping is performed on the cumulative histogram, and the mapped grayscale values ​​are quantized to obtain a target photovoltaic image:

[0078]

[0079] in, represents the grayscale range, Indicates the total number of pixels in the image. Represents the grayscale value after mapping, Represents the rounding function.

[0080] In addition, to achieve the above-mentioned purpose, the present invention further proposes a photovoltaic module infrared image segmentation device, the photovoltaic module infrared image segmentation device comprising:

[0081] An image acquisition module is used to acquire original infrared images of photovoltaic modules, where the infrared images are acquired by an infrared camera of a drone;

[0082] An image processing module is used to pre-process the original infrared image to obtain a target photovoltaic image, wherein the pre-processing includes geometric correction processing, filtering processing and image enhancement processing;

[0083] a gray-level symbiosis analysis module, configured to calculate the gray-level symbiosis matrix entropy of the target photovoltaic image and obtain a texture feature image based on the gray-level symbiosis matrix entropy;

[0084] a feature stitching module, configured to stitch the texture feature image with the grayscale image of the target photovoltaic image to obtain an input feature image;

[0085] A multi-scale feature analysis module, configured to perform multi-scale feature analysis on the input feature image to obtain a fused feature image;

[0086] An image segmentation module is used to input the fused feature image into an image segmentation model to perform image segmentation and obtain a segmented feature image;

[0087] The image segmentation model is constructed based on the U-Net model, and the image segmentation model includes a multi-scale encoder and a multi-scale decoder;

[0088] The multi-scale encoder includes a plurality of convolutional layers, pooling layers and a multi-scale fusion layer of different scales. The convolutional layers are used to extract feature information of different scales in the fusion feature map. The pooling layers are used to reduce the feature dimension. The multi-scale fusion layer is used to fuse the feature images output by the convolutional layers of each scale.

[0089] The multi-scale decoder includes multiple deconvolution layers of different scales, skip connection layers, feature fusion layers and upsampling layers. The deconvolution layers are used to extract feature information of different scales. The skip connection layers are used to fuse the feature information of the multi-scale encoder with the feature information of the multi-scale decoder. The feature fusion layer is used to perform multi-scale feature integration on the feature information output by the multiple deconvolution layers of different scales and the feature information output by the skip connection layers. The upsampling layer is used to restore the feature image.

[0090] In addition, to achieve the above-mentioned purpose, the present application also proposes a photovoltaic module infrared image segmentation device, which includes: a memory, a processor, and a computer program stored on the memory and runnable on the processor, and the computer program is configured to implement the steps of the photovoltaic module infrared image segmentation method as described above.

[0091] In addition, to achieve the above-mentioned purpose, the present application also proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the photovoltaic component infrared image segmentation method described above are implemented.

[0092] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the photovoltaic component infrared image segmentation method as described above.

[0093] The present invention collects the original infrared image of the photovoltaic component, and the infrared image is collected by the infrared camera of the unmanned aerial vehicle, and the original infrared image is preprocessed to obtain the target photovoltaic image, and the preprocessing includes geometric correction processing, filtering processing and image enhancement processing, and the grayscale co-occurrence matrix entropy of the target photovoltaic image is calculated, and the texture feature image is obtained based on the grayscale co-occurrence matrix entropy. The texture feature image is spliced ​​with the grayscale image of the target photovoltaic image to obtain an input feature image, and the input feature image is subjected to multi-scale feature analysis to obtain a fused feature image. The fused feature image is input into an image segmentation model for image segmentation to obtain a segmented feature image, wherein the image segmentation model is constructed based on the U-Net model. The image segmentation model includes a multi-scale encoder and a multi-scale decoder. Since the present invention pre-processes the original infrared image, it effectively eliminates the geometric distortion caused by angle, perspective relationship, photography and other factors in the original infrared image, greatly reduces the noise in the image, strengthens the key features in the image, and weakens the interference of irrelevant areas. By introducing grayscale symbiosis analysis, the texture features in the image are effectively captured, and the loss of key information is avoided. Through multi-scale feature analysis, the global structural information and local feature information in the image are accurately captured, and the information loss caused by single-scale feature extraction is reduced. The robustness of the model in dealing with complex scenes and scale changes is improved, thereby improving the model's infrared image segmentation performance of photovoltaic modules. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0095] Figure 1 It is a schematic structural diagram of a photovoltaic module infrared image segmentation device in a hardware operating environment according to an embodiment of the present invention;

[0096] Figure 2 This is a flow chart of an embodiment of a photovoltaic module infrared image segmentation method according to the present invention;

[0097] Figure 3 This is a schematic diagram of an aerial photograph of a photovoltaic module infrared image captured by a drone in an embodiment of the photovoltaic module infrared image segmentation method of the present invention;

[0098] Figure 4 Schematic diagram of the structure of a multi-scale encoder in an embodiment of the photovoltaic module infrared image segmentation method of the present invention;

[0099] Figure 5Schematic diagram of the structure of a multi-scale decoder in an embodiment of the photovoltaic module infrared image segmentation method of the present invention;

[0100] Figure 6 This is a network structure framework diagram of the image segmentation model in one embodiment of the photovoltaic module infrared image segmentation method of the present invention;

[0101] Figure 7 This is a structural block diagram of an embodiment of a photovoltaic module infrared image segmentation device of the present invention.

[0102] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0103] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0104] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a photovoltaic module infrared image segmentation device in the hardware operating environment involved in an embodiment of the present invention.

[0105] like Figure 1 As shown, the photovoltaic module infrared image segmentation device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display and an input unit, such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a wireless fidelity (WI-FI) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. Optionally, the memory 1005 may be a storage device independent of the processor 1001.

[0106] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the photovoltaic assembly infrared image segmentation device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0107] like Figure 1As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a photovoltaic module infrared image segmentation program.

[0108] exist Figure 1 In the photovoltaic module infrared image segmentation device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the photovoltaic module infrared image segmentation device of the present invention can be set in the photovoltaic module infrared image segmentation device, and the photovoltaic module infrared image segmentation device calls the photovoltaic module infrared image segmentation program stored in the memory 1005 through the processor 1001 and executes the photovoltaic module infrared image segmentation method provided by the embodiment of the present invention.

[0109] The embodiment of the present invention provides a photovoltaic module infrared image segmentation method, referring to Figure 2 , Figure 2 Schematic diagram of a flow chart of an embodiment of a photovoltaic module infrared image segmentation method according to the present invention.

[0110] In this embodiment, the photovoltaic module infrared image segmentation method includes the following steps:

[0111] Step S10: collecting original infrared images of the photovoltaic modules.

[0112] It should be understood that the execution subject of this embodiment may be a computing service device with data processing, network communication, and program execution capabilities, such as a tablet computer, personal computer, mobile phone, etc., or a terminal electronic device capable of performing the aforementioned functions. The following uses a photovoltaic module infrared image segmentation device (hereinafter referred to as the segmentation device) as an example to illustrate this embodiment and the following embodiments.

[0113] It should be noted that the infrared image is collected by the infrared camera of the drone.

[0114] In some embodiments, the segmentation device may first perform preliminary preparations for the drone, including equipment inspection, environmental assessment, and route planning. After completing these preparations, the infrared camera is properly installed on the drone and debugged to ensure proper function and flexible adjustment of the gimbal angle. The drone then takes off according to the planned route and maintains a stable hover after reaching the designated hovering point. The gimbal camera angle is adjusted to ensure it is perpendicular to the photovoltaic module for optimal image quality. At the hovering point, infrared images of the photovoltaic module are captured, and the GPS data for the captured images is recorded.

[0115] Reference Figure 3 , Figure 3 Aerial diagram of drones collecting infrared images of photovoltaic modules. Figure 3 The drone aerial photography diagram clearly depicts the drone collecting multiple images of photovoltaic panels in the air, including the drone k+1 Acquire data and build infrared image sequences based on the acquisition results , the infrared image sequence includes k+1 Through precise aerial photography planning, the drone can fly stably along the preset path, ensuring the collected infrared images are high-resolution and highly consistent, providing a high-quality data foundation for subsequent image analysis.

[0116] Step S20: pre-processing the original infrared image to obtain a target photovoltaic image.

[0117] It should be noted that the preprocessing includes geometric correction processing, filtering processing and image enhancement processing.

[0118] In some embodiments, the segmentation device may perform geometric correction on the original infrared image. By performing appropriate transformations on the image, geometric distortion caused by factors such as angle, perspective, and photography may be eliminated.

[0119] In some embodiments, the segmentation device may perform filtering processing on the geometrically corrected infrared image, thereby reducing the impact of interference noise on the image.

[0120] In some embodiments, the segmentation device can enhance the infrared image after filtering, strengthen the studied area, and weaken the interfering background area, so that the image information is more intuitive.

[0121] Furthermore, in order to effectively correct the geometric distortion of the image and improve the image quality, the above step S20 may include:

[0122] Step S201: performing geometric correction processing on the original infrared image.

[0123] Step S202: filtering the original infrared image after geometric correction to obtain a candidate infrared image;

[0124] Step S203: performing image enhancement processing on the candidate infrared image to obtain a target photovoltaic image.

[0125] It should be noted that the segmentation device needs to perform geometric correction processing on the photovoltaic module infrared image before segmenting it. The geometric correction processing includes translation transformation, mirror transformation, rotation transformation, similarity transformation, affine transformation and perspective transformation.

[0126] It should be noted that the translation transformation can be to move all points in the image horizontally or vertically according to a specified translation amount. The translation transformation refers to the following formula:

[0127]

[0128] in, and Represent the horizontal coordinate and vertical coordinate in the original infrared image, represents the horizontal translation amount, represents the vertical translation, and Indicates the horizontal and vertical coordinates after translation;

[0129] Mirror transformation is an isometric transformation, which means that it keeps the shape and size of the figure unchanged, but only changes the position and direction of the figure. At the same time, mirror transformation is also a symmetric transformation. The transformed figure is symmetrical with the original figure about the mirror line and has the same symmetry. The mirror transformation includes horizontal mirror transformation and vertical mirror transformation, refer to the following formula:

[0130]

[0131]

[0132] in, and Respectively represent the horizontal coordinate and vertical coordinate after mirror transformation, Indicates the image width, Indicates the image length;

[0133] A rotation transformation flips a figure or object along a straight line (called a mirror line or axis of symmetry) so that the transformed figure is symmetrical with the original figure about this line. The rotation transformation is based on the following formula:

[0134]

[0135] in, and Respectively represent the horizontal coordinate and vertical coordinate after rotation transformation, Indicates the rotation angle;

[0136] A similarity transformation transforms a figure or object into another similar figure through translation, rotation, and scaling (including uniform scaling and non-uniform scaling). The similarity transformation refers to the following formula:

[0137]

[0138] in, and Respectively represent the horizontal coordinate and vertical coordinate after similarity transformation, represents the scaling factor;

[0139] An affine transformation is a geometric transformation that preserves straight lines and parallelism. It transforms one shape or object into another through translation, rotation, scaling (including uniform and non-uniform scaling), and shearing. The affine transformation is described by the following formula:

[0140]

[0141] in, and Respectively represent the horizontal coordinate and vertical coordinate after affine transformation, 、 、 and are all linear transformation coefficients;

[0142] Perspective transformation is a geometric transformation that maps points on one plane (or space) to another plane (or space). It simulates the imaging process of the human eye or camera, projecting points in three-dimensional space onto a two-dimensional plane. The perspective transformation refers to the following formula:

[0143]

[0144] in, and Respectively represent the horizontal coordinate and vertical coordinate after perspective transformation, Represents the transformation element of the perspective transformation matrix.

[0145] Furthermore, in order to minimize the impact of interference noise on the image, the above step S202 may include:

[0146] Step S2021: performing median filtering on the original infrared image after geometric correction to obtain an initial filtered image;

[0147] Step S2022: performing Gaussian filtering on the initial filtered image to obtain a candidate infrared image.

[0148] It should be noted that the segmentation device eliminates geometric distortion caused by angle, perspective, photography and other factors by performing appropriate transformations on the image. At this time, the image needs to be denoised to minimize the impact of interference noise on the image, including median filtering, Gaussian filtering and mean filtering.

[0149] It should be noted that the segmentation device first performs median filtering on the original infrared image to effectively suppress the noisy signal, especially the processing of impact, random and other interference. The median filtering process refers to the following formula:

[0150]

[0151]

[0152] in, represents the initial filtered image obtained after median filtering, Indicates that the filter window becomes longer, Represents the first pixel in the sorted pixel value sequence elements, Represents a sorted sequence of pixel values.

[0153] The segmentation device then performs Gaussian filtering on the median filtered image to effectively smooth the Gaussian noise. The Gaussian filtering process refers to the following formula:

[0154]

[0155]

[0156]

[0157] in, Represents the pixel value of the filtered image at coordinates, represents the Gaussian weight function, represents the standard deviation of the Gaussian distribution, and Represents the horizontal and vertical offsets respectively. Indicates the initial filtered image in The pixel value at position, Indicated in pixels Center the field window size.

[0158] In some embodiments, the segmentation device may further perform mean filtering on the image. Mean filtering replaces each pixel value in the image with the average value of its neighborhood through linear filtering to achieve a filtering effect. This is suitable for removing uniformly distributed noise. Mean filtering refers to the following formula:

[0159]

[0160] In the formula, assume that there is a Window, ( , ) is any point in the original image, is the original image at position The pixel value of is the filtered image at position The pixel value of is the filter window, which contains all pixel positions involved in the calculation, It's a window The number of pixels in .

[0161] Furthermore, in order to effectively extract key information from the image and enhance image features and details, the above step S203 may include:

[0162] Step S2031: analyzing the pixel grayscale distribution of the candidate infrared image, determining the number of pixels at each grayscale level, and generating an original histogram based on the number of pixels at each grayscale level;

[0163] Step S2032: Accumulating the original histograms to generate a cumulative histogram;

[0164] Step S2033: grayscale mapping is performed on the cumulative histogram, and the mapped grayscale values ​​are quantized to obtain a target photovoltaic image.

[0165] It is necessary to address the problem that the infrared image of the photovoltaic module obtained after image filtering still has blurred key information and the quality of local information is higher than that of key information. Therefore, histogram equalization is used to improve the contrast of the image. This is very useful for infrared images with too bright or too dark background and foreground. The brightness of the enhanced infrared image is more uniform on the histogram, which can bring better display of bone structure and details in overexposed or underexposed photos.

[0166] Histogram equalization is divided into calculating the histogram first, then calculating the cumulative histogram, mapping the grayscale value, and finally quantizing. By adjusting the histogram distribution of the image to make it more uniform, the contrast of the image is enhanced. The histogram equalization of the kth grayscale image of the photovoltaic module infrared image is calculated according to the following formula:

[0167]

[0168] in, represents the number of pixels with gray level in the candidate infrared image, Indicates the gray levels.

[0169] The cumulative histogram is calculated according to the following formula:

[0170]

[0171] in, Indicates the cumulative distribution function, which is used to accumulate gray levels from 0 to The number of pixels.

[0172] Mapping grayscale values ​​and quantization refer to the following formula:

[0173]

[0174] in, represents the grayscale range, Indicates the total number of pixels in the image. Represents the grayscale value after mapping, Represents the rounding function.

[0175] Step S30: calculating the gray-level co-occurrence matrix entropy of the target photovoltaic image, and acquiring a texture feature image based on the gray-level co-occurrence matrix entropy.

[0176] It should be noted that this embodiment introduces a gray-level co-occurrence matrix to better identify image texture characteristics and improve segmentation of detailed features such as edges and textures. This matrix utilizes information such as the relative position and grayscale value differences of each pixel in the image to extract image texture features. This mechanism allows for capturing detailed features such as edges and textures, avoiding the loss of key image information.

[0177] It can be understood that the segmentation device can facilitate each pixel pair in the target photovoltaic image, count the grayscale co-occurrence frequency of the pixel pairs based on the grayscale values ​​obtained by traversal, construct a grayscale co-occurrence matrix based on the grayscale co-occurrence frequency, thereby determining the grayscale co-occurrence matrix entropy, and obtain a texture feature image based on the grayscale co-occurrence matrix entropy.

[0178] In some embodiments, the segmentation device may construct a grayscale difference co-occurrence matrix to extract texture features by statistically analyzing the grayscale value difference distribution of adjacent pixels in the image, specifically including:

[0179] Traverse all pairs of pixels to be analyzed that meet the target direction and target distance in the target photovoltaic image, and calculate the grayscale difference between each pair of pixels to be analyzed:

[0180]

[0181] Among them, represents the grayscale difference, and Respectively indicate that the target direction is met Distance to target The pixel pairs to be analyzed and Gray value of

[0182] Construct the grayscale difference co-occurrence matrix based on the grayscale difference:

[0183]

[0184] in, and represents the grayscale value of adjacent pixels, Indicates the difference is The number of pixel pairs;

[0185] Will Normalized to a probability distribution , get the probability distribution of multiple directions and distances , based on the probability distribution of multiple directions and distances, feature splicing is performed to obtain high-dimensional features:

[0186]

[0187] in, Represents high-dimensional features, Indicates different directions and distance The grayscale difference is The probability distribution of pixel pairs.

[0188] The segmentation device can extract small difference emphasis information, large difference emphasis information, difference variance and difference entropy from the grayscale difference co-occurrence matrix, thereby extracting texture features in the photovoltaic image. Among them, small difference emphasis information is used to reflect the proportion of low-contrast areas in the image (the larger the value, the more uniform the texture); large difference emphasis information is used to reflect the proportion of high-contrast areas in the image (the larger the value, the rougher the texture); difference variance is used to measure the degree of discreteness of grayscale difference (the larger the value, the more drastic the texture change); difference entropy is used to measure the uncertainty of grayscale difference (the larger the value, the more complex the texture).

[0189] Furthermore, in order to accurately calculate the gray-level co-occurrence matrix entropy and thus precisely extract image texture features, the above step S30 may include:

[0190] Step S301: normalizing the grayscale value of each pixel in the target photovoltaic image:

[0191]

[0192] in, Representing coordinates The grayscale value of the pixel after normalization, Representing coordinates The original grayscale value of the pixel, represents the maximum grayscale value of the target photovoltaic image, Indicates the number of gray levels;

[0193] Step S302: Counting the pixel spacing and pixel direction of each pixel pair based on the normalized pixels, where the pixel pair consists of two normalized pixels;

[0194] Step S303: traversing each pixel pair in the target photovoltaic image according to the pixel spacing and the pixel direction to obtain grayscale symbiosis information;

[0195] Step S304: constructing a gray-level symbiosis matrix based on the gray-level symbiosis information, and calculating gray-level symbiosis matrix entropy based on the gray-level symbiosis matrix;

[0196] Step S305: splicing the gray-level co-occurrence matrix entropies at multiple directions and distances in the target photovoltaic image to obtain a target entropy feature vector;

[0197] Step S306: Mapping the target entropy feature vector to an image space to generate a texture feature image.

[0198] It should be noted that each element in the gray level co-occurrence matrix represents the frequency of two pixels appearing at a certain position and distance, that is, the gray level co-occurrence information. The gray level co-occurrence matrix is ​​defined as the number of pixels from the pixel with gray level i to the pixel with distance The grayscale is j The probability of the pixel point. Representing probability (i,j=0,1, 2,……,L-1) .in i and j represents the grayscale of the pixel, L represents the grayscale of the image, d Represents the spatial position relationship between two pixels. d The difference in determines the distance between two pixels. θ It is the direction in which the gray-level co-occurrence matrix is ​​generated, generally 0°, 45°, 90°, and 135°.

[0199] In some embodiments, the gray-level co-occurrence matrix is ​​constructed as follows: for a given image, a specific orientation and distance are selected; for each pixel, the co-occurrence frequency of gray values ​​between pixels with the same orientation and adjacent distance within the selected distance is counted to form a gray-level co-occurrence matrix; for each selected orientation and distance, each gray-level co-occurrence matrix is ​​obtained; when the distance d between two pixels is determined, the gray-level co-occurrence matrix at that distance can be expressed as follows:

[0200]

[0201] Where, Representing probability (i,j=0,1,2,……,L-1) , i and j represents the grayscale of the pixel, L represents the grayscale of the image, d Represents the spatial position relationship between two pixels.

[0202] It is understandable that the pixel grayscale value of each pixel in the target photovoltaic image is normalized, and the normalization refers to the following formula:

[0203]

[0204] in, Representing coordinates The grayscale value of the pixel after normalization, Representing coordinates The original grayscale value of the pixel, represents the maximum grayscale value of the target photovoltaic image, Indicates the number of gray levels.

[0205] It should be noted that the segmentation device calculates the gray-level co-occurrence matrix entropy value of the image based on the gray-level co-occurrence matrix. The entropy value reflects the complex texture feature information of the image. The gray-level co-occurrence matrix entropy value is calculated according to the following formula:

[0206]

[0207] in, Indicates the target distance and target direction Next, grayscale value and The co-occurrence frequency of pixel pairs.

[0208] It is understandable that in order to obtain a more comprehensive texture feature image, the segmentation device can integrate the entropy values ​​in different directions (for example, calculating the entropy values ​​in the directions of 0°, 45°, 90°, and 135°) and distances, and splice the entropy values ​​in each direction into a high-dimensional feature vector to obtain the target entropy feature vector. The target entropy feature vector calculation formula is as follows:

[0209]

[0210] in, represents the target entropy feature vector, Indicates direction and distance Gray-level co-occurrence matrix entropy under Represents a transpose operation.

[0211] Step S40: splicing the texture feature image with the grayscale image of the target photovoltaic image to obtain an input feature image.

[0212] It can be understood that in order to enrich the feature representation of the image and improve the image segmentation performance of the model, this embodiment can splice the texture feature image with the grayscale image. The original grayscale image contains the basic pixel intensity information of the target photovoltaic image and reflects the brightness distribution of the image; while the texture feature image captures the texture information of the image. Splicing the two allows the model to simultaneously utilize pixel-level grayscale information and structural-level texture information to form a more comprehensive and rich feature representation, which helps the model learn more discriminative image representations. In the U-Net network, the richer the input features, the more the model can capture the complex patterns of the image.

[0213] Step S50: performing multi-scale feature analysis on the input feature image to obtain a fused feature image.

[0214] It should be noted that this embodiment introduces multi-scale feature fusion. By integrating feature information at different scales, the model's ability to detect and recognize objects of various scales is significantly improved, and the image segmentation model's robustness in handling complex scenes and scale changes is enhanced, while reducing the information loss problem caused by extracting features at a single scale.

[0215] In a specific implementation, the segmentation device can perform multi-scale decomposition on the input feature image, segment the input feature image into multiple feature maps of different scales, and capture features of the feature maps of different scales respectively, so as to effectively extract coarse-grained features and fine-grained features, and splice and fuse the coarse-grained features and fine-grained features to obtain a fused feature image.

[0216] It should be noted that feature maps of different scales correspond to different levels of information. Small-scale feature maps (fine-grained) preserve image details (such as edges and textures), while large-scale feature maps (coarse-grained) capture overall structure and semantic information. After splicing, the model can simultaneously utilize both detailed and global information, improving the image segmentation model's understanding of complex images.

[0217] Furthermore, in order to accurately capture feature information at multiple scales, the above step S50 may include:

[0218] Step S501: performing multi-scale decomposition on the input feature image by wavelet decomposition to obtain feature images of multiple scales;

[0219] Step S502: Arranging the feature images of the multiple scales according to the scale order to construct a feature pyramid matrix;

[0220] Step S503 performs multi-scale feature fusion based on the pyramid matrix and the channel attention mechanism to obtain a fused feature image.

[0221] It should be noted that the segmentation device can use Wavelet Transform to decompose the input image at multiple scales to generate feature maps of different scales:

[0222]

[0223] in, represents the wavelet decomposition operation, represents the input feature image, represents a low-pass filter, represents a high-pass filter, represents the decomposition scale, Represents the decomposed feature image.

[0224] It is understandable that the segmentation device decomposes the feature map Arrange them in order from low to high scale, and build a feature pyramid based on the arrangement results , corresponding to global trends (low frequency), mesostructures, and local details (high frequency), respectively.

[0225] It should be noted that the segmentation device realizes the fusion of multi-scale features through weighted splicing and channel attention mechanism. The multi-scale fusion refers to the following formula:

[0226]

[0227]

[0228]

[0229] in, represents the fused feature image, represents the attention weight, 、 and Represent feature maps of different scales, 、 and Represent the weight coefficients, represents the feature concatenation operation, represents the weighted concatenated feature image, and Represents the fully connected layer, which is the Sigmoid activation function. represents a nonlinear function.

[0230] Step S60: inputting the fused feature image into an image segmentation model to perform image segmentation to obtain a segmented feature image.

[0231] It is understandable that the traditional U-Net method is based on single-scale feature extraction and cannot simultaneously capture image characteristics at multiple scales, making it difficult to meet the needs of complex problems. Therefore, this embodiment organically combines the idea of ​​multi-scale feature fusion with the encoding and decoding architecture of U-Net. Through multi-scale encoding and multi-scale decoding operations, it improves the receptive field and feature representation capabilities of the U-Net network segmentation model.

[0232] It should be noted that the image segmentation model is constructed based on the U-Net model, and the image segmentation model includes a multi-scale encoder and a multi-scale decoder.

[0233] Reference Figure 4 , Figure 4This is a schematic diagram of the structure of the multi-scale encoder of this embodiment. The multi-scale encoder includes multiple convolutional layers, pooling layers, and multi-scale fusion layers of different scales. The convolutional layers are used to extract feature information of different scales in the fused feature map. The pooling layers are used to reduce the feature dimension. The multi-scale fusion layer is used to fuse the feature images output by the convolutional layers of each scale.

[0234] Reference Figure 5 , Figure 5 This is a structural diagram of the multi-scale decoder of this embodiment. The multi-scale decoder includes multiple deconvolution layers, skip connection layers, feature fusion layers and upsampling layers of different scales. The deconvolution layer is used to extract feature information of different scales. The skip connection layer is used to fuse the feature information of the multi-scale encoder with the feature information of the multi-scale decoder. The feature fusion layer is used to perform multi-scale feature integration on the feature information output by the multiple deconvolution layers of different scales and the feature information output by the skip connection layer. The upsampling layer is used to restore the feature image.

[0235] It can be understood that in the convolution operation at the encoding end, this embodiment divides the encoder into branches of multiple scales, and each branch uses a convolution layer of a different scale to achieve effective fusion of feature information at each level, thereby obtaining richer and more accurate feature expression. The improved multi-scale encoder of this embodiment is as follows Figure 4 This form of multi-scale encoding achieves the effect of information interaction between channels through the dimensionality reduction operation of 1×1 convolution. Most importantly, each branch uses convolution kernels of different sizes to obtain information at different scales, expanding the receptive field of the convolution operation in the encoding part and extracting richer semantic information.

[0236] This embodiment can introduce multi-scale decoding (MFF) into the decoding process, and reconstruct the multi-scale features through upsampling, convolution and other operations on the multi-scale features to improve the segmentation effect. Decoding is a process of restoring feature information to the segmented image through a series of operations. In this embodiment, the input information of the multi-scale decoder comes from three sources: feature information input by the up-sampled convolution layer, feature information obtained by the jump connection process, and 3×3 deconvolution. Since the traditional u-net cannot achieve very good image segmentation performance, it is considered to use multi-scale feature fusion instead of the 3×3 convolution layer for decoding. The improved multi-scale decoder is as follows: Figure 5As shown. Using 3x3 deconvolution as the core, 1x1 deconvolution and 5x5 deconvolution are introduced. Using 1x1 deconvolution operations, while maintaining the original features, the nonlinear features in the image are effectively enhanced, improving the segmentation effect and generalization ability of the model. On this basis, this embodiment introduces a "jump" connection technology in the decoding process, combining the corresponding hierarchical features in the decoding process with the corresponding hierarchical features in the decoding process to improve the efficiency of feature migration and reuse.

[0237] It should be noted that, referring to Figure 6 , Figure 6 This is a network structure framework diagram of the image segmentation model of this embodiment.

[0238] The downsampling stage of the multi-scale encoder in the image segmentation model includes: the input image first enters the encoder part on the left side of the network for feature extraction and downsampling. Each layer first passes through multiple 3 3 Convolution (Conv) and ReLU activation function extract features, and then use 2 2. Pooling downsamples the feature map, halving its spatial size and doubling its number of channels (e.g., gradually increasing from 64 to 512 or higher). This process is repeated multiple times, gradually capturing abstract semantic information of the image while reducing the spatial dimension.

[0239] When reaching the bottom of the network, the feature map size is the smallest (such as 32 32 or less), but with a maximum number of channels (e.g., 1024). Continuous convolution operations are used here to further extract deep features, forming the bottleneck of the network and focusing on the core semantics of the image.

[0240] The upsampling stage of the multi-scale decoder includes: after passing the bottleneck layer, it enters the decoder part on the right. 2. Deconvolution (transposed convolution) for upsampling, expanding the spatial size of the feature map while reducing the number of channels. After upsampling, the feature map of the corresponding level of the encoder is concatenated with the current decoder feature through a jump connection (concatenate), fusing the underlying details with the high-level semantics. The concatenated feature map is then passed through 3 3. Convolution and ReLU activation functions are used to further fuse information. This process is repeated layer by layer to gradually restore the spatial size of the image.

[0241] Finally, through 1 1. Convolution adjusts the number of channels and outputs a predicted image. The size of this image is the same as the input image, and the number of channels corresponds to the segmentation category or task target (for example, binary classification outputs one channel, multi-classification outputs multiple channels), completing the prediction of image segmentation or related tasks.

[0242] It is understood that this embodiment, by optimizing and improving the traditional U-Net network, can better adapt to the needs of complex image segmentation, especially when processing images with rich textures and complex backgrounds. By combining the texture features of the gray-level co-occurrence matrix with the feature extraction capabilities of multi-scale encoding and decoding, the improved and optimized U-Net network of this embodiment can achieve high-precision image segmentation, providing powerful technical support for image analysis and applications.

[0243] This embodiment collects the original infrared image of the photovoltaic component, and the infrared image is collected by the infrared camera of the drone, and the original infrared image is preprocessed to obtain the target photovoltaic image, and the preprocessing includes geometric correction processing, filtering processing and image enhancement processing, and the grayscale co-occurrence matrix entropy of the target photovoltaic image is calculated, and the texture feature image is obtained based on the grayscale co-occurrence matrix entropy, and the texture feature image is spliced ​​with the grayscale image of the target photovoltaic image to obtain an input feature image, and the input feature image is subjected to multi-scale feature analysis to obtain a fused feature image, and the fused feature image is input into the image segmentation model for image segmentation to obtain a segmented feature image, wherein the image segmentation model is constructed based on the U-Net model The image segmentation model includes a multi-scale encoder and a multi-scale decoder. Since this embodiment preprocesses the original infrared image, it effectively eliminates the geometric distortion caused by angle, perspective relationship, photography and other factors in the original infrared image, greatly reduces the noise in the image, strengthens the key features in the image, and weakens the interference of irrelevant areas. By introducing grayscale symbiosis analysis, it effectively captures the texture features in the image and avoids the loss of key information. Through multi-scale feature analysis, it accurately captures the global structural information and local feature information in the image, reduces the information loss caused by single-scale feature extraction, improves the robustness of the model in dealing with complex scenes and scale changes, and thus improves the model's infrared image segmentation performance of photovoltaic modules.

[0244] In addition, an embodiment of the present invention also proposes a computer-readable storage medium, on which a photovoltaic component infrared image segmentation program is stored. When the photovoltaic component infrared image segmentation program is executed by a processor, the steps of the photovoltaic component infrared image segmentation method described above are implemented.

[0245] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0246] The computer-readable storage medium may be included in the photovoltaic module infrared image segmentation device; or may exist independently without being assembled into the photovoltaic module infrared image segmentation device.

[0247] In addition, an embodiment of the present invention further provides a computer program product, including a photovoltaic assembly infrared image segmentation program, which implements the steps of the photovoltaic assembly infrared image segmentation method described above when executed by a processor.

[0248] The specific implementation of the computer program product of the present invention is basically the same as the embodiments of the above-mentioned photovoltaic module infrared image segmentation method, and will not be repeated here.

[0249] Reference Figure 7 , Figure 7 This is a structural block diagram of an embodiment of a photovoltaic module infrared image segmentation device of the present invention.

[0250] like Figure 7 As shown, the photovoltaic module infrared image segmentation device proposed in the embodiment of the present invention includes:

[0251] The image acquisition module 10 is used to acquire the original infrared image of the photovoltaic module, and the infrared image is acquired by the infrared camera of the drone;

[0252] An image processing module 20 is used to pre-process the original infrared image to obtain a target photovoltaic image, wherein the pre-processing includes geometric correction processing, filtering processing and image enhancement processing;

[0253] A gray-level symbiosis analysis module 30 is configured to calculate the gray-level symbiosis matrix entropy of the target photovoltaic image and obtain a texture feature image based on the gray-level symbiosis matrix entropy;

[0254] A feature stitching module 40 is configured to stitch the texture feature image with the grayscale image of the target photovoltaic image to obtain an input feature image;

[0255] A multi-scale feature analysis module 50 is used to perform multi-scale feature analysis on the input feature image to obtain a fused feature image;

[0256] An image segmentation module 60 is configured to input the fused feature image into an image segmentation model to perform image segmentation and obtain a segmented feature image;

[0257] The image segmentation model is constructed based on the U-Net model, and the image segmentation model includes a multi-scale encoder and a multi-scale decoder;

[0258] The multi-scale encoder includes a plurality of convolutional layers, pooling layers and a multi-scale fusion layer of different scales. The convolutional layers are used to extract feature information of different scales in the fusion feature map. The pooling layers are used to reduce the feature dimension. The multi-scale fusion layer is used to fuse the feature images output by the convolutional layers of each scale.

[0259] The multi-scale decoder includes multiple deconvolution layers of different scales, skip connection layers, feature fusion layers and upsampling layers. The deconvolution layers are used to extract feature information of different scales. The skip connection layers are used to fuse the feature information of the multi-scale encoder with the feature information of the multi-scale decoder. The feature fusion layer is used to perform multi-scale feature integration on the feature information output by the multiple deconvolution layers of different scales and the feature information output by the skip connection layers. The upsampling layer is used to restore the feature image.

[0260] This embodiment collects the original infrared image of the photovoltaic component, and the infrared image is collected by the infrared camera of the drone, and the original infrared image is preprocessed to obtain the target photovoltaic image, and the preprocessing includes geometric correction processing, filtering processing and image enhancement processing, and the grayscale co-occurrence matrix entropy of the target photovoltaic image is calculated, and the texture feature image is obtained based on the grayscale co-occurrence matrix entropy, and the texture feature image is spliced ​​with the grayscale image of the target photovoltaic image to obtain an input feature image, and the input feature image is subjected to multi-scale feature analysis to obtain a fused feature image, and the fused feature image is input into the image segmentation model for image segmentation to obtain a segmented feature image, wherein the image segmentation model is constructed based on the U-Net model The image segmentation model includes a multi-scale encoder and a multi-scale decoder. Since this embodiment preprocesses the original infrared image, it effectively eliminates the geometric distortion caused by angle, perspective relationship, photography and other factors in the original infrared image, greatly reduces the noise in the image, strengthens the key features in the image, and weakens the interference of irrelevant areas. By introducing grayscale symbiosis analysis, it effectively captures the texture features in the image and avoids the loss of key information. Through multi-scale feature analysis, it accurately captures the global structural information and local feature information in the image, reduces the information loss caused by single-scale feature extraction, improves the robustness of the model in dealing with complex scenes and scale changes, and thus improves the model's infrared image segmentation performance of photovoltaic modules.

[0261] The photovoltaic module infrared image segmentation device provided in this application utilizes the photovoltaic module infrared image segmentation method described in the aforementioned embodiments, thereby resolving the technical issues surrounding photovoltaic module infrared image segmentation. Compared to the prior art, the photovoltaic module infrared image segmentation device provided in this application achieves the same beneficial effects as the photovoltaic module infrared image segmentation method described in the aforementioned embodiments. Other technical features of the photovoltaic module infrared image segmentation device are the same as those disclosed in the aforementioned embodiments and are not further detailed here.

[0262] It should be understood that the above is only an example and does not constitute any limitation to the technical solution of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any limitation on this.

[0263] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the embodiment scheme, and no limitation is made here.

[0264] In addition, for technical details not fully described in this embodiment, reference can be made to the photovoltaic module infrared image segmentation method provided in any embodiment of the present invention, and will not be repeated here.

[0265] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0266] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0267] Through the above description of the embodiments, those skilled in the art will clearly understand that the above-mentioned embodiments and methods can be implemented by means of software plus the necessary general-purpose hardware platform. Of course, hardware can also be used, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, a magnetic disk, or an optical disk) and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0268] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A photovoltaic module infrared image segmentation method, characterized in that: The photovoltaic module infrared image segmentation method comprises: Collecting original infrared images of photovoltaic modules, where the infrared images are collected by an infrared camera of a drone; Preprocessing the original infrared image to obtain a target photovoltaic image, wherein the preprocessing includes geometric correction processing, filtering processing, and image enhancement processing; Calculating the gray-level co-occurrence matrix entropy of the target photovoltaic image, and acquiring a texture feature image based on the gray-level co-occurrence matrix entropy; splicing the texture feature image with the grayscale image of the target photovoltaic image to obtain an input feature image; Performing multi-scale feature analysis on the input feature image to obtain a fused feature image; Inputting the fused feature image into an image segmentation model to perform image segmentation to obtain a segmented feature image; The image segmentation model is constructed based on the U-Net model, and the image segmentation model includes a multi-scale encoder and a multi-scale decoder; The multi-scale encoder includes a plurality of convolutional layers, pooling layers and a multi-scale fusion layer of different scales. The convolutional layers are used to extract feature information of different scales in the fusion feature map. The pooling layers are used to reduce the feature dimension. The multi-scale fusion layer is used to fuse the feature images output by the convolutional layers of each scale. The multi-scale decoder includes multiple deconvolution layers of different scales, skip connection layers, feature fusion layers and upsampling layers. The deconvolution layers are used to extract feature information of different scales. The skip connection layers are used to fuse the feature information of the multi-scale encoder with the feature information of the multi-scale decoder. The feature fusion layer is used to perform multi-scale feature integration on the feature information output by the multiple deconvolution layers of different scales and the feature information output by the skip connection layers. The upsampling layer is used to restore the feature image.

2. The photovoltaic module infrared image segmentation method according to claim 1, wherein: The step of calculating the gray level co-occurrence matrix entropy of the target photovoltaic image and acquiring a texture feature image based on the gray level co-occurrence matrix entropy includes: Normalize the pixel grayscale value of each pixel in the target photovoltaic image: in, Representing coordinates The grayscale value of the pixel after normalization, Representing coordinates The original grayscale value of the pixel, represents the maximum grayscale value of the target photovoltaic image, Indicates the number of gray levels; Counting the pixel spacing and pixel direction of each pixel pair based on the normalized pixels, where the pixel pair consists of two normalized pixels; traversing each pixel pair in the target photovoltaic image according to the pixel spacing and the pixel direction to obtain grayscale symbiosis information; A gray-level symbiosis matrix is ​​constructed based on the gray-level symbiosis information, and the gray-level symbiosis matrix entropy is calculated based on the gray-level symbiosis matrix: in, Indicates the target distance and target direction Next, grayscale value and The co-occurrence frequency of pixel pairs; The target entropy feature vector is obtained by concatenating the gray-level co-occurrence matrix entropies at multiple directions and distances in the target photovoltaic image: in, represents the target entropy feature vector, Indicates direction and distance Gray-level co-occurrence matrix entropy under Represents a transpose operation; The target entropy feature vector is mapped to the image space to generate a texture feature image.

3. The photovoltaic module infrared image segmentation method according to claim 2, characterized in that: The performing multi-scale feature analysis on the input feature image to obtain a fused feature image includes: The input feature image is decomposed into multiple scales by wavelet decomposition to obtain feature images of multiple scales: in, represents the wavelet decomposition operation, represents the input feature image, represents a low-pass filter, represents a high-pass filter, represents the decomposition scale, Represents the decomposed feature image; Arrange the feature images of the multiple scales according to the scale order to construct a feature pyramid matrix; Multi-scale feature fusion is performed based on the pyramid matrix and the channel attention mechanism to obtain a fused feature image. The multi-scale fusion refers to the following formula: in, represents the fused feature image, represents the attention weight, 、 and Represent feature maps of different scales, 、 and Represent the weight coefficients, represents the feature concatenation operation, represents the weighted concatenated feature image, and Represents the fully connected layer, which is the Sigmoid activation function. represents a nonlinear function.

4. The photovoltaic module infrared image segmentation method according to any one of claims 1 to 3, characterized in that: The preprocessing of the original infrared image to obtain a target photovoltaic image includes: The original infrared image is subjected to geometric correction processing, wherein the geometric correction processing includes translation transformation, mirror transformation, rotation transformation, similarity transformation, affine transformation and perspective transformation. The translation transformation refers to the following formula: in, and Represent the horizontal coordinate and vertical coordinate in the original infrared image, represents the horizontal translation amount, represents the vertical translation, and Indicates the horizontal and vertical coordinates after translation; The mirror transformation includes horizontal mirror transformation and vertical mirror transformation, referring to the following formula: in, and Respectively represent the horizontal coordinate and vertical coordinate after mirror transformation, Indicates the image width, Indicates the image length; The rotation transformation refers to the following formula: in, and Respectively represent the horizontal coordinate and vertical coordinate after rotation transformation, Indicates the rotation angle; The similarity transformation refers to the following formula: in, and Respectively represent the horizontal coordinate and vertical coordinate after similarity transformation, represents the scaling factor; The affine transformation refers to the following formula: in, and Respectively represent the horizontal coordinate and vertical coordinate after affine transformation, 、 、 and are all linear transformation coefficients; The perspective transformation refers to the following formula: in, and Respectively represent the horizontal coordinate and vertical coordinate after perspective transformation, Represents the transformation element of the perspective transformation matrix; Performing filtering on the original infrared image after geometric correction to obtain a candidate infrared image; Perform image enhancement processing on the candidate infrared image to obtain a target photovoltaic image.

5. The photovoltaic module infrared image segmentation method according to claim 4, characterized in that: The filtering process is performed on the original infrared image after the geometric correction process to obtain a candidate infrared image, including: The original infrared image after geometric correction is subjected to median filtering to obtain an initial filtered image. The median filtering process refers to the following formula: in, represents the initial filtered image obtained after median filtering, Indicates that the filter window becomes longer, Represents the first pixel in the sorted pixel value sequence elements, Represents a sorted sequence of pixel values; The initial filtered image is subjected to Gaussian filtering to obtain a candidate infrared image. The Gaussian filtering is performed according to the following formula: in, Represents the pixel value of the filtered image at coordinates, represents the Gaussian weight function, represents the standard deviation of the Gaussian distribution, and Represents the horizontal and vertical offsets respectively. Indicates the initial filtered image in The pixel value at position, Indicated in pixels Center the sphere window size.

6. The photovoltaic module infrared image segmentation method according to claim 5, characterized in that: The performing image enhancement processing on the candidate infrared image to obtain a target photovoltaic image includes: Analyze the pixel grayscale distribution of the candidate infrared image to determine the number of pixels at each grayscale level, and generate an original histogram based on the number of pixels at each grayscale level: in, represents the number of pixels with gray level in the candidate infrared image, Indicates the Gray levels; The original histograms are accumulated to generate a cumulative histogram: in, Indicates the cumulative distribution function, which is used to accumulate gray levels from 0 to The number of pixels; Grayscale mapping is performed on the cumulative histogram, and the mapped grayscale values ​​are quantized to obtain a target photovoltaic image: in, represents the grayscale range, Indicates the total number of pixels in the image. Represents the grayscale value after mapping, Represents the rounding function.

7. A photovoltaic module infrared image segmentation device, characterized in that: The photovoltaic assembly infrared image segmentation device comprises: An image acquisition module is used to acquire original infrared images of photovoltaic modules, where the infrared images are acquired by an infrared camera of a drone; An image processing module is used to pre-process the original infrared image to obtain a target photovoltaic image, wherein the pre-processing includes geometric correction processing, filtering processing and image enhancement processing; a gray-level symbiosis analysis module, configured to calculate the gray-level symbiosis matrix entropy of the target photovoltaic image and obtain a texture feature image based on the gray-level symbiosis matrix entropy; a feature stitching module, configured to stitch the texture feature image with the grayscale image of the target photovoltaic image to obtain an input feature image; A multi-scale feature analysis module, configured to perform multi-scale feature analysis on the input feature image to obtain a fused feature image; An image segmentation module is used to input the fused feature image into an image segmentation model to perform image segmentation and obtain a segmented feature image; The image segmentation model is constructed based on the U-Net model, and the image segmentation model includes a multi-scale encoder and a multi-scale decoder; The multi-scale encoder includes a plurality of convolutional layers, pooling layers and a multi-scale fusion layer of different scales. The convolutional layers are used to extract feature information of different scales in the fusion feature map. The pooling layers are used to reduce the feature dimension. The multi-scale fusion layer is used to fuse the feature images output by the convolutional layers of each scale. The multi-scale decoder includes multiple deconvolution layers of different scales, skip connection layers, feature fusion layers and upsampling layers. The deconvolution layers are used to extract feature information of different scales. The skip connection layers are used to fuse the feature information of the multi-scale encoder with the feature information of the multi-scale decoder. The feature fusion layer is used to perform multi-scale feature integration on the feature information output by the multiple deconvolution layers of different scales and the feature information output by the skip connection layers. The upsampling layer is used to restore the feature image.

8. A photovoltaic module infrared image segmentation device, characterized in that: The photovoltaic assembly infrared image segmentation device includes: a memory, a processor, and a photovoltaic assembly infrared image segmentation program stored in the memory and executable on the processor. The photovoltaic assembly infrared image segmentation program is configured to implement the photovoltaic assembly infrared image segmentation method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a photovoltaic assembly infrared image segmentation program, and when the photovoltaic assembly infrared image segmentation program is executed by a processor, the photovoltaic assembly infrared image segmentation method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product includes a photovoltaic assembly infrared image segmentation program, which implements the steps of the photovoltaic assembly infrared image segmentation method according to any one of claims 1 to 6 when executed by a processor.

Images