Method and device for determining fault area of photovoltaic image, and storage medium

Abstract

The invention discloses a method and device for determining a fault area of a photovoltaic image, a storage medium, an electronic device and a computer program product, and relates to the field of image processing, and the method comprises the steps: obtaining fault image data of a target photovoltaic module, the fault image data comprising an image of a photovoltaic panel area of the target photovoltaic module; segmenting the fault image data according to the multi-dimensional features of the fault image data by adopting a preset semantic segmentation model so as to determine a photovoltaic panel area in the fault image data and position information of a preset colorimetric strip; determining the temperature distribution condition of the photovoltaic panel area based on the position information of the colorimetric strip; and determining a fault area in the photovoltaic panel area based on the temperature distribution condition of the photovoltaic panel area. Semantic segmentation can be accurately carried out on fault image data to obtain a photovoltaic panel area, so that the accuracy of subsequent fault diagnosis is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular, to a method and device for determining a fault area of ​​a photovoltaic image, a storage medium, an electronic device, and a computer program product. Background Art

[0002] With the rapid development of the photovoltaic industry in recent years, the inspection and maintenance of photovoltaic power stations has become an integral part of the industry.

[0003] In the related technology, drones equipped with infrared cameras / infrared thermal imagers are used for inspection and diagnosis. Photovoltaic modules are photographed by the infrared cameras / infrared thermal imagers, and then the temperature difference between the hot spots of each module in the photo and other positions is calculated to find out faults such as diode damage and hot spots, thereby realizing fault analysis of photovoltaic modules.

[0004] However, since photovoltaic modules are usually located in complex environments with varying conditions, there may be overlap between the photovoltaic area and the background area in the image, resulting in a large amount of interference information in the infrared image. Therefore, the fault analysis results obtained using related technologies are not accurate enough. Summary of the invention

[0005] Embodiments of the present application provide a method and device for determining a fault area of ​​a photovoltaic image, a storage medium, an electronic device, and a computer program product.

[0006] According to one aspect of an embodiment of the present application, a method for determining a fault area of ​​a photovoltaic image is provided, the method comprising: acquiring fault image data of a target photovoltaic component, wherein the fault image data comprises an image of a photovoltaic panel area of ​​the target photovoltaic component; using a preset semantic segmentation model to segment the fault image data according to features of multiple dimensions of the fault image data to determine the photovoltaic panel area in the fault image data, and position information of a preset colorimetric bar; determining a temperature distribution of the photovoltaic panel area based on the position information of the colorimetric bar; and determining a fault area in the photovoltaic panel area based on the temperature distribution of the photovoltaic panel area.

[0007] In an exemplary embodiment, the preset semantic segmentation model is used to segment the fault image data according to the features of multiple dimensions of the fault image data to determine the photovoltaic panel area in the fault image data, including: using the full-dimensional dynamic convolution module in the segmentation model to extract features of the fault image data to obtain image features in multiple dimensions of the fault image data; using the hybrid pooling module in the segmentation model to expand the image features in multiple dimensions of the fault image data, wherein the hybrid pooling module is used to perform at least two types of pooling operations on the image features; using the spatial transformation network in the segmentation model to perform spatial transformation processing on the image features in multiple dimensions after expansion; using the semantic segmentation model to perform semantic segmentation on the image features after the spatial transformation processing to determine the photovoltaic panel area in the fault image data.

[0008] In an exemplary embodiment, before using a preset semantic segmentation model to segment the fault image data according to features of multiple dimensions of the fault image data to determine the photovoltaic panel area in the fault image data, the method also includes: acquiring multiple sample image data of photovoltaic components; preprocessing the multiple sample image data; constructing a data set based on the preprocessed multiple sample image data; constructing an initial semantic segmentation model based on a deep convolutional neural network architecture; and using the data set to train the initial semantic segmentation model to obtain the preset semantic segmentation model.

[0009] In an exemplary embodiment, the preprocessing of the multiple sample image data includes: grayscale processing the multiple sample image data to obtain multiple grayscale image data; filtering the grayscale image data; and image enhancement processing the grayscale image data after filtering.

[0010] In an exemplary embodiment, constructing a data set based on the preprocessed multiple sample image data includes: annotating the multiple sample image data to mark the feature areas in the multiple sample image data; and performing image amplification on the annotated multiple sample image data to obtain the data set.

[0011] In an exemplary embodiment, the method of determining the temperature distribution of the photovoltaic panel area based on the position information of the colorimetric strip includes: determining the temperature maximum area in the photovoltaic panel area according to the position information of the colorimetric strip; identifying the temperature maximum area to determine the temperature value of the temperature maximum area; determining the temperature values ​​of each area in the photovoltaic panel area according to the temperature value of the temperature maximum area to determine the temperature distribution of the photovoltaic panel area.

[0012] According to another aspect of an embodiment of the present application, a device for determining a photovoltaic panel area is also provided, the device comprising:

[0013] An image acquisition module, used to acquire fault image data of a target photovoltaic assembly, wherein the fault image data includes an image of a photovoltaic panel area of ​​the target photovoltaic assembly;

[0014] An image segmentation module, used to segment the fault image data according to the features of multiple dimensions of the fault image data using a preset semantic segmentation model, so as to determine the photovoltaic panel area in the fault image data and the position information of the preset colorimetric strip;

[0015] A temperature determination module, used to determine the temperature distribution of the photovoltaic panel area based on the position information of the colorimetric strip;

[0016] The fault determination module is used to determine a fault area in the photovoltaic panel area based on the temperature distribution of the photovoltaic panel area.

[0017] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for determining a fault area of ​​a photovoltaic image when running.

[0018] According to another aspect of an embodiment of the present application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for determining the fault area of ​​the photovoltaic image through the computer program.

[0019] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the method described in each embodiment of the present application are implemented.

[0020] The method for determining the fault area of ​​the above-mentioned photovoltaic image first obtains the fault image data of the target photovoltaic component. Then, a preset semantic segmentation model is used to segment the fault image data according to the features of multiple dimensions of the fault image data to determine the photovoltaic panel area in the fault image data and the position information of the preset colorimetric strip. Next, based on the position information of the colorimetric strip, the temperature distribution of the photovoltaic panel area is determined. Finally, based on the temperature distribution of the photovoltaic panel area, the fault area is determined in the photovoltaic panel area. In this way, the features of the fault image data can be captured from multiple dimensions, which can better adapt to the complex content of the fault image data, improve the accuracy of feature capture, improve the accuracy of image segmentation, accurately perform semantic segmentation on the fault image data, and obtain the photovoltaic panel area, so as to improve the accuracy of subsequent fault diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] Figure 1 is a hardware structure block diagram of a method for determining a fault area of ​​a photovoltaic image according to an embodiment of the present application;

[0024] Figure 2 is a flow chart of a method for determining a fault area of ​​a photovoltaic image according to an embodiment of the present application;

[0025] Figure 3 is a flow chart of a photovoltaic image processing method according to an embodiment of the present application;

[0026] Figure 4 is a schematic diagram of processing an image by a semantic segmentation model according to an embodiment of the present application;

[0027] Figure 5 is a flowchart of a method for constructing a semantic model according to an embodiment of the present application;

[0028] Figure 6 is a flow chart of an image preprocessing method according to an embodiment of the present application;

[0029] Figure 7 is a flow chart of a method for preparing an image data set according to an embodiment of the present application;

[0030] Figure 8 is a flow chart of a method for determining the temperature distribution of the photovoltaic panel area according to an embodiment of the present application;

[0031] Fig. 9 It is a structural block diagram of a device for determining a fault area of ​​a photovoltaic image according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0034] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a computer terminal of a method for determining a fault area of ​​a photovoltaic image according to an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 The computer terminal includes a processor 102 (only one is shown in the figure) (the processor 102 may include but is not limited to a microprocessor (Microprocessor Unit, MPU for short) or a programmable logic device (PLD for short)) and a memory 104 for storing data. In an exemplary embodiment, the computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 A different configuration with more features is shown.

[0035] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the fault area of ​​the photovoltaic image in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0037] In view of the above problems, a method for determining a fault area of ​​a photovoltaic image is provided in this embodiment. Figure 2 : is a flowchart of an optional method for determining a fault area of ​​a photovoltaic image according to an embodiment of the present application, the process includes the following steps S200-S230:

[0038] Step S200, acquiring fault image data of a target photovoltaic module.

[0039] The fault image data includes an image of a photovoltaic panel area of ​​a target photovoltaic assembly.

[0040] Step S210 , using a preset semantic segmentation model, segmenting the fault image data according to features of multiple dimensions of the fault image data, so as to determine the photovoltaic panel area in the fault image data and the position information of the preset colorimetric strip.

[0041] The colorimetric bar is used to identify the color of the image, which can be different colors or grayscales. With the help of the colorimetric bar, the temperature of different areas of the image can be easily identified. The semantic segmentation model can accurately segment the colorimetric bar and photovoltaic components (photovoltaic panel area) from the image.

[0042] Step S220, determining the temperature distribution of the photovoltaic panel area based on the position information of the colorimetric strip.

[0043] Among them, color bars of different colors / gray levels can represent different temperatures, and the temperature of each position can be determined based on the color / gray level of the color bar at each position.

[0044] Step S230 , determining a fault area in the photovoltaic panel area based on the temperature distribution in the photovoltaic panel area.

[0045] Among them, the segmented colorimetric strips can be used to locate and cut out the temperature maximum area, and then the pre-trained Paddle OCR toolkit can be used to realize temperature recognition, determine the highest temperature and the lowest temperature, and then determine the temperature of each area. Next, the segmented photovoltaic panel area can be mainly separated into single-phase areas, and then the hotspot information of each area and the temperature difference information between different areas can be determined to facilitate fault diagnosis analysis (for example, temperature exceeding a certain threshold represents a fault, and temperature difference greater than a certain threshold represents a fault). Finally, the fault area can be determined.

[0046] In this embodiment, the fault image data of the target photovoltaic component is first acquired. Then, a preset semantic segmentation model is used to segment the fault image data according to the features of multiple dimensions of the fault image data to determine the photovoltaic panel area in the fault image data and the position information of the preset colorimetric strip. Next, based on the position information of the colorimetric strip, the temperature distribution of the photovoltaic panel area is determined. Finally, based on the temperature distribution of the photovoltaic panel area, the fault area is determined in the photovoltaic panel area. In this way, the features of the fault image data can be captured from multiple dimensions, which can better adapt to the complex content of the fault image data, improve the accuracy of feature capture, improve the accuracy of image segmentation, accurately perform semantic segmentation on the fault image data, and obtain the photovoltaic panel area, so as to improve the accuracy of subsequent fault diagnosis.

[0047] In one embodiment, Figure 3 As shown, step S210, using a preset semantic segmentation model, segments the fault image data according to the features of multiple dimensions of the fault image data to determine the photovoltaic panel area in the fault image data, including: steps S300-330.

[0048] Step S300: Use the full-dimensional dynamic convolution module in the segmentation model to perform feature extraction on the fault image data to obtain image features in multiple dimensions of the fault image data.

[0049] Among them, dynamic convolution introduces a dynamic attention mechanism on multiple different convolution kernels to increase the size and capacity of the network. In the related art, a single attention is used, and the input has the same attention value for the output convolution kernel, but the other three dimensions (about the spatial size, the number of input channels and the number of output channels) of the convolution kernel's spatial dimension, input channel dimension and output channel dimension) are ignored. The concept of dynamic convolution is extended to multiple dimensions, thereby achieving flexible and adaptive feature extraction. The full-dimensional dynamic convolution module uses a multi-dimensional attention mechanism to learn four types of attention of the convolution kernel in parallel along the four dimensions of the kernel space, and gradually applies these attentions to the corresponding convolution kernels, which greatly enhances the feature extraction capability of the basic convolution operation of the Convolutional Neural Networks (CNN). When performing convolution operations, the full-dimensional dynamic convolution uses a multi-dimensional attention mechanism to model multiple dimensions (for example, four dimensions) of the convolution kernel, which can enhance the feature extraction capability of the basic convolution operation and reduce additional parameters. The full-dimensional dynamic convolution module is used to replace the 3x3 convolution of the original network to enhance the feature extraction capability of the network, thereby improving the segmentation performance of the network.

[0050] Furthermore, the full-dimensional dynamic convolution module dynamically adjusts the size and shape of the convolution kernel during the feature extraction process to adapt to the complex structure of photovoltaic infrared images. This dynamic adjustment capability enables the model to better adapt to changes in different lighting, temperature, and component states, and maintain high segmentation accuracy even in harsh environments. In the intelligent photovoltaic monitoring system, this flexibility can ensure that components can be accurately identified even when the weather changes or the component surface is irregular, providing stable data support for the system.

[0051] Step S310: Use the hybrid pooling module in the segmentation model to expand the image features of the fault image data in multiple dimensions.

[0052] Among them, the hybrid pooling module is used to perform at least two types of pooling operations on image features. Specifically, at least two types of pooling operations may include standard pooling and stripe pooling. On the one hand, two standard pooling operations are performed on the image data to obtain the pooled image, and then the original input is added, a total of three feature images, which are respectively subjected to 2D convolution, and the pooling path is upsampled, and then the three feature images are fused, and then subjected to a 2D convolution. On the other hand, stripe pooling is performed in the horizontal and vertical directions on the image data to obtain a one-dimensional vector, which is then upsampled to the original resolution after 1D convolution, and finally subjected to a 2D convolution. The original network's 2x2 maximum pooling is completely replaced with a hybrid pooling module, which can expand the network's receptive field, while collecting contextual information, improving the model's sensitivity to the length direction, and focusing on the global important features of the image, thereby enhancing the network's feature extraction capabilities.

[0053] Step S320: Use the spatial transformation network in the segmentation model to perform spatial transformation processing on the image features in multiple dimensions after expansion.

[0054] Among them, the spatial transformation network does not require the calibration of key points, and can adaptively transform and align data spatially (including translation, scaling, rotation, and other geometric transformations) according to classification or other tasks. In the case of large spatial differences in input data, this network can be added to the existing convolutional network to improve classification accuracy.

[0055] Step S330, using a semantic segmentation model to perform semantic segmentation on the image features after the spatial transformation processing, and determine the photovoltaic panel area in the fault image data.

[0056] The semantic segmentation model can be built on the basis of the Unet deep convolutional neural network model that is good at processing large-scale image segmentation. The 3x3 convolution is replaced with a full-dimensional dynamic convolution module to enhance the network's ability to capture multi-dimensional feature information. The 2x2 maximum pooling layer is replaced with a hybrid pooling module to optimize the efficiency and accuracy of feature extraction. The spatial transformation network is embedded in the network to further improve the network's robustness to image spatial transformation.

[0057] For example, Figure 4 As shown, it is a schematic diagram of the semantic segmentation model processing an image. The relevant content has been introduced in this embodiment and will not be repeated here.

[0058] In this embodiment, a full-dimensional dynamic convolution module can maintain high segmentation accuracy even in harsh environments. In the intelligent photovoltaic monitoring system, this flexibility can ensure that components can be accurately identified even when the weather changes or the surface of the component is irregular, providing stable data support for the system. The features of the photovoltaic infrared image are further processed by the hybrid pooling module, which combines standard pooling and stripe pooling to expand the network's receptive field and collect contextual information. This pooling strategy not only helps to reduce computational complexity, but also retains key stripe information, which is crucial for identifying subtle differences in photovoltaic components, especially when the components are aged or partially blocked, and can provide richer diagnostic evidence. By adding a spatial transformation network to the original network, the input image can be spatially transformed, which enables the model to more robustly handle changes in different postures, scales, and deformations, thereby improving the accuracy of semantic segmentation. In addition, by optimizing the original network structure and reducing the number of network parameters, the overfitting problem can be effectively alleviated.

[0059] In one embodiment, Figure 5 As shown, before step S220, the method for determining the fault area of ​​the photovoltaic image further includes steps S500-S540. Among them:

[0060] Step S500, acquiring a plurality of sample image data of a photovoltaic module.

[0061] Step S510: pre-processing a plurality of sample image data.

[0062] Step S520: constructing a data set based on the preprocessed plurality of sample image data.

[0063] Step S530, constructing an initial semantic segmentation model based on a deep convolutional neural network architecture.

[0064] Among them, the deep convolutional neural network architecture is the U-Net convolutional neural network architecture. The U-Net convolutional neural network is divided into two parts, which can be referred to Figure 4 As shown in the figure, the entire network is like the letter U. The left part of the network is responsible for feature extraction. As the network layer deepens, the number of image channels gradually increases, while the size gradually decreases. After two convolutions to extract features, the image size is reduced after a maximum pooling. The right part of the network is responsible for feature restoration. As the network layer deepens, the number of image channels gradually decreases, while the size gradually increases. After one deconvolution to enlarge the image size, the image is extracted after two convolutions. The entire network is actually a codec. It should be noted that Figure 4The arrow in the U-shaped structure indicates the process of merging the feature image obtained in the encoding process with the feature image obtained in the decoding process. During the encoding process, some information is lost due to the network's maximum pooling (Max Pooling) and two-dimensional convolution (Conv2D). Therefore, during decoding, it is necessary to add the corresponding encoding layer information to the feature image of the decoding layer.

[0065] Step S540: Use the data set to train an initial semantic segmentation model to obtain a preset semantic segmentation model.

[0066] Among them, the U-Net network can be used to train the data set and the label data, and the photovoltaic panel area of ​​the image can be automatically detected by using the trained model.

[0067] In this embodiment, by training the neural network model with the preprocessed data set, a preset semantic segmentation model can be obtained, which facilitates identifying the photovoltaic panel area in the detection image through the preset semantic segmentation model to achieve semantic segmentation of the image.

[0068] In one embodiment, Figure 6 As shown, step S510 is to pre-process a plurality of sample image data, including steps S600-S620.

[0069] Step S600 , grayscale processing is performed on a plurality of sample image data to obtain a plurality of grayscale image data.

[0070] Among them, due to the rich color information in photovoltaic infrared images and the large amount of data during image calculation, color images are usually converted into grayscale images during image processing. During the shooting of infrared video materials, the change in light intensity will cause a large difference in the grayscale distribution of video frame images. The grayscale of some frame images is too concentrated, and the image contrast is low, which affects the segmentation of photovoltaic components. Therefore, grayscale image normalization is used, that is, the grayscale stretching method is used to spread the grayscale distribution in the original image to an image with the entire grayscale level. The weighted average method can be used to grayscale the image.

[0071] Step S610, filtering the grayscale image data.

[0072] Among them, the infrared image that does not contain photovoltaic components or the segments in which photovoltaic components account for less than 50% of the overall image can be removed, and the image can be denoised using a Gaussian filtering method, or the image can be denoised using a bilateral filtering method.

[0073] Step S620: performing image enhancement processing on the grayscale image data after the filtering processing.

[0074] Among them, an improved Gaussian-Laplacian pyramid enhancement algorithm may be used to perform image enhancement processing.

[0075] In this embodiment, by graying, filtering, and enhancing the image, the quality of the image can be improved, thereby facilitating the provision of high-quality image data for subsequent image segmentation and improving the accuracy of image segmentation.

[0076] In one embodiment, the steps Figure 7 As shown, step S520, constructing a data set based on the preprocessed multiple sample image data, includes: steps S700-S710.

[0077] Step S700 , performing labeling processing on a plurality of sample image data, and labeling characteristic regions in the plurality of sample image data.

[0078] Among them, labels may be added to the image sample image data to mark out the background part and the feature part.

[0079] Step S710, performing image amplification processing on the multiple sample image data after the labeling processing to obtain a data set.

[0080] Among them, the image can be enlarged by using geometric transformations such as rotation and flipping.

[0081] In this embodiment, a data set of sample images is constructed by performing image annotation and image amplification processing, so as to use the data set to subsequently train a neural network model and improve the performance of the model.

[0082] In one embodiment, Figure 8 As shown, step S220, based on the position information of the colorimetric strip, determines the temperature distribution of the photovoltaic panel area. It includes: steps S800-S820. Among them:

[0083] Step S800: determining the maximum temperature region in the photovoltaic panel region according to the position information of the colorimetric strip.

[0084] Step S810, identifying the maximum temperature region and determining the temperature value of the maximum temperature region.

[0085] Among them, the pre-trained Paddle OCR toolkit can be used to recognize temperature and determine the maximum and minimum temperatures.

[0086] Step S820, determining the temperature value of each area in the photovoltaic panel area according to the temperature value of the maximum temperature area, so as to determine the temperature distribution of the photovoltaic panel area.

[0087] After the temperature value of the maximum temperature region is determined, other regions can be obtained by linear interpolation based on the temperature value of the maximum temperature region, so that the temperature values ​​of all regions can be determined.

[0088] In this embodiment, the colorimetric strip is used to locate and extract the temperature maximum area, and then the maximum temperature and the minimum temperature are determined, and then the temperature of each area is determined. Next, the temperature value of each area in the photovoltaic panel area can be determined to determine the temperature distribution of the photovoltaic panel area, so as to facilitate subsequent fault diagnosis analysis (for example, temperature exceeding a certain threshold represents a fault, and temperature difference greater than a certain threshold represents a fault), and finally the fault area can be determined.

[0089] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0090] In this embodiment, a device for determining a fault area of ​​a photovoltaic image is also provided, and the device for determining a fault area of ​​the photovoltaic image is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0091] Fig. 9 is a structural block diagram of an optional device for determining a fault area of ​​a photovoltaic image according to an embodiment of the present application; Fig. 9 As shown, including:

[0092] An image acquisition module 901 is used to acquire fault image data of a target photovoltaic assembly, wherein the fault image data includes an image of a photovoltaic panel area of ​​the target photovoltaic assembly;

[0093] An image segmentation module 902 is used to segment the fault image data according to the features of multiple dimensions of the fault image data using a preset semantic segmentation model, so as to determine the photovoltaic panel area in the fault image data and the position information of the preset colorimetric strip;

[0094] The temperature determination module 903 is used to determine the temperature distribution of the photovoltaic panel area based on the position information of the colorimetric strip;

[0095] The fault determination module 904 is used to determine a fault area in the photovoltaic panel area based on the temperature distribution of the photovoltaic panel area.

[0096] Through the above-mentioned device, the fault image data of the target photovoltaic component is first obtained. Then, a preset semantic segmentation model is used to segment the fault image data according to the features of multiple dimensions of the fault image data to determine the photovoltaic panel area in the fault image data and the position information of the preset colorimetric strip. Next, based on the position information of the colorimetric strip, the temperature distribution of the photovoltaic panel area is determined. Finally, based on the temperature distribution of the photovoltaic panel area, the fault area is determined in the photovoltaic panel area. In this way, the features of the fault image data can be captured from multiple dimensions, which can better adapt to the complex content of the fault image data, improve the accuracy of feature capture, improve the accuracy of image segmentation, accurately perform semantic segmentation on the fault image data, and obtain the photovoltaic panel area, so as to improve the accuracy of subsequent fault diagnosis.

[0097] In an exemplary embodiment, the image segmentation module 902 further includes:

[0098] A feature extraction unit, used to extract features from the fault image data using a full-dimensional dynamic convolution module in the segmentation model, so as to obtain image features of the fault image data in multiple dimensions;

[0099] An expansion unit, used to expand the image features of the fault image data in multiple dimensions by using a hybrid pooling module in the segmentation model, wherein the hybrid pooling module is used to perform at least two types of pooling operations on the image features;

[0100] A spatial transformation unit, used for performing spatial transformation processing on the image features in multiple dimensions after expansion by using the spatial transformation network in the segmentation model;

[0101] The semantic segmentation unit is used to perform semantic segmentation on the image features after the spatial transformation processing by using the semantic segmentation model, and determine the photovoltaic panel area in the fault image data.

[0102] In an exemplary embodiment, the above-mentioned device is also used to obtain multiple sample image data of photovoltaic components; preprocess the multiple sample image data; construct a data set based on the preprocessed multiple sample image data; construct an initial semantic segmentation model based on a deep convolutional neural network architecture; use the data set to train the initial semantic segmentation model to obtain a preset semantic segmentation model.

[0103] In an exemplary embodiment, the above device further includes:

[0104] A grayscale processing module is used to perform grayscale processing on a plurality of sample image data to obtain a plurality of grayscale image data;

[0105] A filtering processing module, used for filtering grayscale image data;

[0106] The enhancement processing module is used to perform image enhancement processing on the grayscale image data after filtering processing.

[0107] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0108] In an exemplary embodiment, the above-mentioned device is also used to perform labeling processing on multiple sample image data, and label the characteristic areas in the multiple sample image data; perform image amplification processing on the multiple sample image data after labeling processing to obtain a data set.

[0109] In an exemplary embodiment, the temperature determination module 903 is also used to determine the maximum temperature area in the photovoltaic panel area based on the position information of the colorimetric strip; identify the maximum temperature area and determine the temperature value of the maximum temperature area; determine the temperature values ​​of each area in the photovoltaic panel area based on the temperature value of the maximum temperature area to determine the temperature distribution of the photovoltaic panel area.

[0110] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0111] S1, acquiring fault image data of a target photovoltaic assembly, wherein the fault image data includes an image of a photovoltaic panel area of ​​the target photovoltaic assembly.

[0112] S2, using a preset semantic segmentation model to segment the fault image data according to features of multiple dimensions of the fault image data, so as to determine the photovoltaic panel area in the fault image data and the position information of the preset colorimetric strip.

[0113] S3, determining the temperature distribution of the photovoltaic panel area based on the position information of the colorimetric strip.

[0114] S4, determining a fault area in the photovoltaic panel area based on the temperature distribution in the photovoltaic panel area.

[0115] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0116] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0117] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0118] S1, acquiring fault image data of a target photovoltaic assembly, wherein the fault image data includes an image of a photovoltaic panel area of ​​the target photovoltaic assembly.

[0119] S2, using a preset semantic segmentation model to segment the fault image data according to features of multiple dimensions of the fault image data, so as to determine the photovoltaic panel area in the fault image data and the position information of the preset colorimetric strip.

[0120] S3, determining the temperature distribution of the photovoltaic panel area based on the position information of the colorimetric strip.

[0121] S4, determining a fault area in the photovoltaic panel area based on the temperature distribution in the photovoltaic panel area.

[0122] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0123] An embodiment of the present application further provides a computer program product, comprising a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program product, and when the computer program is executed by a processor, the steps of the method described in each embodiment of the present application are implemented.

[0124] Optionally, in this embodiment, the above computer program may be configured to implement the following steps when executed by a processor:

[0125] S1, acquiring fault image data of a target photovoltaic component, wherein the fault image data includes an image of a photovoltaic panel area of ​​the target photovoltaic component.

[0126] S2, using a preset semantic segmentation model to segment the fault image data according to features of multiple dimensions of the fault image data, so as to determine the photovoltaic panel area in the fault image data and the position information of the preset colorimetric strip.

[0127] S3, determining the temperature distribution of the photovoltaic panel area based on the position information of the colorimetric strip.

[0128] S4, determining a fault area in the photovoltaic panel area based on the temperature distribution of the photovoltaic panel area.

[0129] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0130] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0131] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining a fault area of ​​a photovoltaic image, characterized in that: The method comprises: Acquire fault image data of a target photovoltaic assembly, wherein the fault image data includes an image of a photovoltaic panel area of ​​the target photovoltaic assembly; Using a preset semantic segmentation model, segmenting the fault image data according to features of multiple dimensions of the fault image data, so as to determine the photovoltaic panel area in the fault image data and the position information of the preset colorimetric strip; Determining the temperature distribution of the photovoltaic panel area based on the position information of the colorimetric strip; Based on the temperature distribution of the photovoltaic panel area, a fault area is determined in the photovoltaic panel area.

2. The method according to claim 1, characterized in that The method of adopting a preset semantic segmentation model to segment the fault image data according to features of multiple dimensions of the fault image data to determine a photovoltaic panel area in the fault image data includes: Using the full-dimensional dynamic convolution module in the segmentation model to perform feature extraction on the fault image data to obtain image features of the fault image data in multiple dimensions; Using a hybrid pooling module in the segmentation model to expand the image features of the fault image data in multiple dimensions, wherein the hybrid pooling module is used to perform at least two types of pooling operations on the image features; Using the spatial transformation network in the segmentation model to perform spatial transformation processing on the image features in multiple dimensions after expansion; The semantic segmentation model is used to perform semantic segmentation on the image features after the spatial transformation processing to determine the photovoltaic panel area in the fault image data.

3. The method according to claim 1, characterized in that Before adopting a preset semantic segmentation model to segment the fault image data according to features of multiple dimensions of the fault image data to determine the photovoltaic panel area in the fault image data, the method further includes: Acquire multiple sample image data of photovoltaic modules; Preprocessing the plurality of sample image data; Building a data set based on a plurality of preprocessed sample image data; Build an initial semantic segmentation model based on a deep convolutional neural network architecture; The initial semantic segmentation model is trained using the data set to obtain the preset semantic segmentation model.

4. The method according to claim 3, characterized in that The preprocessing of the plurality of sample image data comprises: Performing grayscale processing on the plurality of sample image data to obtain a plurality of grayscale image data; Performing filtering processing on the grayscale image data; Perform image enhancement processing on the grayscale image data after filtering.

5. The method according to claim 3, characterized in that: The step of constructing a data set based on the preprocessed plurality of sample image data comprises: Performing labeling processing on the plurality of sample image data to label characteristic regions in the plurality of sample image data; The image amplification process is performed on the multiple sample image data after the labeling process to obtain the data set.

6. The method according to claim 1, characterized in that The determining of the temperature distribution of the photovoltaic panel area based on the position information of the colorimetric strip includes: Determine the maximum temperature area in the photovoltaic panel area according to the position information of the colorimetric strip; Identifying the maximum temperature region and determining the temperature value of the maximum temperature region; According to the temperature value of the maximum temperature area, the temperature value of each area in the photovoltaic panel area is determined to determine the temperature distribution of the photovoltaic panel area.

7. A device for determining a photovoltaic panel area, characterized in that: The device comprises: An image acquisition module, used to acquire fault image data of a target photovoltaic assembly, wherein the fault image data includes an image of a photovoltaic panel area of ​​the target photovoltaic assembly; An image segmentation module, used to segment the fault image data according to the features of multiple dimensions of the fault image data using a preset semantic segmentation model, so as to determine the photovoltaic panel area in the fault image data and the position information of the preset colorimetric strip; A temperature determination module, used to determine the temperature distribution of the photovoltaic panel area based on the position information of the colorimetric strip; The fault determination module is used to determine a fault area in the photovoltaic panel area based on the temperature distribution of the photovoltaic panel area.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 6 when executed.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 6 through the computer program.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images