Photovoltaic inspection method and device, electronic equipment and storage medium

Abstract

The invention discloses a photovoltaic inspection method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a target infrared image and a target visible light image of a photovoltaic module to be inspected, and extracting a first feature of the target infrared image and a second feature of the target visible light image; performing image fusion on the target infrared image and the target visible light image according to the first feature and the second feature to obtain a fused photovoltaic module image; extracting a photovoltaic panel area from the fused photovoltaic module image by using a pre-trained photovoltaic fault classification model, and determining a three-dimensional attention weight for features of the photovoltaic panel area by using an attention mechanism of the pre-trained photovoltaic fault classification model; and determining a fault classification result for the photovoltaic panel area according to the three-dimensional attention weight so as to realize inspection of the to-be-inspected photovoltaic module. According to the technical scheme of the invention, the fault type of the photovoltaic module can be identified more accurately and meticulously.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a photovoltaic inspection method, device, electronic equipment and storage medium. Background Art

[0002] With the development of photovoltaic power generation, drone inspection plays an increasingly important role. Traditional drone-based photovoltaic inspection strategies mainly rely on photovoltaic infrared images. Faulty components will show obvious bright spots on infrared images due to abnormal heating. Power station staff can judge whether the target component has defects based on the different imaging characteristics of faulty components and normal components on infrared images. However, with the rapid development of target detection technology, the requirements for photovoltaic component fault detection are getting higher and higher. Simply identifying faulty components cannot meet further requirements. Summary of the invention

[0003] The present invention provides a photovoltaic inspection method, device, electronic equipment and storage medium to more accurately and meticulously identify the fault type of a photovoltaic component.

[0004] According to one aspect of the present invention, a photovoltaic inspection method is provided, the method comprising:

[0005] Acquire a target infrared image and a target visible light image of the photovoltaic assembly to be inspected, and extract a first feature of the target infrared image and a second feature of the target visible light image;

[0006] Performing image fusion on the target infrared image and the target visible light image according to the first feature and the second feature to obtain a fused photovoltaic module image;

[0007] Extracting a photovoltaic panel area from the fused photovoltaic component image using a pre-trained photovoltaic fault classification model, and determining a three-dimensional attention weight for features of the photovoltaic panel area using an attention mechanism of the pre-trained photovoltaic fault classification model;

[0008] A fault classification result is determined for the photovoltaic panel area according to the three-dimensional attention weight to implement inspection of the photovoltaic assembly to be inspected.

[0009] According to another aspect of the present invention, a photovoltaic inspection device is provided, the device comprising:

[0010] An image feature extraction module, used to obtain a target infrared image and a target visible light image of the photovoltaic assembly to be inspected, and extract a first feature of the target infrared image and a second feature of the target visible light image;

[0011] An image fusion module, used for fusing the target infrared image and the target visible light image according to the first feature and the second feature to obtain a fused photovoltaic module image;

[0012] A three-dimensional attention weight determination module, used to extract a photovoltaic panel area from the fused photovoltaic component image using a pre-trained photovoltaic fault classification model, and determine a three-dimensional attention weight for features of the photovoltaic panel area using an attention mechanism of the pre-trained photovoltaic fault classification model;

[0013] The photovoltaic inspection module is used to determine a fault classification result for the photovoltaic panel area according to the three-dimensional attention weight so as to implement the inspection of the photovoltaic assembly to be inspected.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the photovoltaic inspection method described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the photovoltaic inspection method described in any embodiment of the present invention when executed.

[0019] The technical solution of the embodiment of the present invention obtains a target infrared image and a target visible light image of a photovoltaic component to be inspected, and extracts a first feature of the target infrared image and a second feature of the target visible light image; fuses the target infrared image and the target visible light image according to the first feature and the second feature to obtain a fused photovoltaic component image; uses a pre-trained photovoltaic fault classification model to extract a photovoltaic panel area from the fused photovoltaic component image, and uses the attention mechanism of the pre-trained photovoltaic fault classification model to determine a three-dimensional attention weight for the features of the photovoltaic panel area; determines a fault classification result for the photovoltaic panel area according to the three-dimensional attention weight to realize the inspection of the photovoltaic component to be inspected, and uses a photovoltaic component image based on a fusion of an infrared image and a visible light image to extract a photovoltaic panel area, and determines a three-dimensional attention weight for each feature of the extracted photovoltaic panel area. This solves the problem that the prior art relies on photovoltaic infrared images to only identify the existence of faults in photovoltaic components and cannot meet further requirements, and can more accurately and meticulously identify the fault category of the photovoltaic component.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A flow chart of a photovoltaic inspection method provided in Embodiment 1 of the present invention;

[0023] Figure 2 A schematic diagram of the structure of a photovoltaic inspection device provided in Embodiment 2 of the present invention;

[0024] Figure 3 It is a structural schematic diagram of an electronic device for implementing the photovoltaic inspection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Embodiment 1

[0028] Figure 1This is a flow chart of a photovoltaic inspection method provided in the first embodiment of the present invention. This embodiment is applicable to the case of fault inspection of photovoltaic modules. The method can be executed by a photovoltaic inspection device. The photovoltaic inspection device can be implemented in the form of hardware and / or software. The photovoltaic inspection device can be configured in a server connected to an inspection drone. Figure 1 As shown, the method includes:

[0029] S110, obtaining a target infrared image and a target visible light image of the photovoltaic assembly to be inspected, and extracting a first feature of the target infrared image and a second feature of the target visible light image.

[0030] The target infrared image may refer to an infrared image of the photovoltaic component to be inspected, which is acquired by the infrared thermal sensor of the inspection drone and has undergone image preprocessing. The target visible light image may refer to a visible light image of the photovoltaic component to be inspected, which is acquired by the visible light image acquisition device (e.g., CMOS sensor) of the inspection drone and has undergone image preprocessing. The first feature may refer to the infrared feature of the target infrared image. The second feature may refer to the color feature, texture feature, etc. of the target visible light image.

[0031] In this embodiment, the original infrared image and the original visible light image of the photovoltaic component to be inspected can be collected by an inspection drone, and the target infrared image and the target visible light image can be obtained after image preprocessing operations on the original infrared image and the original visible light image, and the first feature of the target infrared image and the second feature of the target visible light image can be extracted.

[0032] In an optional embodiment, obtaining a target infrared image and a target visible light image of a photovoltaic component to be inspected may include: collecting an original infrared image and an original visible light image of the photovoltaic component to be inspected; performing image denoising on the original infrared image and the original visible light image to obtain a denoised infrared image and a denoised visible light image; performing image enhancement on the denoised infrared image and the denoised visible light image to obtain an enhanced infrared image and an enhanced visible light image; performing image registration on the enhanced infrared image and the enhanced visible light image to obtain a target infrared image and a target visible light image.

[0033] In this embodiment, during the infrared image and visible light image acquisition process of the photovoltaic module to be inspected, due to the defects of the infrared thermal sensor, CMOS sensor, etc. and the interference of the external environment, the original infrared image and the original visible light image obtained may be contaminated by noise, which may easily produce artifacts and ghosting, resulting in unsatisfactory visual effects and poor image quality. If the original images are directly fused, the fusion effect of the infrared image and the visible light image will be greatly reduced and cannot meet the requirements of actual applications. Therefore, it is necessary to pre-process the acquired images to improve the image quality. For example, mean filtering can be used to denoise the original infrared image and the original visible light image.

[0034] Furthermore, the infrared sensor generates infrared images based on heat radiation, which are not easily affected by factors such as weather. However, due to its own conditions, the infrared images obtained are relatively blurry and the edges are not clear. The resolution of visible light images is high, but the visibility is low and the clarity is insufficient. Therefore, the infrared images and the original visible light images can be further enhanced. For example, the Retinex theory can be introduced to enhance the denoised infrared images and visible light images to improve the image saturation and clarity.

[0035] Furthermore, in order to comprehensively utilize the complementary information of the two images and improve the accuracy of subsequent image analysis, recognition and other tasks, the infrared image and visible light image of the photovoltaic module to be inspected can be registered. Exemplarily, the infrared image and the visible light image can be registered using a grayscale-based method. Specifically, the pixels can be aligned according to the grayscale information of the image or image block to minimize the difference in image information. For example, the cross-correlation method, the mutual information method and the error square sum method can be used for registration.

[0036] S120 , performing image fusion on the target infrared image and the target visible light image according to the first feature and the second feature to obtain a fused photovoltaic module image.

[0037] In this embodiment, the target infrared image and the target visible light image may be fused based on a convolutional neural network.

[0038] Exemplarily, the convolutional layer of the convolutional neural network can be used to first extract features from the input infrared image and visible light image respectively. The convolution kernel slides on the image and performs convolution operations with the local area of ​​the image, thereby extracting the local features of the image. For example, for the thermal target area in the infrared image, the convolution kernel can learn the feature pattern related to thermal radiation; for the scene detail part in the visible light image, the convolution kernel can capture features such as texture and color. Since the infrared image and the visible light image contain different types of information (thermal target information of the infrared image and scene detail information of the visible light image), the convolution layer can extract the first feature and the second feature that are valuable for fusion from the two images respectively. For another example, multi-scale feature extraction can be achieved by stacking multiple convolution layers. Shallow convolution layers can usually capture the basic features of the image, such as edges, simple textures, etc.; while deep convolution layers can extract more complex and abstract features, such as the overall shape of the target, semantic information, etc. In the fusion of infrared images and visible light images, this multi-scale feature extraction helps to fully obtain the information of the two images. For example, for small targets in infrared images, the shallow convolution layer may first detect the edge features of the target. As the convolution layer deepens, the complete shape of the target and its position in the image can be gradually determined. For complex scenes in visible light images, multi-scale feature extraction can help identify the relationship between different objects and the overall scene structure.

[0039] In an optional embodiment, performing image fusion of the target infrared image and the target visible light image according to the first feature and the second feature to obtain a fused photovoltaic assembly image may include: calculating a first similarity between the first feature and the second feature, and performing image fusion of the target infrared image and the target visible light image according to the first similarity to obtain a fused photovoltaic assembly image.

[0040] The first similarity may be determined based on various distance metrics or similarity metrics, such as Euclidean distance, cosine similarity, and the like.

[0041] In this embodiment, the similarity between the first feature and the second feature can be calculated to determine which features can be fused together, so as to fuse the target infrared image and the target visible light image to obtain a fused photovoltaic assembly image.

[0042] S130. Extract the photovoltaic panel area from the fused photovoltaic component image using the pre-trained photovoltaic fault classification model, and determine the three-dimensional attention weight for the features of the photovoltaic panel area using the attention mechanism of the pre-trained photovoltaic fault classification model.

[0043] In this embodiment, taking into account the complex background of the photovoltaic panels, the photovoltaic panel area can be extracted from the fused photovoltaic component image through a pre-trained photovoltaic fault classification model, and an attention mechanism is introduced to enhance its attention to the areas with hot spot characteristics, reduce the interference of the complex infrared background in the data set, and determine the three-dimensional attention weight for the characteristics of the photovoltaic panel area.

[0044] In an optional implementation, extracting the photovoltaic panel area from the fused photovoltaic component image using a pre-trained photovoltaic fault classification model may include: extracting the photovoltaic panel area from the fused photovoltaic component image using a graphic extraction algorithm using the pre-trained photovoltaic fault classification model.

[0045] Specifically, a graphic extraction algorithm may be used to extract the photovoltaic panel area through visible light recognition, morphological changes, and other means.

[0046] In an optional embodiment, the three-dimensional attention weight is determined for the features of the photovoltaic panel area using the attention mechanism of a pre-trained photovoltaic fault classification model, which may include: calculating the second similarity between each pixel in the photovoltaic panel area and its adjacent pixels using the attention mechanism of the pre-trained photovoltaic fault classification model; and calculating the three-dimensional attention weight for each pixel in the photovoltaic panel area based on the second similarity.

[0047] In this embodiment, the attention mechanism used may be the SimAM attention mechanism, and the energy function of the SimAM attention mechanism calculates the attention weight by calculating the local self-similarity of the feature map: in an image, adjacent pixels usually have strong similarities, while the similarity between distant pixels is weaker. SimAM uses this feature to generate attention weights by calculating the similarity between each pixel in the feature map and its adjacent pixels.

[0048] The calculation formula of SimAM can be expressed as: Among them, w i represents the attention weight of the ith pixel; k represents the normalization constant; Ni represents the set of neighboring pixels of the ith pixel; s(f i ,f j ) represents the similarity between the i-th pixel and the j-th pixel, calculated by the Euclidean distance:

[0049] Optionally, the pre-trained photovoltaic fault classification model can be pre-trained in the following way: obtaining a photovoltaic component sample image set; performing image augmentation processing on each image in the photovoltaic component sample image set to obtain a photovoltaic component target sample image set; using the photovoltaic component target sample image set to train an initial photovoltaic fault classification model that introduces an attention mechanism to obtain a trained initial photovoltaic fault classification model; performing model testing on the trained initial photovoltaic fault classification model to adjust the parameters of the trained initial photovoltaic fault classification model and obtain a pre-trained photovoltaic fault classification model.

[0050] Among them, the initial photovoltaic fault classification model introducing the attention mechanism is trained using the photovoltaic component target sample image set to obtain the trained initial photovoltaic fault classification model, which can include: extracting the photovoltaic panel area for each photovoltaic component target sample image in the photovoltaic component target sample image set through the initial photovoltaic fault classification model; calculating the three-dimensional attention weight for the features of the photovoltaic panel area extraction results through the attention mechanism of the initial photovoltaic fault classification model; and outputting the corresponding photovoltaic fault classification result for each photovoltaic component target sample image according to the three-dimensional attention weight.

[0051] In this embodiment, a photovoltaic fault classification model can be obtained based on the YOLOv5 algorithm. The pre-trained photovoltaic fault classification model can be obtained through the following steps.

[0052] Step 1: PuzzleMix image data augmentation

[0053] When processing the PV module sample image set, PuzzleMix calculates the salient areas of each image, and then crops the salient areas to perform a series of operations such as image stitching, shifting, rotating, adjusting brightness, and adding noise to obtain the PV module target sample image set.

[0054] Step 2: Introduce the attention mechanism

[0055] Considering the complex background of photovoltaic panels, the photovoltaic panel area is extracted through visible light recognition, morphological changes, etc. In the process of model training, the attention mechanism is introduced to enhance its attention to the parts with hot spot characteristics and reduce the interference of complex infrared background in the data set. Here, the SimAM attention mechanism algorithm is used to extract photovoltaic panel problems.

[0056] Step 2.1: Photovoltaic panel extraction

[0057] The graphic extraction algorithm is used to identify photovoltaic panels and extract the photovoltaic panel area.

[0058] Step 2.2: Attention Mechanism

[0059] The photovoltaic panel area is introduced, and the SimAM attention mechanism algorithm is used to optimize the energy function to explore the importance of each neuron, thereby inferring the three-dimensional attention weight for the feature map.

[0060] Step 3: Model training

[0061] The target sample image set of photovoltaic modules is input into the model for model training to obtain the YOLOV5 model embedded with the 3D parameter-free SimAM attention mechanism.

[0062] Step 4: Model testing

[0063] Test the model and adjust parameters based on the results.

[0064] S140. Determine a fault classification result for the photovoltaic panel area according to the three-dimensional attention weight to implement inspection of the photovoltaic components to be inspected.

[0065] In this embodiment, the fault classification result of the photovoltaic component to be inspected can be determined based on the three-dimensional attention weight calculation result of the photovoltaic panel area.

[0066] The technical solution of the embodiment of the present invention obtains a target infrared image and a target visible light image of a photovoltaic component to be inspected, and extracts a first feature of the target infrared image and a second feature of the target visible light image; fuses the target infrared image and the target visible light image according to the first feature and the second feature to obtain a fused photovoltaic component image; uses a pre-trained photovoltaic fault classification model to extract a photovoltaic panel area from the fused photovoltaic component image, and uses the attention mechanism of the pre-trained photovoltaic fault classification model to determine a three-dimensional attention weight for the features of the photovoltaic panel area; determines a fault classification result for the photovoltaic panel area according to the three-dimensional attention weight to realize the inspection of the photovoltaic component to be inspected, and uses a photovoltaic component image based on a fusion of an infrared image and a visible light image to extract a photovoltaic panel area, and determines a three-dimensional attention weight for each feature of the extracted photovoltaic panel area. This solves the problem that the prior art relies on photovoltaic infrared images to only identify the existence of faults in photovoltaic components and cannot meet further requirements, and can more accurately and meticulously identify the fault category of the photovoltaic component.

[0067] Embodiment 2

[0068] Figure 2 This is a schematic diagram of the structure of a photovoltaic inspection device provided in Embodiment 2 of the present invention. Figure 2 As shown, the device includes: an image feature extraction module 210, an image fusion module 220, a three-dimensional attention weight determination module 230 and a photovoltaic inspection module 240. Among them:

[0069] An image feature extraction module 210 is used to obtain a target infrared image and a target visible light image of the photovoltaic assembly to be inspected, and extract a first feature of the target infrared image and a second feature of the target visible light image;

[0070] An image fusion module 220, configured to fuse the target infrared image and the target visible light image according to the first feature and the second feature to obtain a fused photovoltaic module image;

[0071] A three-dimensional attention weight determination module 230 is used to extract a photovoltaic panel area from the fused photovoltaic component image using a pre-trained photovoltaic fault classification model, and determine a three-dimensional attention weight for features of the photovoltaic panel area using an attention mechanism of the pre-trained photovoltaic fault classification model;

[0072] The photovoltaic inspection module 240 is used to determine a fault classification result for the photovoltaic panel area according to the three-dimensional attention weight, so as to implement the inspection of the photovoltaic assembly to be inspected.

[0073] The technical solution of the embodiment of the present invention obtains a target infrared image and a target visible light image of a photovoltaic component to be inspected, and extracts a first feature of the target infrared image and a second feature of the target visible light image; fuses the target infrared image and the target visible light image according to the first feature and the second feature to obtain a fused photovoltaic component image; uses a pre-trained photovoltaic fault classification model to extract a photovoltaic panel area from the fused photovoltaic component image, and uses the attention mechanism of the pre-trained photovoltaic fault classification model to determine a three-dimensional attention weight for the features of the photovoltaic panel area; determines a fault classification result for the photovoltaic panel area according to the three-dimensional attention weight to realize the inspection of the photovoltaic component to be inspected, and uses a photovoltaic component image based on a fusion of an infrared image and a visible light image to extract a photovoltaic panel area, and determines a three-dimensional attention weight for each feature of the extracted photovoltaic panel area. This solves the problem that the prior art relies on photovoltaic infrared images to only identify the existence of faults in photovoltaic components and cannot meet further requirements, and can more accurately and meticulously identify the fault category of the photovoltaic component.

[0074] Optionally, the image feature extraction module 210 may be specifically used for:

[0075] Collecting the original infrared image and the original visible light image of the photovoltaic assembly to be inspected;

[0076] Performing image denoising on the original infrared image and the original visible light image to obtain a denoised infrared image and a denoised visible light image;

[0077] Performing image enhancement processing on the denoised infrared image and the denoised visible light image to obtain an enhanced infrared image and an enhanced visible light image;

[0078] The enhanced infrared image and the enhanced visible light image are registered to obtain the target infrared image and the target visible light image.

[0079] Optionally, the image fusion module 220 may be specifically used for:

[0080] A first similarity between the first feature and the second feature is calculated, and according to the first similarity, the target infrared image and the target visible light image are fused to obtain a fused photovoltaic assembly image.

[0081] Optionally, the three-dimensional attention weight determination module 230 may be specifically used for:

[0082] The pre-trained photovoltaic fault classification model is used to extract the photovoltaic panel area from the fused photovoltaic component image using a graphic extraction algorithm.

[0083] Optionally, the three-dimensional attention weight determination module 230 may be specifically used for:

[0084] Calculating a second similarity between each pixel in the photovoltaic panel area and its adjacent pixels using the attention mechanism of the pre-trained photovoltaic fault classification model;

[0085] The three-dimensional attention weight is calculated for each pixel of the photovoltaic panel area according to the second similarity.

[0086] Optionally, the photovoltaic inspection device further includes a model pre-training module, including:

[0087] A sample image set acquisition unit, used to acquire a sample image set of photovoltaic modules;

[0088] A target sample image set acquisition unit, configured to perform image augmentation processing on each image in the photovoltaic module sample image set to obtain a photovoltaic module target sample image set;

[0089] A first model acquisition module is used to train an initial photovoltaic fault classification model that introduces an attention mechanism using the photovoltaic component target sample image set to obtain a trained initial photovoltaic fault classification model;

[0090] The second model acquisition module is used to perform model testing on the trained initial photovoltaic fault classification model to adjust the parameters of the trained initial photovoltaic fault classification model and obtain the pre-trained photovoltaic fault classification model.

[0091] The photovoltaic inspection device provided in the embodiment of the present invention can execute the photovoltaic inspection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0092] Embodiment 3

[0093] Figure 3 A schematic diagram of an electronic device 300 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers or various forms of mobile devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0094] like Figure 3 As shown, the electronic device 300 includes at least one processor 301, and a memory connected to the at least one processor 301, such as a read-only memory (ROM) 302, a random access memory (RAM) 303, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 301 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 302 or the computer program loaded from the storage unit 308 to the random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 can also be stored. The processor 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0095] A number of components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0096] The processor 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 301 executes the various methods and processes described above, such as a photovoltaic inspection method.

[0097] In some embodiments, the photovoltaic inspection method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the processor 301, one or more steps of the photovoltaic inspection method described above can be performed. Alternatively, in other embodiments, the processor 301 can be configured to perform the photovoltaic inspection method in any other appropriate manner (for example, by means of firmware).

[0098] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0099] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0100] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0101] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0102] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0103] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0104] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0105] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A photovoltaic inspection method, characterized in that: include: Acquire a target infrared image and a target visible light image of the photovoltaic assembly to be inspected, and extract a first feature of the target infrared image and a second feature of the target visible light image; Performing image fusion on the target infrared image and the target visible light image according to the first feature and the second feature to obtain a fused photovoltaic module image; Extracting a photovoltaic panel area from the fused photovoltaic component image using a pre-trained photovoltaic fault classification model, and determining a three-dimensional attention weight for features of the photovoltaic panel area using an attention mechanism of the pre-trained photovoltaic fault classification model; A fault classification result is determined for the photovoltaic panel area according to the three-dimensional attention weight to implement inspection of the photovoltaic assembly to be inspected.

2. The method according to claim 1, characterized in that Obtain target infrared images and target visible light images of the photovoltaic modules to be inspected, including: Collecting the original infrared image and the original visible light image of the photovoltaic assembly to be inspected; Performing image denoising on the original infrared image and the original visible light image to obtain a denoised infrared image and a denoised visible light image; Performing image enhancement processing on the denoised infrared image and the denoised visible light image to obtain an enhanced infrared image and an enhanced visible light image; The enhanced infrared image and the enhanced visible light image are registered to obtain the target infrared image and the target visible light image.

3. The method according to claim 1, characterized in that The target infrared image and the target visible light image are fused according to the first feature and the second feature to obtain a fused photovoltaic module image, including: A first similarity between the first feature and the second feature is calculated, and according to the first similarity, the target infrared image and the target visible light image are fused to obtain a fused photovoltaic assembly image.

4. The method according to claim 1, characterized in that: The photovoltaic panel area is extracted from the fused photovoltaic component image using a pre-trained photovoltaic fault classification model, including: The pre-trained photovoltaic fault classification model is used to extract the photovoltaic panel area from the fused photovoltaic component image using a graphic extraction algorithm.

5. The method according to claim 1, characterized in that Determining a three-dimensional attention weight for the features of the photovoltaic panel area using the attention mechanism of the pre-trained photovoltaic fault classification model includes: Calculating a second similarity between each pixel in the photovoltaic panel area and its adjacent pixels using the attention mechanism of the pre-trained photovoltaic fault classification model; The three-dimensional attention weight is calculated for each pixel of the photovoltaic panel area according to the second similarity.

6. The method according to claim 1, characterized in that The pre-trained photovoltaic fault classification model is pre-trained in the following manner: Acquire a sample image set of photovoltaic modules; Performing image augmentation processing on each image in the photovoltaic module sample image set to obtain a photovoltaic module target sample image set; Using the photovoltaic component target sample image set to train the initial photovoltaic fault classification model that introduces the attention mechanism, to obtain a trained initial photovoltaic fault classification model; A model test is performed on the trained initial photovoltaic fault classification model to adjust the parameters of the trained initial photovoltaic fault classification model and obtain the pre-trained photovoltaic fault classification model.

7. The method according to claim 6, characterized in that The photovoltaic component target sample image set is used to train the initial photovoltaic fault classification model that introduces the attention mechanism, and the trained initial photovoltaic fault classification model is obtained, including: Extracting photovoltaic panel regions from each photovoltaic component target sample image in the photovoltaic component target sample image set by using an initial photovoltaic fault classification model; Calculating three-dimensional attention weights for features of photovoltaic panel region extraction results through the attention mechanism of the initial photovoltaic fault classification model; Output a corresponding photovoltaic fault classification result for each photovoltaic component target sample image according to the three-dimensional attention weight.

8. A photovoltaic inspection device, characterized in that: include: An image feature extraction module, used to obtain a target infrared image and a target visible light image of the photovoltaic assembly to be inspected, and extract a first feature of the target infrared image and a second feature of the target visible light image; An image fusion module, used for fusing the target infrared image and the target visible light image according to the first feature and the second feature to obtain a fused photovoltaic module image; A three-dimensional attention weight determination module, used to extract a photovoltaic panel area from the fused photovoltaic component image using a pre-trained photovoltaic fault classification model, and determine a three-dimensional attention weight for features of the photovoltaic panel area using an attention mechanism of the pre-trained photovoltaic fault classification model; The photovoltaic inspection module is used to determine a fault classification result for the photovoltaic panel area according to the three-dimensional attention weight so as to implement the inspection of the photovoltaic assembly to be inspected.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can perform the photovoltaic inspection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the photovoltaic inspection method according to any one of claims 1 to 7 when executed.

Images