Hot spot image automatic identification system for automatic inspection of unmanned aerial vehicle

Through the UAV collecting photovoltaic module images and integrating deep learning feature extraction and information compensation, the problems of low efficiency and insufficient accuracy of traditional inspections are solved, and efficient identification and accurate detection of small heat spots on the surface of photovoltaic modules are achieved.

CN118941996APending Publication Date: 2024-11-12HUANENG XINJIANG ENERGY DEVELOPMENT CO LTD SOUTHERN XINJIANG CLEAN ENERGY BRANCH +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411270182.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The inspection of traditional photovoltaic modules relies on manual labor, is inefficient, and is easily affected by experience and skills. Infrared light image detection lacks accuracy when occluding small foreign objects, making it difficult to meet the inspection needs of large-scale photovoltaic power stations.

Method used

UAVs are used to collect visible light images of photovoltaic modules, extract deep and shallow surface features through deep learning technology, and perform information compensation fusion to accurately identify small heat spots on the surface of photovoltaic modules.

Benefits of technology

It improves the identification accuracy of small heat spots on the surface of photovoltaic modules, reduces the cost and risks of manual inspection, and achieves efficient inspection of photovoltaic modules.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941996A_ABST
    Figure CN118941996A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a hot spot image automatic identification system for unmanned aerial vehicle automatic inspection, and the system collects a visible light image of a photovoltaic module through an unmanned aerial vehicle, and employs a machine vision technology based on deep learning to extract the deep features and shallow features of the surface state of the photovoltaic module. And information compensation type fusion is carried out on the deep-layer features and the shallow-layer features based on the feature difference between the deep-layer features and the shallow-layer features of the surface state of the photovoltaic module, so that comprehensive characterization of the surface state of the photovoltaic module is realized, and small hot spots on the surface of the photovoltaic module are accurately identified. Therefore, the identification precision of the small hot spots on the surface of the photovoltaic module can be improved, the cost and risk of manual inspection are reduced, and efficient inspection of the photovoltaic module is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of intelligent detection technology, and specifically to an automatic recognition system for hot spot images used for automatic inspection by unmanned aerial vehicles. Background Art

[0002] With the increasing attention paid to renewable energy and the rapid development of photovoltaic technology, photovoltaic power generation has become an important part of the renewable energy field and has been widely used around the world. However, with the expansion of the scale of photovoltaic power stations and the increase in the number of photovoltaic modules, the failure of photovoltaic equipment has gradually increased, especially the hot spot defect of photovoltaic modules has become one of the main causes of failure in the operation of photovoltaic power generation systems.

[0003] Hot spots refer to the phenomenon that heat accumulates when some cells are blocked due to local shadows, stains or internal faults inside the photovoltaic module under sunlight. Such hot spots not only reduce the power generation efficiency of photovoltaic modules, but may also cause serious safety problems such as module damage and even fire. Therefore, regular inspections of photovoltaic modules and timely detection of hot spot defects are crucial to ensure the safe and stable operation of photovoltaic power stations.

[0004] The traditional inspection method of photovoltaic modules mainly relies on manual inspection, that is, inspectors inspect photovoltaic modules one by one through naked eye observation or using handheld infrared thermal imagers and other equipment. However, this inspection method has many shortcomings. First, manual inspection is inefficient and difficult to meet the inspection needs of large-scale photovoltaic power stations; second, manual inspection is easily affected by factors such as the experience and skill level of inspectors, and there is a risk of misjudgment and missed inspection; finally, the traditional infrared light image detection method has limitations when identifying small foreign objects (such as leaves, bird droppings, etc.) that block the inspection, affecting the detection accuracy.

[0005] In recent years, with the rapid development of drone technology and the continuous advancement of machine vision technology, drone automatic inspection technology has gradually become a new trend in photovoltaic power station inspection. Therefore, an automatic recognition system for hot spot images used in drone automatic inspection is expected. Summary of the invention

[0006] The present invention aims to solve at least one of the technical problems existing in the prior art, and provides an automatic recognition system for hot spot images used for automatic inspection by unmanned aerial vehicles.

[0007] The embodiment of the present invention provides a hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles, comprising:

[0008] A photovoltaic module image acquisition module is used to acquire a visible light image of a photovoltaic module collected by a camera deployed on a drone;

[0009] A depth feature extraction module is used to extract depth features from the visible light image of the photovoltaic module to obtain a shallow feature map of the photovoltaic module surface and a deep feature map of the photovoltaic module surface;

[0010] A semantic enhancement module, used for performing semantic enhancement on the deep feature map of the photovoltaic module surface to obtain a spatial semantic enhancement feature map of the photovoltaic module surface;

[0011] An information compensation fusion module is used to perform information compensation fusion on the photovoltaic module surface spatial semantic enhancement feature map and the photovoltaic module surface shallow feature map to obtain a compensated photovoltaic module surface multi-scale fusion representation feature map;

[0012] The hot spot detection module is used to determine the detection result based on the multi-scale fusion representation feature map of the surface of the compensated photovoltaic module.

[0013] In some possible embodiments, the depth feature extraction module is used to:

[0014] The photovoltaic module visible light image is passed through a photovoltaic module surface feature multi-scale sensor based on a hole pyramid network to obtain a shallow feature map of the photovoltaic module surface and a deep feature map of the photovoltaic module surface.

[0015] In some possible embodiments, the semantic enhancement module is used to:

[0016] The deep feature map of the photovoltaic module surface is input into the spatial mask-based attention module to obtain the spatial semantic enhanced feature map of the photovoltaic module surface.

[0017] In some possible embodiments, the semantic enhancement module includes:

[0018] A spatial significance scoring unit, used for calculating a photovoltaic module surface feature significance score map of the photovoltaic module surface deep feature map;

[0019] A masking unit, configured to perform masking processing on the photovoltaic module surface feature significance score map based on a predetermined threshold value to obtain a spatial significance attention mask map;

[0020] A pixel-by-pixel weighting unit is used to calculate the photovoltaic component surface deep feature map and the spatial significance attention mask map by multiplying the position points by the spatial significance attention mask map to obtain the photovoltaic component surface spatial semantic enhancement feature map.

[0021] In some possible embodiments, the spatial significance scoring unit is used to:

[0022] Calculating the reciprocal of the exponential function value with the natural constant as the base of each characteristic value of the deep characteristic map of the surface of the photovoltaic module to obtain an indexed deep characteristic map of the surface of the photovoltaic module;

[0023] Calculating the cumulative sum of each characteristic value of the indexed photovoltaic module surface deep feature map to obtain the global characteristic intensity of the photovoltaic module surface deep feature map;

[0024] The feature values ​​of each position of the indexed photovoltaic module surface deep feature map are divided by the addition result of the global feature intensity semantic addition to obtain the photovoltaic module surface feature significance score map.

[0025] In some possible embodiments, the information compensation fusion module is used to:

[0026] The photovoltaic module surface spatial semantic enhancement feature map and the photovoltaic module surface shallow feature map are passed through an information transmission loss compensation network to obtain the compensated photovoltaic module surface multi-scale fusion representation feature map.

[0027] In some possible embodiments, the information compensation fusion module includes:

[0028] A feature difference unit, used for calculating the feature difference between the shallow feature map of the photovoltaic module surface and the spatial semantic enhancement feature map of the photovoltaic module surface to obtain a shallow detail-deep semantic difference feature map of the photovoltaic module surface;

[0029] A loss information capture unit, used for passing the photovoltaic module surface shallow detail-deep semantic difference feature map through a loss information capture module to obtain a photovoltaic module surface shallow detail-deep semantic information loss feature map, wherein the loss information capture module includes a point convolution layer, a nonlinear activation layer based on a tanh function, and an upsampling layer;

[0030] The information compensation fusion unit is used to perform information compensation fusion on the shallow feature map of the photovoltaic module surface and the spatial semantic enhancement feature map of the photovoltaic module surface based on the shallow detail-deep semantic information loss feature map of the photovoltaic module surface to obtain the compensated photovoltaic module surface multi-scale fusion representation feature map.

[0031] In some possible embodiments, the feature difference unit includes:

[0032] A downsampling subunit, used for downsampling the shallow feature map of the photovoltaic module surface to obtain a downsampled shallow feature map of the photovoltaic module surface;

[0033] The position difference subunit is used to calculate the position subtraction between the shallow feature map of the downsampled photovoltaic component surface and the spatial semantic enhancement feature map of the photovoltaic component surface to obtain the shallow detail-deep semantic difference feature map of the photovoltaic component surface.

[0034] In some possible embodiments, the information compensation fusion unit includes:

[0035] A shallow feature information compensation subunit is used to calculate the photovoltaic module surface shallow detail-deep semantic information loss feature map and the photovoltaic module surface shallow feature map by position point multiplication to obtain an information compensation photovoltaic module surface shallow feature map;

[0036] The fusion subunit is used to calculate the position-weighted sum of the shallow feature map of the information-compensated photovoltaic component surface and the spatial semantic enhancement feature map of the photovoltaic component surface to obtain the multi-scale fusion representation feature map of the compensated photovoltaic component surface.

[0037] In some possible embodiments, the hot spot detection module is used to:

[0038] The multi-scale fusion representation feature map of the surface of the compensated photovoltaic component is input into a detector based on a classifier to obtain the detection result, and the detection result is used to indicate whether there is a small hot spot on the photovoltaic component.

[0039] The hot spot image automatic recognition system for automatic inspection by drones of the embodiment of the present invention uses drones to collect visible light images of photovoltaic modules, and uses machine vision technology based on deep learning to extract deep and shallow features of the surface state of photovoltaic modules, and then based on the feature differences between the deep and shallow features of the surface state of photovoltaic modules, information compensation fusion is performed on the two, so as to achieve a comprehensive characterization of the surface state of photovoltaic modules, and thus accurately identify small hot spots on the surface of photovoltaic modules. In this way, the recognition accuracy of small hot spots on the surface of photovoltaic modules can be improved, the cost and risk of manual inspection can be reduced, and efficient inspection of photovoltaic modules can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 is a block diagram of a hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles according to an embodiment of the present invention;

[0042] Figure 2 Schematic diagram of the architecture of a hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles according to an embodiment of the present invention.

[0043] Figure 3 It is a block diagram of a semantic enhancement module in a hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles according to an embodiment of the present invention;

[0044] Figure 4 is a block diagram of an information compensation fusion module in a hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles according to an embodiment of the present invention;

[0045] Figure 5 A block diagram of a feature difference unit in a hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles according to an embodiment of the present invention;

[0046] Figure 6 The block diagram is a block diagram of an information compensation fusion unit in a hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0048] Unless otherwise specified, the technical terms or scientific terms used in the embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. "Including" or "comprising" used in the embodiments of the present invention neither limit the shapes, numbers, steps, actions, operations, components, originals and / or their groups mentioned, nor exclude the appearance or addition of one or more other different shapes, numbers, steps, actions, operations, components, originals and / or their groups, or the addition of these. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number and order of the indicated technical features. Thus, the features defined as "first" and "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0049] Unless otherwise specifically stated, the relative arrangement of the components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention. At the same time, it should be understood that for ease of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship, and the techniques, methods and devices known to ordinary technicians in the relevant fields may not be discussed in detail, but where appropriate, the techniques, methods and devices shown should be considered as part of the authorized specification. In all examples shown and discussed here, any specific other examples may have different values. It should be noted that similar symbols and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0050] In the description of the embodiments of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In the embodiments of the present invention, the schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in the embodiments of the present invention and the features of the different embodiments or examples, without contradiction.

[0051] As mentioned in the above background technology, traditional inspection of photovoltaic modules relies on manual work, mainly through naked eyes or using handheld infrared thermal imagers and other equipment for inspection. However, this method has low detection efficiency, is difficult to meet large-scale needs, and is easily affected by personnel experience and skills, and has the risk of misjudgment and missed detection. In addition, the traditional infrared light image detection method has limitations when identifying small foreign objects (such as leaves, bird droppings, etc.) that block, affecting the detection accuracy. In response to the above technical problems, the technical concept of the present invention is to collect visible light images of photovoltaic modules through drones, and use machine vision technology based on deep learning to extract deep features and shallow features of the surface state of photovoltaic modules, and then based on the feature differences between the deep features and shallow features of the surface state of photovoltaic modules, the two are information-compensated and fused, so as to achieve a comprehensive characterization of the surface state of photovoltaic modules, thereby accurately identifying small hot spots on the surface of photovoltaic modules. In this way, the recognition accuracy of small hot spots on the surface of photovoltaic modules can be improved, the cost and risk of manual inspections can be reduced, and efficient inspections of photovoltaic modules can be achieved.

[0052] Figure 1 4 is a block diagram of a hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles according to an embodiment of the present invention. Figure 2This is a schematic diagram of the architecture of a hot spot image automatic recognition system for automatic inspection of drones according to an embodiment of the present invention. Figure 1 and Figure 2 As shown, according to an embodiment of the present invention, a hot spot image automatic recognition system 100 for automatic inspection by unmanned aerial vehicles includes: a photovoltaic component image acquisition module 110, which is used to acquire a visible light image of a photovoltaic component collected by a camera deployed on a drone; a depth feature extraction module 120, which is used to perform depth feature extraction on the visible light image of the photovoltaic component to obtain a shallow feature map of the photovoltaic component surface and a deep feature map of the photovoltaic component surface; a semantic enhancement module 130, which is used to perform semantic enhancement on the deep feature map of the photovoltaic component surface to obtain a spatial semantic enhancement feature map of the photovoltaic component surface; an information compensation fusion module 140, which is used to perform information compensation fusion on the spatial semantic enhancement feature map of the photovoltaic component surface and the shallow feature map of the photovoltaic component surface to obtain a multi-scale fusion representation feature map of the compensated photovoltaic component surface; and a hot spot detection module 150, which is used to determine a detection result based on the multi-scale fusion representation feature map of the compensated photovoltaic component surface.

[0053] In the above-mentioned hot spot image automatic recognition system 100 for automatic inspection by drones, the photovoltaic component image acquisition module 110 is used to acquire the visible light image of the photovoltaic component collected by the camera deployed on the drone. It should be understood that photovoltaic power stations are widely distributed, and manual inspections are not only time-consuming and labor-intensive, but also inefficient. Drones can quickly fly over various areas of photovoltaic power stations, and the cameras they carry can capture images of photovoltaic components with high resolution and frequency. Automatic inspections are carried out through this efficient data acquisition method, which realizes efficient and comprehensive hot spot fault inspections of photovoltaic components. At the same time, the visible light images of photovoltaic components collected by drones contain various information such as color, texture, shape, etc. on the surface of photovoltaic components, providing a data basis for the subsequent identification of small hot spot defects in photovoltaic components.

[0054] In the above-mentioned hot spot image automatic identification system 100 for automatic inspection by unmanned aerial vehicles, the depth feature extraction module 120 is used to extract depth features from the visible light image of the photovoltaic module to obtain a shallow feature map of the photovoltaic module surface and a deep feature map of the photovoltaic module surface. In a specific example of the present invention, the processing method for extracting depth features from the visible light image of the photovoltaic module is to pass the visible light image of the photovoltaic module through a photovoltaic module surface feature multi-scale sensor based on a hollow pyramid network to obtain a shallow feature map of the photovoltaic module surface and a deep feature map of the photovoltaic module surface. It should be understood that, considering that the hot spots on the surface of the photovoltaic module may have different scales and manifestations, in order to comprehensively and accurately identify the hot spot phenomenon on the surface of the photovoltaic module, in the technical solution of the present invention, a photovoltaic module surface feature multi-scale sensor based on a hollow pyramid network is used to process the visible light image of the photovoltaic module. Specifically, by introducing hole convolution kernels of different sizes, the dilated pyramid network can perform downsampling and upsampling operations on the visible light image of the photovoltaic module at different levels to capture the surface features of the photovoltaic module at different scales, thereby obtaining a shallow feature map of the photovoltaic module surface and a deep feature map of the photovoltaic module surface. Among them, the shallow feature map of the photovoltaic module surface mainly reflects the detailed information such as the color and texture of the photovoltaic module surface, while the shallow feature map of the photovoltaic module surface reveals more global semantic information such as the structure and shape of the module, realizing multi-scale perception of the surface features of the photovoltaic module, which helps to more accurately identify hot spot defects on the surface of the photovoltaic module.

[0055] In the above-mentioned hot spot image automatic recognition system 100 for automatic inspection by unmanned aerial vehicles, the semantic enhancement module 130 is used to semantically enhance the deep feature map of the surface of the photovoltaic module to obtain the spatial semantic enhancement feature map of the surface of the photovoltaic module. It should be understood that, considering that the hot spot defects on the surface of the photovoltaic module often manifest as abnormal changes in the local area, when identifying hot spot defects, in order to focus on the key areas in the deep feature map of the surface of the photovoltaic module and ignore the interference of irrelevant or redundant information, it is expected to further semantically enhance the deep feature map of the surface of the photovoltaic module. In a specific example of the present invention, the processing method for semantically enhancing the deep feature map of the surface of the photovoltaic module is to input the deep feature map of the surface of the photovoltaic module into an attention module based on a spatial mask to obtain the spatial semantic enhancement feature map of the surface of the photovoltaic module. Specifically, the spatial mask-based attention module can adaptively learn the importance of each spatial position in the deep feature map of the photovoltaic module surface, generate a corresponding spatial saliency attention mask map, and then spatially weight the deep feature map of the photovoltaic module surface based on the generated spatial saliency attention mask map to highlight the key hot spot area feature expression while suppressing the interference of irrelevant or redundant information, thereby improving the recognition accuracy and stability of hot spot defects.

[0056] Figure 3 FIG. 4 is a block diagram of a semantic enhancement module in a hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles according to an embodiment of the present invention. Figure 3 As shown, the semantic enhancement module 130 includes: a spatial significance scoring unit 131, which is used to calculate the photovoltaic module surface feature significance score map of the photovoltaic module surface deep feature map; a masking unit 132, which is used to mask the photovoltaic module surface feature significance score map based on a predetermined threshold to obtain a spatial significance attention mask map; a pixel-by-pixel weighting unit 133, which is used to calculate the position point multiplication of the photovoltaic module surface deep feature map and the spatial significance attention mask map using the spatial significance attention mask map as a weight to obtain the photovoltaic module surface spatial semantic enhancement feature map.

[0057] More specifically, the spatial significance scoring unit 131 is used to: calculate the reciprocal of the exponential function value with a natural constant as the base of each eigenvalue of the deep surface feature map of the photovoltaic module to obtain an indexed deep surface feature map of the photovoltaic module; calculate the cumulative sum of each eigenvalue of the deep surface feature map of the photovoltaic module to obtain the global feature intensity of the deep surface feature map of the photovoltaic module; divide the eigenvalue of each position of the deep surface feature map of the photovoltaic module by the addition result of the global feature intensity and one to obtain the photovoltaic module surface feature significance score map.

[0058] That is, the semantic enhancement module 130 is used to process the deep feature map of the photovoltaic module surface with the following semantic enhancement formula to obtain the spatial semantic enhancement feature map of the photovoltaic module surface; wherein the semantic enhancement formula is:

[0059]

[0060] F i,j,k =M(i,j,k)⊙A(i,j,k)

[0061] Where A(i,j,k) is the feature value of the position (i,j,k) in the deep feature map of the photovoltaic module surface, mask(·) represents the masking process, ε represents the predetermined threshold, M(i,j,k) is the feature value of the position (i,j,k) in the spatial saliency attention mask map, ⊙ represents the point multiplication by position, and F i,j,k is the feature value of the position (i, j, k) in the spatial semantic enhancement feature map of the photovoltaic module surface.

[0062] In the above-mentioned hot spot image automatic identification system 100 for automatic inspection by unmanned aerial vehicles, the information compensation fusion module 140 is used to perform information compensation fusion on the spatial semantic enhancement feature map of the photovoltaic component surface and the shallow feature map of the photovoltaic component surface to obtain a compensated photovoltaic component surface multi-scale fusion representation feature map. In a specific example of the present invention, the processing method of performing information compensation fusion on the spatial semantic enhancement feature map of the photovoltaic component surface and the shallow feature map of the photovoltaic component surface is to pass the spatial semantic enhancement feature map of the photovoltaic component surface and the shallow feature map of the photovoltaic component surface through an information transmission loss compensation network to obtain the compensated photovoltaic component surface multi-scale fusion representation feature map. It should be understood that the spatial semantic enhancement feature map of the photovoltaic component surface and the shallow feature map of the photovoltaic component surface are two different levels of feature expressions of photovoltaic component surface information, wherein the former focuses more on global and abstract semantic information, while the latter focuses on local and specific detail information. By fusing the two, the advantages of different scale features can be fully utilized to improve the accuracy of hot spot detection. However, due to the different feature extraction methods of the spatial semantic enhancement feature map of the photovoltaic module surface and the shallow feature map of the photovoltaic module surface, there are differences in feature expression between the two. Directly performing simple feature splicing and other fusion operations may lead to poor fusion effect, thereby affecting the accuracy of hot spot defect recognition. Based on this, in the technical solution of the present invention, an information transmission loss compensation network is introduced to perform information compensation fusion processing on the spatial semantic enhancement feature map of the photovoltaic module surface and the shallow feature map of the photovoltaic module surface. Specifically, the information transmission loss compensation network mines the feature differences between the spatial semantic enhancement feature map of the photovoltaic module surface and the shallow feature map of the photovoltaic module surface, and transfers the global semantic information in the spatial semantic enhancement feature map of the photovoltaic module surface to the shallow feature map of the photovoltaic module surface to compensate and enhance the information. Through information transmission and loss compensation, the information supplementation and fusion between the two different levels of features are realized, which not only effectively retains the global semantic information and local detail information of the photovoltaic module surface, but also effectively reduces information loss and conflict, providing a more comprehensive and accurate feature expression for subsequent hot spot defect recognition.

[0063] Figure 4 FIG. 1 is a block diagram of an information compensation fusion module in a hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles according to an embodiment of the present invention. Figure 4As shown, the information compensation fusion module 140 includes: a feature difference unit 141, which is used to calculate the feature difference between the shallow feature map of the photovoltaic component surface and the spatial semantic enhancement feature map of the photovoltaic component surface to obtain a shallow detail-deep semantic difference feature map of the photovoltaic component surface; a loss information capture unit 142, which is used to pass the shallow detail-deep semantic difference feature map of the photovoltaic component surface through the loss information capture module to obtain a shallow detail-deep semantic information loss feature map of the photovoltaic component surface, wherein the loss information capture module includes a point convolution layer, a nonlinear activation layer based on a tanh function, and an upsampling layer; an information compensation fusion unit 143, which is used to perform information compensation fusion on the shallow feature map of the photovoltaic component surface and the spatial semantic enhancement feature map of the photovoltaic component surface based on the shallow detail-deep semantic information loss feature map of the photovoltaic component surface to obtain the compensated photovoltaic component surface multi-scale fusion representation feature map.

[0064] More specifically, Figure 5 FIG. 1 is a block diagram of a feature difference unit in a hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles according to an embodiment of the present invention. Figure 5 As shown, the feature difference unit 141 includes: a downsampling subunit 1411, which is used to downsample the shallow feature map of the photovoltaic component surface to obtain a downsampled shallow feature map of the photovoltaic component surface; and a position difference subunit 1412, which is used to calculate the positional subtraction between the downsampled shallow feature map of the photovoltaic component surface and the spatial semantic enhancement feature map of the photovoltaic component surface to obtain the shallow detail-deep semantic difference feature map of the photovoltaic component surface.

[0065] Figure 6 FIG. 1 is a block diagram of an information compensation fusion unit in a hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles according to an embodiment of the present invention. Figure 6 As shown, the information compensation fusion unit 143 includes: a shallow feature information compensation subunit 1431, which is used to calculate the shallow detail-deep semantic information loss feature map of the photovoltaic component surface and the shallow feature map of the photovoltaic component surface by position point multiplication to obtain the information compensated photovoltaic component surface shallow feature map; a fusion subunit 1432, which is used to calculate the position weighted sum of the information compensated photovoltaic component surface shallow feature map and the photovoltaic component surface spatial semantic enhancement feature map to obtain the compensated photovoltaic component surface multi-scale fusion representation feature map.

[0066] That is, the information compensation fusion module 140 is used to process the photovoltaic module surface spatial semantic enhancement feature map and the photovoltaic module surface shallow feature map using the following information compensation fusion formula to obtain a compensated photovoltaic module surface multi-scale fusion representation feature map, wherein the information compensation fusion formula is:

[0067]

[0068] RA(F1,F2)=αF1*Loss+βF2

[0069] Among them, F1 represents the shallow feature map of the photovoltaic module surface, F2 represents the spatial semantic enhancement feature map of the photovoltaic module surface, DownSampling(·) represents downsampling processing, UpSampling(·) represents upsampling processing, Conv 1×1 (·) represents 1×1 convolution processing, tanh is a nonlinear activation function, Loss represents the shallow details-deep semantic information loss feature map of the photovoltaic module surface, α and β are predetermined weight parameters, * represents point multiplication by position, + represents the addition of corresponding elements, - represents the subtraction of corresponding elements, and RA(F1, F2) represents the multi-scale fusion representation feature map of the compensated photovoltaic module surface.

[0070] In the above-mentioned hot spot image automatic recognition system 100 for automatic inspection by unmanned aerial vehicles, the hot spot detection module 150 is used to determine the detection result based on the multi-scale fusion representation feature map of the surface of the compensated photovoltaic component. In a specific example of the present invention, the method of determining the detection result based on the multi-scale fusion representation feature map of the surface of the compensated photovoltaic component is to input the multi-scale fusion representation feature map of the surface of the compensated photovoltaic component into a classifier-based detector to obtain the detection result, and the detection result is used to indicate whether there are small hot spots on the photovoltaic component. It should be understood that in the technical solution of the present invention, the classifier-based detector is a trained deep learning model, which learns the difference between hot spot characteristics and normal photovoltaic component surface characteristics through a training process, so as to accurately identify small hot spot defects on the surface of photovoltaic components. By inputting the multi-scale fusion representation feature map of the compensated photovoltaic module surface into the classifier-based detector, it learns and understands the multi-scale fusion representation feature map of the compensated photovoltaic module surface to map it to the corresponding preset label, and outputs a detection result indicating whether there are small hot spots on the photovoltaic module, thereby achieving efficient detection of small hot spots on the surface of photovoltaic modules and providing strong technical support for the operation and maintenance management of photovoltaic power stations.

[0071] In summary, the hot spot image automatic recognition system for automatic inspection by drone according to the embodiment of the present invention is explained, which uses drones to collect visible light images of photovoltaic modules, and uses machine vision technology based on deep learning to extract deep and shallow features of the surface state of photovoltaic modules, and then based on the feature difference between the deep and shallow features of the surface state of photovoltaic modules, the two are information-compensatedly fused, so as to achieve a comprehensive characterization of the surface state of photovoltaic modules, thereby accurately identifying small hot spots on the surface of photovoltaic modules. In this way, the recognition accuracy of small hot spots on the surface of photovoltaic modules can be improved, the cost and risk of manual inspection can be reduced, and efficient inspection of photovoltaic modules can be achieved.

[0072] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0073] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical sub-unit, that is, it may be located in one place, or it may be distributed on multiple network sub-units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0074] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0075] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A hot spot image automatic recognition system for automatic inspection by unmanned aerial vehicles, characterized in that: include: A photovoltaic module image acquisition module is used to acquire a visible light image of a photovoltaic module collected by a camera deployed on a drone; A depth feature extraction module is used to extract depth features from the visible light image of the photovoltaic module to obtain a shallow feature map of the photovoltaic module surface and a deep feature map of the photovoltaic module surface; A semantic enhancement module, used for performing semantic enhancement on the deep feature map of the photovoltaic module surface to obtain a spatial semantic enhancement feature map of the photovoltaic module surface; An information compensation fusion module is used to perform information compensation fusion on the photovoltaic module surface spatial semantic enhancement feature map and the photovoltaic module surface shallow feature map to obtain a compensated photovoltaic module surface multi-scale fusion representation feature map; The hot spot detection module is used to determine the detection result based on the multi-scale fusion representation feature map of the surface of the compensated photovoltaic module.

2. The hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles according to claim 1 is characterized in that: The depth feature extraction module is used to: The photovoltaic module visible light image is passed through a photovoltaic module surface feature multi-scale sensor based on a hole pyramid network to obtain a shallow feature map of the photovoltaic module surface and a deep feature map of the photovoltaic module surface.

3. The hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles according to claim 2 is characterized in that: The semantic enhancement module is used to: The deep feature map of the photovoltaic module surface is input into the spatial mask-based attention module to obtain the spatial semantic enhanced feature map of the photovoltaic module surface.

4. The hot spot image automatic recognition system for automatic inspection by unmanned aerial vehicles according to claim 3 is characterized in that: The semantic enhancement module comprises: A spatial significance scoring unit, used for calculating a photovoltaic module surface feature significance score map of the photovoltaic module surface deep feature map; A masking unit, configured to perform masking processing on the photovoltaic module surface feature significance score map based on a predetermined threshold value to obtain a spatial significance attention mask map; A pixel-by-pixel weighting unit is used to calculate the photovoltaic component surface deep feature map and the spatial significance attention mask map by multiplying the position points by the spatial significance attention mask map to obtain the photovoltaic component surface spatial semantic enhancement feature map.

5. The hot spot image automatic recognition system for automatic inspection by unmanned aerial vehicles according to claim 4 is characterized in that: The spatial significance scoring unit is used to: Calculating the reciprocal of the exponential function value with the natural constant as the base of each characteristic value of the deep characteristic map of the surface of the photovoltaic module to obtain an indexed deep characteristic map of the surface of the photovoltaic module; Calculating the cumulative sum of each characteristic value of the indexed photovoltaic module surface deep feature map to obtain the global characteristic intensity of the photovoltaic module surface deep feature map; The feature values ​​of each position of the indexed photovoltaic module surface deep feature map are divided by the addition result of the global feature intensity semantic addition to obtain the photovoltaic module surface feature significance score map.

6. The hot spot image automatic recognition system for automatic inspection by unmanned aerial vehicles according to claim 5 is characterized in that: The information compensation fusion module is used to: The photovoltaic module surface spatial semantic enhancement feature map and the photovoltaic module surface shallow feature map are passed through an information transmission loss compensation network to obtain the compensated photovoltaic module surface multi-scale fusion representation feature map.

7. The hot spot image automatic recognition system for automatic inspection by unmanned aerial vehicles according to claim 6 is characterized in that: The information compensation fusion module includes: A feature difference unit, used for calculating the feature difference between the shallow feature map of the photovoltaic module surface and the spatial semantic enhancement feature map of the photovoltaic module surface to obtain a shallow detail-deep semantic difference feature map of the photovoltaic module surface; A loss information capture unit, used for passing the photovoltaic module surface shallow detail-deep semantic difference feature map through a loss information capture module to obtain a photovoltaic module surface shallow detail-deep semantic information loss feature map, wherein the loss information capture module includes a point convolution layer, a nonlinear activation layer based on a tanh function, and an upsampling layer; The information compensation fusion unit is used to perform information compensation fusion on the shallow feature map of the photovoltaic module surface and the spatial semantic enhancement feature map of the photovoltaic module surface based on the shallow detail-deep semantic information loss feature map of the photovoltaic module surface to obtain the compensated photovoltaic module surface multi-scale fusion representation feature map.

8. The hot spot image automatic recognition system for automatic inspection by unmanned aerial vehicles according to claim 7 is characterized in that: The feature difference unit comprises: A downsampling subunit, used for downsampling the shallow feature map of the photovoltaic module surface to obtain a downsampled shallow feature map of the photovoltaic module surface; The position difference subunit is used to calculate the position subtraction between the shallow feature map of the downsampled photovoltaic component surface and the spatial semantic enhancement feature map of the photovoltaic component surface to obtain the shallow detail-deep semantic difference feature map of the photovoltaic component surface.

9. The hot spot image automatic recognition system for automatic inspection of unmanned aerial vehicles according to claim 8 is characterized in that: The information compensation fusion unit comprises: A shallow feature information compensation subunit is used to calculate the photovoltaic module surface shallow detail-deep semantic information loss feature map and the photovoltaic module surface shallow feature map by position point multiplication to obtain an information compensation photovoltaic module surface shallow feature map; The fusion subunit is used to calculate the position-weighted sum of the shallow feature map of the information-compensated photovoltaic component surface and the spatial semantic enhancement feature map of the photovoltaic component surface to obtain the multi-scale fusion representation feature map of the compensated photovoltaic component surface.

10. The hot spot image automatic recognition system for automatic inspection by unmanned aerial vehicles according to claim 9, characterized in that: The hot spot detection module is used for: The multi-scale fusion representation feature map of the surface of the compensated photovoltaic component is input into a detector based on a classifier to obtain the detection result, and the detection result is used to indicate whether there is a small hot spot on the photovoltaic component.