Visual guiding method applied to defects of photovoltaic power generation unit in mountain scene

Abstract

The invention relates to a visual guiding method applied to defects of a photovoltaic power generation unit in a mountain scene. Comprising the following steps: collecting image data and three-dimensional point cloud data of an area nearby a photovoltaic power generation unit in a mountain scene, and constructing a digital surface model based on the image data and the three-dimensional point cloud data of the area nearby the photovoltaic power generation unit in the mountain scene; extracting image data of the photovoltaic power generation unit in the image information of the area near the photovoltaic power generation unit in the mountain scene; constructing a photovoltaic power generation unit defect extraction model, inputting the image data of the photovoltaic power generation units into the photovoltaic power generation unit defect extraction model, and outputting the defective photovoltaic power generation units; and mapping the defective photovoltaic power generation units into the digital surface model, calculating a maintenance priority index of each defective photovoltaic power generation unit, and planning a maintenance route in the digital surface model based on the maintenance priority index of each defective photovoltaic power generation unit.

Description

Technical Field

[0001] This invention relates to a method for visualizing and guiding defects in photovoltaic power generation units in mountainous environments, belonging to the field of computer vision technology. Background Technology

[0002] With the rapid development of the photovoltaic industry, a large number of photovoltaic power generation projects have been built and put into operation. The maintenance difficulties of mountain photovoltaic power stations have gradually become prominent. The photovoltaic module layout area is scattered, equipment positioning is difficult, the module installation height is too high, equipment defects and faults are difficult to troubleshoot, and traditional troubleshooting and maintenance operations are difficult, inefficient, and cannot effectively troubleshoot hot spots and hidden cracks in the modules. Due to the special terrain of the mountainous area, the equipment could not be constructed according to the design. The unusable area of ​​the designed land could reach 30% to 50%. At the same time, due to the large area occupied, the review of the as-built drawings could not be fully covered, resulting in insufficient reference value of the construction drawings handed over to production, and failing to provide reliable guidance for production personnel to carry out equipment maintenance work. Due to the complex mountainous terrain and limited operating range, maintenance personnel need to constantly traverse complex terrain during equipment maintenance. The existing operating conditions result in high maintenance difficulty, and the maintenance relies on on-site manual screening using infrared imagers, which is a huge workload and inefficient.

[0003] The mountainous terrain is complex, and maintenance personnel need to constantly traverse complex terrain. The work involves working near edges, working at heights, and electrical work, which poses a high risk of falls from heights, mechanical injuries, and electric shocks. At the same time, the equipment is haphazardly constructed, there are no reliable electronic maps, and no effective guidance, which poses a risk of personnel accidentally entering non-working areas. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention proposes a method for visualizing and guiding defects in photovoltaic power generation units in mountainous environments.

[0005] The technical solution of the present invention is as follows: On the one hand, this invention provides a method for visualizing and guiding defects in photovoltaic power generation units in mountainous areas, comprising the following steps: Collect image data and 3D point cloud data of the area near the photovoltaic power generation unit in the mountain scene; A digital surface model is constructed based on image data and 3D point cloud data of the area near the photovoltaic power generation unit in the mountain scene. Extract image data of photovoltaic power generation units from image information of the area near photovoltaic power generation units in mountainous scenes; A defect extraction model for photovoltaic power generation units is constructed. Image data of photovoltaic power generation units are input into the model, and defective photovoltaic power generation units are output. Defective photovoltaic power generation units are mapped into a digital surface model, and the maintenance priority index for each defective photovoltaic power generation unit is calculated. Based on the maintenance priority index of each defective photovoltaic power generation unit, a maintenance route is planned in the digital surface model.

[0006] Preferably, the image data of the area near the photovoltaic power generation unit in the mountain scene includes visible light image data and infrared image data.

[0007] Preferably, the photovoltaic power generation unit defect extraction model includes a visible light image feature extraction branch, an infrared image feature extraction branch, a multi-scale attention fusion module, and a classification module.

[0008] Preferably, the formula for calculating the maintenance priority index is: ; in: Indicates the first Repair priority indicators for defective photovoltaic power generation units; Indicates the first The average slope of a fixed area surrounding a defective photovoltaic power generation unit; Indicates the first The risk of debris flow in a fixed area around a defective photovoltaic power generation unit; Indicates the first The power generation loss caused by a defective photovoltaic power generation unit; , , The weight parameters represent the formula for calculating maintenance priority.

[0009] Preferably, the specific steps for planning maintenance routes in the digital surface model based on the maintenance priority of each defective photovoltaic power generation unit are as follows: Each defective photovoltaic power generation unit is treated as a node, all nodes are interconnected, and weights are assigned to the connection edges between nodes to obtain a weighted navigation graph. The weight of the connection edge between the nodes is specifically obtained by weighted summation of the maintenance priority index of the two nodes and the distance between the two nodes; Based on the edge weights of all nodes, the Dijkstra algorithm is used to plan the maintenance route in the weighted navigation graph to obtain the shortest maintenance route.

[0010] Preferably, after drawing the corresponding shortest maintenance route in the digital surface model, the digital surface model is projected onto a plane to obtain a planar navigation guidance map.

[0011] On the other hand, the present invention also provides a defect visualization guidance system for photovoltaic power generation units in mountainous areas, including a data acquisition module, a digital surface model construction module, a defect extraction module, and a maintenance path planning module. The data acquisition module is used to collect image data and 3D point cloud data of the area near the photovoltaic power generation unit in the mountain scene. The digital surface model construction module is used to construct a digital surface model based on image data and three-dimensional point cloud data of the area near the photovoltaic power generation unit in the mountain scene. The defect extraction module is used to extract image data of photovoltaic power generation units in the image information of the area near the photovoltaic power generation unit in the mountain scene; to construct a photovoltaic power generation unit defect extraction model, input the image data of the photovoltaic power generation unit into the photovoltaic power generation unit defect extraction model, and output the defective photovoltaic power generation unit; The maintenance path planning module is used to map defective photovoltaic power generation units into a digital surface model, calculate the maintenance priority index of each defective photovoltaic power generation unit, and plan maintenance routes in the digital surface model based on the maintenance priority index of each defective photovoltaic power generation unit.

[0012] In another aspect, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the present invention.

[0013] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method described in the present invention.

[0014] The present invention has the following beneficial effects: 1. This invention enhances the robustness and accuracy of defect identification in photovoltaic power generation units by fusing visible light image and infrared image data and employing a defect extraction model with visible light image feature extraction branch, infrared image feature extraction branch, and multi-scale attention fusion model, thus adapting to complex and ever-changing mountainous environments.

[0015] 2. This invention combines image data and 3D point cloud data to construct a digital surface model, effectively restoring the complex terrain features of mountainous areas, and providing high-precision basic support for subsequent defect location and path planning.

[0016] 3. This invention utilizes indicators such as slope information, debris flow risk, and power generation loss to construct a maintenance priority model, reasonably assess the importance of each defect point, and realize priority-based intelligent path planning. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0020] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0021] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0022] The term “and / or” refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes these combinations.

[0023] See Figure 1 A method for visualizing and guiding defects in photovoltaic power generation units in mountainous areas includes the following steps: Collect image data and 3D point cloud data of the area near the photovoltaic power generation unit in the mountain scene; A digital surface model is constructed based on image data and 3D point cloud data of the area near the photovoltaic power generation unit in the mountain scene. Extract image data of photovoltaic power generation units from image information of the area near photovoltaic power generation units in mountainous scenes; A defect extraction model for photovoltaic power generation units is constructed. Image data of photovoltaic power generation units are input into the model, and defective photovoltaic power generation units are output. Defective photovoltaic power generation units are mapped into a digital surface model, and the maintenance priority index for each defective photovoltaic power generation unit is calculated. Based on the maintenance priority index of each defective photovoltaic power generation unit, a maintenance route is planned in the digital surface model.

[0024] In some embodiments, the image data of the area near the photovoltaic power generation unit in the mountain scene includes visible light image data and infrared image data.

[0025] In some embodiments, the photovoltaic power generation unit defect extraction model includes a visible light image feature extraction branch, an infrared image feature extraction branch, a multi-scale attention fusion module, and a classification module.

[0026] In one specific embodiment, the visible light image feature extraction branch is constructed based on the YOLOv8 model. The YOLOv8 model backbone network is constructed from three CSPNEXt modules, and the number of output channels of the three CSPNEXt modules are 128, 256, and 512, respectively. The CSPNEXt module contains multiple repeated blocks stacked hierarchically, each repeated block including... Dimensionality reduction convolutional layer Grouped convolutional layers, normalized activation layers, Dimensional convolutional layers and output layers; The visible light image data is processed by three CSPNEXt modules to extract features, resulting in three visible light feature maps at different scales.

[0027] In one specific embodiment, the infrared image feature extraction branch is built based on the ResNet50 model, which consists of four consecutive layers, each connected to a CBAM attention module. The outputs of each layer are weighted. The first layer includes three bottleneck layers and is responsible for the earliest infrared image data feature extraction, with an output resolution of 1 / 4 of the input feature map. The second layer includes four bottleneck layers, the third layer includes six bottleneck layers, and the fourth layer includes three bottleneck layers. The second, third, and fourth layers are successively located in deeper layers of the network. Each time the output of the previous layer enters the next layer, the spatial resolution is downsampled, and the number of channels increases exponentially. Infrared image data is processed through four levels of feature extraction to obtain an infrared feature map.

[0028] In one specific embodiment, the multi-scale attention fusion module is used to fuse visible light feature maps of different scales with infrared feature maps, specifically: For any scale of visible light feature map, it is fused with the infrared feature map, as shown in the following formula: ; ; ; ; in: Indicates the first The query vector matrix of visible light feature maps at each scale; express Convolution operation; Indicates the first Key vector matrix of visible light feature maps at various scales; Indicates a splicing operation; Indicates the first Infrared image feature weights at various scales; Indicates the transpose operation; Indicates the first Dimensions of each scale; Represents infrared feature map; express Convolution operation; Represents matrix multiplication; Indicates the first The fusion results of visible light feature maps and infrared feature maps at various scales.

[0029] In one specific embodiment, the multi-classification module includes a shared convolutional unit, a classification head, and a fusion unit; The shared convolutional unit includes two layers. Convolutional layers are used to perform two rounds of convolution operations on the fusion result of visible light feature maps and infrared feature maps at any scale to generate intermediate features at the current scale. The intermediate features at the current scale are input into the classification head, which consists of fully connected layers and outputs the defect probability at the current scale through the Softmax function or the Sigmoid function. The fusion unit is used to fuse the defect probabilities at various scales to obtain the final defect probability, and to judge photovoltaic power generation units with defect probabilities greater than a preset threshold as defective photovoltaic power generation units.

[0030] In some embodiments, the formula for calculating the maintenance priority index is: ; in: Indicates the first Repair priority indicators for defective photovoltaic power generation units; Indicates the first The average slope of a fixed area surrounding a defective photovoltaic power generation unit; Indicates the first The risk of debris flow in a fixed area around a defective photovoltaic power generation unit; Indicates the first The power generation loss caused by a defective photovoltaic power generation unit; , , The weight parameters represent the formula for calculating maintenance priority.

[0031] In one specific embodiment, the formula for calculating the debris flow risk is: ; in: Indicates the first Real-time rainfall intensity in a fixed area surrounding a defective photovoltaic power generation unit; Indicates reference rainfall intensity; Indicates the first The vegetation coverage of a fixed area around a defective photovoltaic power generation unit (value range 0 to 1). In some embodiments, the specific steps for planning maintenance routes in the digital surface model based on the maintenance priority of each defective photovoltaic power generation unit are as follows: Each defective photovoltaic power generation unit is treated as a node, all nodes are interconnected, and weights are assigned to the connection edges between nodes to obtain a weighted navigation graph. The weight of the connection edge between the nodes is specifically obtained by weighted summation of the maintenance priority index of the two nodes and the distance between the two nodes; Based on the edge weights of all nodes, the Dijkstra algorithm is used to plan the maintenance route in the weighted navigation graph to obtain the shortest maintenance route.

[0032] In some embodiments, after drawing the corresponding shortest maintenance route in the digital surface model, the digital surface model is projected onto a plane to obtain a planar navigation guidance map.

[0033] In some embodiments, a defect visualization guidance system for photovoltaic power generation units in mountainous areas is proposed, including a data acquisition module, a digital surface model construction module, a defect extraction module, and a maintenance path planning module; The data acquisition module is used to collect image data and 3D point cloud data of the area near the photovoltaic power generation unit in the mountain scene. The digital surface model construction module is used to construct a digital surface model based on image data and three-dimensional point cloud data of the area near the photovoltaic power generation unit in the mountain scene. The defect extraction module is used to extract image data of photovoltaic power generation units in the image information of the area near the photovoltaic power generation unit in the mountain scene; to construct a photovoltaic power generation unit defect extraction model, input the image data of the photovoltaic power generation unit into the photovoltaic power generation unit defect extraction model, and output the defective photovoltaic power generation unit; The maintenance path planning module is used to map defective photovoltaic power generation units into a digital surface model, calculate the maintenance priority index of each defective photovoltaic power generation unit, and plan maintenance routes in the digital surface model based on the maintenance priority index of each defective photovoltaic power generation unit.

[0034] In some embodiments, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method as described in any embodiment of the present invention.

[0035] In some embodiments, a computer-readable storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the method as described in any embodiment of the present invention.

[0036] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0037] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0038] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0039] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0040] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for visualizing and guiding defects in photovoltaic power generation units in mountainous environments, characterized in that, Includes the following steps: Collect image data and 3D point cloud data of the area near the photovoltaic power generation unit in the mountain scene; A digital surface model is constructed based on image data and 3D point cloud data of the area near the photovoltaic power generation unit in the mountain scene. Extract image data of photovoltaic power generation units from image information of the area near photovoltaic power generation units in mountainous scenes; A defect extraction model for photovoltaic power generation units is constructed. Image data of photovoltaic power generation units are input into the model, and defective photovoltaic power generation units are output. Defective photovoltaic power generation units are mapped into a digital surface model, and the maintenance priority index for each defective photovoltaic power generation unit is calculated. Based on the maintenance priority index of each defective photovoltaic power generation unit, a maintenance route is planned in the digital surface model.

2. The method for visualizing and guiding defects in photovoltaic power generation units in mountainous areas according to claim 1, characterized in that, The image data of the area near the photovoltaic power generation unit in the mountain scene includes visible light image data and infrared image data.

3. The method for visualizing and guiding defects in photovoltaic power generation units in mountainous areas according to claim 1, characterized in that, The photovoltaic power generation unit defect extraction model includes a visible light image feature extraction branch, an infrared image feature extraction branch, a multi-scale attention fusion module, and a classification module.

4. The method for visualizing and guiding defects in photovoltaic power generation units in mountainous areas according to claim 1, characterized in that, The formula for calculating the maintenance priority index is as follows: ； in: Indicates the first Repair priority indicators for defective photovoltaic power generation units; Indicates the first The average slope of a fixed area surrounding a defective photovoltaic power generation unit; Indicates the first The risk of debris flow in a fixed area around a defective photovoltaic power generation unit; Indicates the first The power generation loss caused by a defective photovoltaic power generation unit; , , The weight parameters represent the formula for calculating maintenance priority.

5. The method for visualizing and guiding defects in photovoltaic power generation units in mountainous areas according to claim 1, characterized in that, The specific steps for planning maintenance routes in the digital surface model based on the maintenance priority of each defective photovoltaic power generation unit are as follows: Each defective photovoltaic power generation unit is treated as a node, all nodes are interconnected, and weights are assigned to the connection edges between nodes to obtain a weighted navigation graph. The weight of the connection edge between the nodes is specifically obtained by weighted summation of the maintenance priority index of the two nodes and the distance between the two nodes; Based on the edge weights of all nodes, the Dijkstra algorithm is used to plan the maintenance route in the weighted navigation graph to obtain the shortest maintenance route.

6. The method for visualizing and guiding defects in photovoltaic power generation units in mountainous areas according to claim 5, characterized in that, After drawing the corresponding shortest maintenance route in the digital surface model, the digital surface model is projected onto the plane to obtain the planar navigation guidance map.

7. A visual guidance system for defects in photovoltaic power generation units applied in mountainous scenarios, characterized in that, It includes a data acquisition module, a digital surface model construction module, a defect extraction module, and a maintenance path planning module; The data acquisition module is used to collect image data and 3D point cloud data of the area near the photovoltaic power generation unit in the mountain scene. The digital surface model construction module is used to construct a digital surface model based on image data and three-dimensional point cloud data of the area near the photovoltaic power generation unit in the mountain scene. The defect extraction module is used to extract image data of photovoltaic power generation units in the image information of the area near the photovoltaic power generation unit in the mountain scene; to construct a photovoltaic power generation unit defect extraction model, input the image data of the photovoltaic power generation unit into the photovoltaic power generation unit defect extraction model, and output the defective photovoltaic power generation unit; The maintenance path planning module is used to map defective photovoltaic power generation units into a digital surface model, calculate the maintenance priority index of each defective photovoltaic power generation unit, and plan maintenance routes in the digital surface model based on the maintenance priority index of each defective photovoltaic power generation unit.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 6.

Images