A machine vision-based photovoltaic field station outdoor inspection method and device

Abstract

Embodiments of the present application provide a photovoltaic field station outdoor inspection method and device based on machine vision. A UAV flies according to an inspection path, uses a visible light sensor to collect a visible light image of a photovoltaic panel, and uses an infrared sensor to collect an infrared image of the photovoltaic panel. The UAV extracts the pixel point distribution of the same pixel value in the visible light image, and determines whether the photovoltaic panel has a defect area according to the pixel point distribution of the same pixel value in the visible light image. When it is determined that there is a defect, the UAV sends the visible light image, the infrared image, and the image shooting point marking the defect area to a control platform. The control platform determines the photovoltaic panel with the defect and the defect type according to the visible light image, the infrared image, and the image shooting point marking the defect area. Without using a convolutional neural network for recognition, the data processing efficiency can be improved, the use of hardware resources can be reduced, and the equipment cost can be reduced.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to a method and apparatus for outdoor inspection of photovoltaic power stations based on machine vision. Background Technology

[0002] With the rapid development of the photovoltaic industry, the scale of photovoltaic power stations is constantly expanding, and the difficulty of inspection work is also gradually increasing.

[0003] Initially, the maintenance and inspection of photovoltaic panels were carried out manually, with inspectors using handheld testing equipment to check each panel one by one. With the development of drone technology, the application of drones in the operation and maintenance inspection of photovoltaic power plants has gradually matured. Traditional drone inspections of photovoltaic power plants mainly involve: after arriving at the photovoltaic power plant, the pilot surveys the site and formulates a flight plan based on experience and / or survey data. After the drone is installed, the pilot controls the drone to fly according to the flight plan. After the flight, images are downloaded from the drone and analyzed using image recognition models.

[0004] However, when using image recognition models for image recognition, the computation time required for feature extraction through multiple convolutions is relatively long, affecting the efficiency of the image recognition model. Furthermore, running image recognition models places high demands on the hardware resources of the device, thus increasing equipment costs. Summary of the Invention

[0005] This application provides a machine vision-based outdoor inspection method and apparatus for photovoltaic power stations, which aims to improve image processing efficiency and reduce hardware resource consumption.

[0006] Some embodiments of this application provide a machine vision-based outdoor inspection method for photovoltaic power stations. The method is applied to a drone and a control platform, with communication between the drone and the control platform. The method includes:

[0007] The drone flies along the inspection path, using a visible light sensor to collect visible light images of the photovoltaic panels and an infrared sensor to collect infrared images of the photovoltaic panels.

[0008] The drone extracts the distribution of pixels with the same pixel value in the visible light image, and determines whether there are defective areas in the photovoltaic panel based on the distribution of pixels with the same pixel value in the visible light image;

[0009] When a defect is identified, the drone sends visible light and infrared images of the defective area, as well as the image capture point, to the control platform.

[0010] The control platform determines the photovoltaic panel with defects and the type of defects based on the visible light image, infrared image, and image capture point of the marked defect area.

[0011] Some embodiments of this application provide a control platform, including: a memory and a processor;

[0012] The memory stores computer-executed instructions;

[0013] The processor executes computer execution instructions stored in the memory, causing the processor to perform the various possible implementations described above.

[0014] This application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the various possible implementation methods described above.

[0015] This application provides a computer program product, including a computer program that, when executed by a processor, implements the various possible implementations described above. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] Figure 1 This is a schematic diagram of the layout of a photovoltaic panel;

[0018] Figure 2 A flowchart illustrating a machine vision-based outdoor inspection method for photovoltaic power stations, provided for some embodiments of this application;

[0019] Figure 3 These are schematic diagrams of the appearance of a photovoltaic panel provided in some embodiments of this application;

[0020] Figure 4 This is a partial schematic diagram of a visible light image of a photovoltaic panel provided in some embodiments of this application;

[0021] Figure 5 This is an imaging schematic diagram of a visible light sensor provided in some embodiments of this application.

[0022] Figure label:

[0023] 10. Photovoltaic panel; 100. Photovoltaic module; 101. Grid line; 102. Connecting line; X, horizontal; Y, vertical.

[0024] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0025] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0026] Figure 1 This is a schematic diagram of the layout of a photovoltaic panel, such as... Figure 1 As shown, numerous photovoltaic (PV) panels are arranged in arrays, typically on mountainsides, water surfaces, or residential rooftops. These panels collect solar energy and convert it into electricity. To utilize the electricity generated by the PV panels, energy storage devices or inverters are also installed in the PV power plant. A portion of the electricity generated by the PV panels is stored in the energy storage devices, or it is boosted by the inverter and then fed into the power grid.

[0027] After the completion of photovoltaic power stations, their operation and maintenance have received more attention. As photovoltaic panels constitute a large number of components in a photovoltaic power station, their maintenance and inspection are a demanding task.

[0028] Initially, the maintenance and inspection of photovoltaic panels were carried out manually, with inspectors using handheld testing equipment to check each panel one by one. With the development of drone technology, the application of drones in the operation and maintenance inspection of photovoltaic power plants has gradually matured. Traditional drone inspections of photovoltaic power plants mainly involve: after arriving at the photovoltaic power plant, the pilot surveys the site and formulates a flight plan based on experience and / or survey data. After the drone is installed, the pilot controls the drone to fly according to the flight plan. After the flight, images are downloaded from the drone and analyzed using image recognition models.

[0029] However, when using image recognition models for image recognition, the computation time required for feature extraction through multiple convolutions is relatively long, affecting the efficiency of the image recognition model. Furthermore, running image recognition models places high demands on the hardware resources of the device, thus increasing equipment costs.

[0030] In view of this, this application provides a machine vision-based outdoor inspection method for photovoltaic power stations, aiming to propose a new image recognition algorithm. Since it does not require image recognition models for image processing, it can reduce hardware configuration, lower equipment costs, and improve the processing efficiency of the recognition algorithm.

[0031] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0032] Some embodiments of this application provide an outdoor inspection device for photovoltaic power plants based on machine vision, the device including a control platform and a drone.

[0033] Drones are controlled to conduct aerial surveys of the entire photovoltaic area, generating an electronic map of the photovoltaic power station. The drones, equipped with high-definition aerial survey cameras, acquire panoramic image data of the photovoltaic power station, which is then used to create an electronic map using image stitching software. By dividing the electronic map into regions for different photovoltaic panels and numbering the panels within each region, the coordinate information of the photovoltaic panels in that region is obtained, thus constructing a photovoltaic panel database.

[0034] The control platform determines the inspection time based on weather and / or the output power of the photovoltaic power station, and generates an inspection route based on an electronic map. The control platform issues flight commands, which include the inspection time and specific instructions. Upon receiving the flight commands, the UAV autonomously cruises along the inspection route and performs terrain-mimicking flight according to the flight parameters set by the control platform. The UAV is equipped with infrared and visible light sensors. During flight, it collects visible light and infrared images of the photovoltaic panels. Figure 2 A flowchart illustrating a machine vision-based outdoor inspection method for photovoltaic power plants, as provided in some embodiments of this application, is shown below. Figure 2 As shown, the method specifically includes:

[0035] S101. The drone flies along the inspection path, using a visible light sensor to collect visible light images of the photovoltaic panels and an infrared sensor to collect infrared images of the photovoltaic panels.

[0036] The process involves conducting on-site surveys of the photovoltaic power station to obtain the area to be inspected and the layout of the photovoltaic panels within that area. An inspection path is then developed based on this layout. A drone then flies along the inspection path to inspect the photovoltaic panels within the area.

[0037] The drone is equipped with visible light and infrared sensors. The visible light sensor is used to detect dirt and cracks on the photovoltaic panels. The infrared sensor is used to detect hot spots on the photovoltaic panels.

[0038] When the drone flies over the area to be inspected, its visible light sensor and infrared sensor are activated. The visible light sensor captures visible light images of the photovoltaic panels, and the infrared sensor captures infrared images of the photovoltaic panels. At the same image capture point, the visible light sensor captures one frame of visible light image, and the infrared sensor captures one frame of infrared image.

[0039] In short, after receiving flight commands from the control platform, the drone flies along a pre-set inspection path. During flight, the drone's visible light sensor and infrared sensor activate simultaneously to photograph the photovoltaic panels. The visible light sensor captures the surface condition of the photovoltaic panels, generating a visible light image; the infrared sensor detects the thermal radiation of the photovoltaic panels, generating an infrared image. These two types of image data will be used for subsequent defect detection and analysis.

[0040] S102. The UAV extracts the distribution of pixels with the same pixel value in the visible light image, and determines whether there are defective areas in the photovoltaic panel based on the distribution of pixels with the same pixel value in the visible light image.

[0041] Optionally, the drone's built-in image processing module processes the acquired visible light images. First, by comparing pixel values ​​column-by-column or block-by-block, it extracts regions in the image with identical and continuous pixel values. These regions typically correspond to grid lines, connecting lines, or abnormal areas formed by defects (such as cracks or dirt) on photovoltaic panels. Then, the size and shape of these regions are compared with preset specification values ​​to determine if any anomalies exist. For example, if the width or shape of a region with consecutive identical pixel values ​​does not meet expectations, it may be considered a defective region.

[0042] Example 3 is a schematic diagram of the appearance of the photovoltaic panel 10 provided in some embodiments of this application, such as... Figure 3 As shown, a photovoltaic panel typically consists of multiple photovoltaic modules 100 arranged in an array. A transparent glass cover and frame (not shown) are also laid on the photovoltaic modules for protection. Each photovoltaic module 100 includes a semiconductor substrate, multiple gate lines 101 located on the surface of the semiconductor substrate, and multiple connecting lines 102 located above the gate lines 101. The multiple gate lines 101 extend longitudinally (Y), and the multiple connecting lines 102 extend laterally (X). The multiple gate lines 101 and the multiple connecting lines 102 together divide the semiconductor substrate into multiple pixel regions of the same color. Both the multiple gate lines 101 and the multiple connecting lines 102 are metallic conductors and also form pixel regions of the same color.

[0043] Figure 4 This is a partial schematic diagram of a visible light image of a photovoltaic panel 10 provided in some embodiments of this application, such as... Figure 4As shown, the pixel value of the slash-filled pixels is 156, and the pixel value of the blank-filled pixels is 230. The slash-filled pixels divide the blank-filled pixels into multiple rectangular regions. If the surface of the photovoltaic module 100 has dirt, cracks, or hot spots, the area where the blank-filled pixels are located will be an irregularly shaped rectangle or a rectangle whose size does not meet the specifications. By statistically analyzing the distribution of the blank-filled pixels, it can be determined whether there are defective areas in the photovoltaic panel 10.

[0044] S103. When it is determined that there are no defects, the drone deletes the visible light image, infrared image, and image capture point.

[0045] In this process, the drone identifies visible light images. If it determines that there are no defects in the visible light image, there is no need to send the visible light image, an infrared image taken at the same shooting point, or the image shooting point to the control platform. This reduces the amount of data transmission between the drone and the control platform, improving the efficiency of the inspection operation.

[0046] S104. When a defect is confirmed, the UAV will send visible light images, infrared images, and image capture points of the marked defect area to the control platform.

[0047] If a defect is determined in the visible light image, the visible light image, infrared image, and image capture point of the marked defect area are sent to the control platform. The visible light image and infrared image of the marked defect area are taken from the same capture point. The control platform combines the infrared image and the visible light image of the marked defect area to determine the defect.

[0048] Optionally, once a defective area is detected, the drone will automatically mark these areas and package the marked visible light image, the corresponding infrared image, and the location information at the time of capture (image capture point) and send them to the control platform. This process ensures that the control platform can accurately receive all necessary data related to the defect for further analysis and processing.

[0049] S105. The control platform determines the photovoltaic panel with defects and the type of defects based on the visible light image, infrared image, and image capture point of the marked defect area.

[0050] Upon receiving the aforementioned data, the control platform converts the visible light image to a geodetic coordinate system based on the image capture point and parameters of the visible light sensor to determine the captured geographical area. It then compares the photovoltaic panel layout data with the captured geographical area to identify defective photovoltaic panels. Finally, based on the pixel values ​​in the visible light image and the temperature data in the infrared image of the marked defective area, it determines whether the defect in the photovoltaic panel is a crack, dirt, or a hot spot.

[0051] Optionally, after receiving data from the drone, the control platform first calculates the actual geographical location of each pixel in the image based on the image capture point and the field of view of the visible light sensor. Then, combining this with the layout data of the photovoltaic field, it matches these geographical locations with the positions of the photovoltaic panels to determine which panel the defect occurs on. Next, the control platform combines the defect area features in the visible light image and the temperature data in the infrared image to further analyze the type of defect. For example, by using shape features in the visible light image and temperature anomalies in the infrared image, different types of defects such as cracks, dirt, and hot spots can be distinguished.

[0052] In the above technical solution, gate lines and interconnects are arranged in a regular pattern above the semiconductor substrate. This results in a regular distribution of pixel values ​​in the captured images. The presence of defective areas in the photovoltaic panel can be determined by analyzing the distribution of pixels with the same pixel value in the visible light image, eliminating the need for convolutional neural networks. This improves data processing efficiency, reduces hardware resource usage, and lowers equipment costs. Due to the increased data processing efficiency, there is no need to allocate significant storage space on the drone to store unprocessed visible light images, nor is it necessary to rely on the control platform to store unprocessed visible light images. Only images containing defects can be transmitted to the control platform, reducing data transmission volume.

[0053] It's worth noting that traditional methods often rely on convolutional neural networks for image recognition, which requires significant computational resources and time. The aforementioned solution, however, identifies defects by directly analyzing the distribution of pixels with the same pixel value in visible light images, avoiding complex convolutional operations and significantly improving data processing efficiency. Due to the increased data processing speed, the drone can analyze images in real-time during flight, quickly identify and mark defects, reduce inspection cycles, and improve the real-time nature of inspections.

[0054] Furthermore, image processing without relying on high-performance computing hardware reduces the hardware configuration requirements of the drone and control platform, thereby reducing hardware costs. Since there is no need to store large amounts of unprocessed image data on the drone, only defective images are transmitted, significantly saving storage space and improving the utilization rate of storage resources.

[0055] Furthermore, by combining visible light and infrared images, the types and locations of defects (such as cracks, dirt, and hot spots) can be identified more accurately. Visible light images are used for initial screening of defect areas, while infrared images provide temperature information to further confirm the defect type. By analyzing the pixel distribution in visible light images, normal photovoltaic panel structures and defective areas can be effectively distinguished, reducing false alarms; at the same time, the auxiliary verification using infrared images also reduces the risk of missed alarms.

[0056] Furthermore, the drones autonomously fly along preset inspection paths and collect images without human intervention, automating the inspection process. Upon receiving defect data, the control platform automatically generates cleaning work orders or maintenance instructions and intelligently dispatches cleaning robots or maintenance personnel to handle the issues, improving overall operation and maintenance efficiency. Timely detection and handling of photovoltaic panel defects can prevent power generation reduction or equipment damage caused by defect expansion. Regular inspections of photovoltaic panels allow for intervention in the early stages of defects, extending the lifespan of the panels and reducing maintenance costs.

[0057] In one possible implementation, S102, the drone extracts the distribution of pixels with the same pixel value in the visible light image, and determines whether there are defective areas in the photovoltaic panel based on the distribution of pixels with the same pixel value in the visible light image, specifically including:

[0058] S201. Extract the pixels in each column of the visible light image, compare the pixels in each column, and obtain the width, starting pixel column, and ending pixel column of consecutive identical columns.

[0059] Among them, consecutive identical columns refer to columns of pixels with the same pixel value distribution that are adjacent to each other.

[0060] Optionally, the drone first extracts the pixels from each column of the visible light image and compares these pixels. Specifically, the drone divides the image into multiple columns and compares the values ​​of adjacent pixels column by column. When a column is found to have all the same pixel values ​​that are also the same as the previous column, these columns are considered a "consecutive identical column". This step aims to find all combinations of pixel columns in the image where the pixel values ​​are consecutive and identical.

[0061] For each consecutive identical column, the drone calculates its width (i.e., the number of pixel columns it contains), the starting pixel column, and the ending pixel column. The width refers to the number of pixel columns contained in the consecutive identical column; the starting and ending pixel columns identify the start and end positions of that consecutive identical column in the image, respectively.

[0062] by Figure 4 Taking a local pixel region of the visible light image as an example, the local pixel region is a pixel array of 17 rows × 11 columns.

[0063] The pixels in columns 1 and 8 are filled with diagonal lines, representing two vertically extending gate lines. The pixel distribution in columns 2 through 7 and columns 9 through 11 is identical: the pixels in rows 2 and 16 are filled with diagonal lines. Pixels in columns 2 through 7 form a continuous identical column, 6 pixels wide. The starting column of this continuous identical column is column 2, and the ending column is column 7. Pixels in columns 9 through 11 form a continuous identical column, 3 pixels wide. The starting column of this continuous identical column is column 9, and the ending column is column 11.

[0064] S202. For each consecutive identical column, determine whether the width of the consecutive identical column is within the preset width set. If yes, proceed to S203; otherwise, proceed to S204.

[0065] The preset width set includes multiple standard widths. The standard width refers to the pixel width difference between two adjacent gate lines, the pixel width difference between a gate line and the edge of the photovoltaic module, the pixel width difference between two adjacent connection lines, and the pixel width difference between a connection line and the edge of the photovoltaic module.

[0066] Optionally, the drone compares the width of each consecutive identical column with a preset width set. This preset width set contains standardized width values ​​calculated based on the photovoltaic panel design and image scaling. If the width of consecutive identical columns falls within the preset width set, the column is considered standard and without defects; otherwise, the process proceeds to the next step.

[0067] The system acquires the distances between two adjacent gate lines, the distance between the photovoltaic module edge and the gate lines, the distance between two adjacent connection lines, and the distance between the connection lines and the photovoltaic module edge. The image scaling is determined based on the internal and external parameters of the visible light image sensor. The standard width is calculated based on the image scaling and the distances between the two adjacent gate lines, the photovoltaic module edge and the gate lines, the image scaling and the distances between the two adjacent connection lines, and the photovoltaic module edge and the gate lines.

[0068] If there are no cracks, dirt, or hot spots on the photovoltaic module, then consecutive identical columns are divided by gate lines or connecting lines, and the width of consecutive identical columns is within the preset width range. If there are cracks, dirt, or hot spots on the photovoltaic module, then consecutive identical columns are divided not only by gate lines or connecting lines but also by defects such as cracks, dirt, or hot spots, and the width of consecutive identical columns is less than the preset width range specification width.

[0069] S203. If the width of consecutive identical columns is determined to be within a preset width set, then it is determined that there are no defects in consecutive identical columns.

[0070] Specifically, for each consecutive identical column, the width of the consecutive identical column is compared with any standard width in the preset width set to determine whether the width of the consecutive identical column is within the preset width set. If so, it can be determined that there is no defect in the consecutive identical column.

[0071] For example: with Figure 4 Taking a local pixel region of the visible light image as an example, the pixels in columns 2 to 7 form a continuous identical column, with a width of 6 pixels. The pixel width between two gate lines is 6 pixels, the pixel width between two connecting lines is 4 pixels, and the difference between the connecting lines and the edge of the photovoltaic panel is 11 pixels. The width of the continuous identical column formed by the pixels in columns 2 to 7 is within a preset width set. Therefore, there are no defects in the continuous identical column formed by the pixels in columns 2 to 7.

[0072] S204. For each consecutive identical column, determine that the width of the consecutive identical column is not within the preset width set, and continue to determine whether the consecutive identical column contains the first column pixels or the last column pixels. If yes, proceed to S205; otherwise, proceed to S206.

[0073] Specifically, for each consecutive identical column, the width of the consecutive identical column is compared with any standard width in the preset width set. If it is determined that the width of the consecutive identical column is not in the preset width set, then it is further determined whether the consecutive identical column contains the first column of pixels or the last column of pixels, thus eliminating the case where the width is not the standard width due to the incomplete photovoltaic panel being photographed.

[0074] Optionally, for consecutive identical columns whose width is not within a preset width set, the drone further determines whether they contain the first or last column of pixels in the image. Since image edges may be incomplete due to shooting angle or cropping, consecutive identical columns containing image edges may appear irregular due to edge effects. If a consecutive identical column contains the first or last column of pixels, it is considered a normal column with no defects; otherwise, the process proceeds to the next step.

[0075] S205. If consecutive identical columns contain pixels from the first column or the last column, then it is determined that there is no defect in consecutive identical columns.

[0076] Specifically, if the width of consecutive identical columns is not within a preset width set, it can be determined that the consecutive identical columns are not divided by two adjacent gate lines, or by connecting lines and borders. To rule out errors due to incomplete photovoltaic module imaging, it is determined whether the consecutive identical columns contain either the first column of pixels or the last column of pixels.

[0077] For example: with Figure 4 Taking a local pixel region of the visible light image as an example, the consecutive identical columns formed by pixels in columns 9 to 11 have a width of 3 pixels and are not located within a preset width set. Furthermore, the consecutive identical columns formed by pixels in columns 9 to 11 include the last column of pixels; therefore, the consecutive identical columns formed by pixels in columns 9 to 11 do not have defects.

[0078] S206. If consecutive identical columns do not contain pixels from the first column and the last column, then the area corresponding to consecutive identical columns shall be regarded as a defect area.

[0079] If it is determined that the width of consecutive identical columns is not within the preset width set, and the consecutive identical columns do not contain the first column of pixels and the last column of pixels, then it can be determined that the consecutive identical columns are not divided by two adjacent gate lines, or are not divided by connecting lines and borders, nor are they caused by not capturing the complete photovoltaic panel. Therefore, the area corresponding to the consecutive identical columns is regarded as a defect area.

[0080] Optionally, for consecutive identical columns that are neither within the preset width range nor contain image edges, the drone marks the corresponding image region as a defect area. These areas are likely due to abnormal pixel value distribution caused by cracks, dirt, or other physical damage on the photovoltaic panel surface. Marking these areas helps in subsequent analysis and processing of the defect type.

[0081] In the above technical solution, since the gate lines and connecting lines are distributed horizontally or vertically, and the pixel values ​​in the area where the underlying semiconductor substrate is located are the same, the semiconductor substrate will be divided into multiple blocks, resulting in many continuous areas with the same pixel values ​​in the image. The solution compares two columns of pixels, extracting adjacent columns with the same pixel value distribution, column by column. Furthermore, the standard width of the continuous identical columns is determined based on the scaling ratio of the photovoltaic panel in the image. After extracting the continuous identical columns from the image, the width of the continuous identical columns is compared with the standard width to determine if defects exist. Comparing at the column level improves data processing efficiency and eliminates cases where the width is not the standard width due to incomplete photovoltaic panel imaging, thus improving the accuracy of defect identification.

[0082] It's worth noting that traditional methods might require pixel-by-pixel analysis of the entire image, while the above scheme compares data column by column, significantly reducing the amount of data to be processed and thus improving the efficiency of data processing and defect identification. Furthermore, by calculating the width of consecutive identical columns and comparing it with a preset width set, columns that do not meet the specified width can be quickly located, thereby identifying defect areas and achieving rapid defect identification.

[0083] Furthermore, the above scheme also considers the possibility that image edges may be incomplete due to shooting angle or cropping. By determining whether consecutive identical columns contain pixels from the first or last column, the influence of edge effects on defect identification is effectively eliminated, improving the accuracy of identification. Moreover, the standardized width of consecutive identical columns is determined based on the scaling ratio of the photovoltaic panel in the image and compared with the actual width. This dynamic adjustment based on the actual situation further improves the accuracy of defect identification.

[0084] Furthermore, this solution employs a simple pixel-column comparison algorithm, eliminating the need for complex image recognition or deep learning algorithms, thus reducing the hardware resource requirements for the drone and control platform. Since only the defective image region needs to be transmitted, rather than the entire image, the amount of data transmitted is significantly reduced, lowering the communication bandwidth requirements.

[0085] Furthermore, regardless of whether the grid lines and connecting lines of the photovoltaic panel are distributed horizontally or vertically, this solution can effectively extract and compare consecutive identical columns, making it suitable for defect detection of different types of photovoltaic panels. The specification width can also be flexibly adjusted according to the actual size of the photovoltaic panel and the image scaling ratio, making the solution more adaptable. Through regular inspections and rapid defect identification, defects in photovoltaic panels can be detected in a timely manner, preventing defects from escalating and causing a decrease in power generation or equipment damage.

[0086] In one possible implementation, S102, the drone extracts the distribution of pixels with the same pixel value in the visible light image, and determines whether there are defective areas in the photovoltaic panel based on the distribution of pixels with the same pixel value in the visible light image, specifically including:

[0087] S301. Extract connected regions with the same pixel value from the visible light image, and determine whether the connected region is a rectangular region. If yes, proceed to S302; otherwise, proceed to S305.

[0088] In photovoltaic modules, multiple gate lines extend vertically, and multiple connecting lines extend horizontally. The multiple gate lines and multiple connecting lines form a grid pattern, creating a grid-shaped connected region in a visible light image. The horizontally extending connecting lines and the vertically extending gate lines divide the semiconductor substrate into multiple regions, forming multiple rectangular connected regions in a visible light image.

[0089] Compare the pixel values ​​of each pixel in the visible light image. If pixels with the same pixel value are arranged continuously to form a connected region without any gaps, this region is considered as a connected region with the same pixel value.

[0090] Optionally, the drone first acquires visible light images of the photovoltaic panel using its onboard visible light sensor. Then, image processing algorithms are used to analyze the image and extract connected regions with the same pixel values. These connected regions may be formed due to identical pixel values ​​in the gate lines, connecting lines, and semiconductor substrate of the photovoltaic panel surface. Connected region extraction can be achieved using image segmentation algorithms, such as thresholding and region growing. The algorithm traverses the entire image, grouping adjacent pixels with the same pixel value into the same connected region.

[0091] by Figure 4 Taking a local pixel region of the visible light image as an example, the local pixel region is a 17-row × 11-column pixel array. The pixels filled with upward diagonal lines in rows 2 and 16 are connecting lines extending horizontally. The pixels filled with upward diagonal lines in columns 1 and 8 are gate lines extending vertically.

[0092] The pixels filled with diagonal lines in the second row, the sixteenth row, the first column, and the eighth column form a connected region in a grid pattern.

[0093] The rectangular area formed by the four vertices of the pixels in the 3rd row of the 2nd column, the 15th row of the 2nd column, the 3rd row of the 7th column, and the 15th row of the 7th column is a connected region filled entirely with white.

[0094] S302. If the connected region is a rectangular region, determine whether the size of the rectangular region is within the preset size set. If yes, proceed to S303; otherwise, proceed to S304.

[0095] Optionally, for each extracted connected region, the UAV first determines whether it is a rectangular region. If it is a rectangular region, its dimensions (such as length and width) are further measured and compared with a preset size set. Rectangular regions are determined by checking whether the boundaries of the connected region are composed of straight lines and whether opposite sides are parallel and of equal length. The preset size set is determined based on the design and manufacturing specifications of photovoltaic panels and includes the expected size range for normal photovoltaic panel surface structures (such as gate lines, connecting lines, etc.). If the dimensions of a rectangular region fall within the preset size set, the region is considered normal and without defects; if the dimensions do not fall within the preset size set, the region is marked as a defective region.

[0096] The scaling factor of the image is determined based on the internal and external parameters of the visible light image sensor.

[0097] For a rectangular region segmented by gate lines and interconnects, the distance between two adjacent gate lines and the distance between two adjacent interconnects on the photovoltaic module are obtained. The standard width of the rectangular region is calculated based on the image scaling and the distance between two adjacent gate lines. The standard length of the rectangular region is obtained based on the image scaling and the distance between two adjacent interconnects.

[0098] For a rectangular region segmented by gate lines, connecting lines, and the longitudinal edge of the photovoltaic module, the standard width of the rectangular region is calculated based on the image scaling ratio and the distance between the gate lines and the longitudinal edge of the photovoltaic module, and the standard length of the rectangular region is calculated based on the image scaling ratio and the distance between two adjacent connecting lines.

[0099] For a rectangular region segmented by gate lines, connecting lines, and the lateral edge of the photovoltaic module, the standard width of the rectangular region is calculated based on the image scaling ratio and the distance between two gate lines, and the standard length of the rectangular region is calculated based on the image scaling ratio and the distance between the connecting lines and the lateral edge of the photovoltaic module.

[0100] For a rectangular region segmented by gate lines, connecting lines, and the horizontal and vertical edges of the photovoltaic module, the standard width of the rectangular region is calculated based on the image scaling ratio and the distance between the gate lines and the vertical edge of the photovoltaic module, and the standard length of the rectangular region is calculated based on the image scaling ratio and the distance between the connecting lines and the horizontal edge of the photovoltaic module.

[0101] The preset size set includes the standard dimensions of multiple rectangular regions, and the standard dimensions of the rectangular regions include the standard width and the standard length of the rectangular regions.

[0102] If the connected region is a rectangular region, the size of the connected region is compared with the standard sizes in the preset size set to determine whether the size of the rectangular region is within the preset size set.

[0103] S303. If the size of the rectangular region is determined to be within a preset size set, there are no defects within the connected region.

[0104] If the size of the rectangular region is determined to be within a preset size set, it indicates that the connected region is divided by gate lines and connecting lines. Therefore, it is determined that there are no defects within this connected region.

[0105] S304. If the size of the rectangular region is not located in the preset size set, the connected region will be regarded as the defect region.

[0106] If the size of the rectangular region is not within the preset size set, it indicates that the connected region is divided by dirt, cracks, or hot spots. Therefore, it is determined that there is a defect in the connected region.

[0107] S305. If the connected region is a grid-shaped region, determine whether the lengths of multiple horizontal regions in the grid-shaped region are the same, and whether the lengths of multiple vertical regions in the grid-shaped region are the same. If they are all the same, proceed to S306; otherwise, proceed to S307.

[0108] The grid-shaped region includes horizontally arranged connecting lines and vertically arranged gate lines, with all gate lines and connecting lines having the same length. To determine if a region is grid-shaped, it is determined whether the lengths of multiple horizontal regions and multiple vertical regions within the grid-shaped region are the same, thus identifying any missing gate lines or connecting lines.

[0109] If all horizontal regions are of the same length, then there are no missing connecting lines. If the lengths of the horizontal regions are not the same, then shorter connecting lines are missing. If the lengths of the vertical regions are the same, then there are no missing gate lines, but shorter gate lines are missing.

[0110] Optionally, for connected regions that are not rectangular but arranged in a tic-tac-toe pattern, the UAV needs to determine whether the lengths of its multiple horizontal and vertical regions are the same. The determination of a tic-tac-toe region can be made by checking whether the connected region is composed of multiple intersecting straight lines (or approximately straight lines) that divide the region into multiple small rectangles or squares. Then, the lengths of these small rectangles or squares are measured, and the lengths of the horizontal and vertical regions are compared to see if they are consistent. If all horizontal regions in the tic-tac-toe region have the same length, and all vertical regions also have the same length, the region is considered normal and without defects; if the lengths of the horizontal or vertical regions are not the same, the region is marked as a defective region.

[0111] S306. If both the horizontal and vertical regions are the same, the connected region has no defects.

[0112] If both the horizontal and vertical regions are identical, it means that neither the horizontally arranged connecting lines nor the vertically arranged gate lines are missing, and therefore there are no defects in the grid-shaped connected regions.

[0113] S307. If the horizontal or vertical regions are not the same, the connected region shall be regarded as the defect region.

[0114] Specifically, if the horizontal regions are not identical, the horizontally arranged connecting lines are defective, and short connecting lines are missing. If the vertical regions are not identical, the vertically arranged gate lines are defective, and short gate lines are missing. If the horizontal or vertical regions are not identical, the connected region is considered a defective region. Furthermore, either the short horizontal region or the short vertical region is considered a defective region.

[0115] Optionally, for connected regions marked as defective areas, the UAV records their position, size, and other information in the visible light image and transmits this information along with the image to the control platform. Defective areas can be marked by adding annotations to the image, changing pixel colors or values, etc. The data transmitted to the control platform may include a compressed version of the image, the coordinates and size of the defective area, and other metadata. After receiving the data, the control platform can further analyze the type and severity of the defect and formulate corresponding repair or maintenance plans accordingly.

[0116] In the above technical solution, since the gate lines and connecting lines are distributed horizontally or vertically, and because the pixel values ​​of the areas where the underlying semiconductor substrate is located are the same, the semiconductor substrate is divided into multiple blocks. The image will have many rectangular connected regions with the same pixel values, as well as connected regions in a grid pattern. By extracting the connected regions in the image, it is determined whether the connected region is a rectangular region or a grid pattern. If neither is true, it indicates that the region has a defect. If it is a rectangular region or a grid pattern, the region size is further determined to see if it is within the standard width, further eliminating false positives and thus improving the accuracy of identification. Because defect identification is based on connected regions, which are usually relatively small, the defect area can be located more accurately.

[0117] It is worth noting that by extracting connected regions with the same pixel value from the visible light image, the technical solution directly focuses on the key information in the image, namely the structural features of the photovoltaic panel surface. This avoids pixel-by-pixel analysis of the entire image, greatly improving the efficiency of defect information extraction.

[0118] For a rectangular connected region, by determining whether its size is within a preset size set, the technical solution can accurately identify regions with abnormal sizes, which are likely to have defects.

[0119] For grid-shaped connected regions, by comparing whether the lengths of multiple horizontal and vertical regions are the same, the technical solution can further filter out regions with irregular structures, which are also potential defective regions.

[0120] As can be seen, the above technical solution not only considers the shape of the connected region (rectangular or grid-like) but also incorporates the judgment of size or length. This multi-dimensional analysis effectively reduces misjudgments caused by a single feature.

[0121] Furthermore, by setting preset size sets and standard widths, the above technical solution can also eliminate slight pixel value changes caused by factors such as image shooting angle and lighting conditions, thereby further improving the accuracy of recognition.

[0122] Furthermore, since defect identification is based on connected regions, which are typically small and concentrated, the technical solution can more accurately locate defect areas, providing precise guidance for subsequent repair and maintenance.

[0123] In one possible implementation, S105, the control platform determines the photovoltaic panel with the defect and the type of defect based on the visible light image, infrared image, and image capture point of the marked defect area, specifically including:

[0124] S401. Determine the coordinates of the geographic area captured in the visible light image based on the field of view of the visible light sensor and the image capture point. Determine the number of the photovoltaic panel captured in the visible light image and the pixel area corresponding to the captured photovoltaic panel based on the coordinates of the captured geographic area and the photovoltaic panel layout data.

[0125] in, Figure 5 This is a schematic diagram of the imaging of a visible light sensor provided in some embodiments of this application, such as... Figure 5 As shown, the height information of the photovoltaic panel is obtained. Based on the height information of the photovoltaic panel and the height information of the image shooting point, the height difference ΔH between the photovoltaic panel and the shooting point is obtained. Based on the height difference ΔH between the photovoltaic panel and the shooting point, the latitude and longitude coordinates of the shooting point, and the field of view (θ1 and θ2) of the visible light sensor, the coordinates of the shooting geographical area are calculated.

[0126] The photovoltaic panel layout data includes the latitude and longitude of each photovoltaic panel. The coordinates of the photographed geographic area are compared with the photovoltaic panel layout data to determine the area occupied by the photovoltaic panels within the photographed geographic area. The number of the photovoltaic panel within the photographed geographic area is used as the number of the photovoltaic panel captured in the visible light image. Based on the area occupied by the photovoltaic panel within the photographed geographic area, the corresponding pixel area of ​​the captured photovoltaic panel is determined.

[0127] Optionally, the control platform first receives the visible light image transmitted by the UAV, along with the corresponding image capture point information. Using the field of view of the visible light sensor (i.e., the angle of the image range that the sensor can capture) and the geographical location information of the image capture point (such as longitude, latitude, and altitude), the coordinate range of the geographic area captured in the visible light image is determined through geometric calculations. This step involves the application of a geographic information system (GIS), requiring the conversion of the three-dimensional coordinates of the image capture point (if including altitude information) into two-dimensional planar coordinates (latitude and longitude), and combining this with the sensor's field of view to calculate the geographic area covered by the image.

[0128] Furthermore, the control platform can compare the calculated geographical coordinates with the photovoltaic panel layout data of the photovoltaic power station. The photovoltaic panel layout data includes the specific location information (such as longitude and latitude) and number of each photovoltaic panel. Through comparison, the control platform can determine which photovoltaic panels were captured in the visible light image and calculate the corresponding pixel areas of these panels in the image. This step requires accurate photovoltaic panel layout data as support.

[0129] S402. Identify the photovoltaic panels with defects based on the corresponding pixel areas and defect areas of the photovoltaic panels in the photograph. Determine the defect type based on the pixel values ​​of the defective areas and the infrared image.

[0130] The process involves comparing the defective area with the corresponding pixel area of ​​a photographed photovoltaic panel. If the defective area overlaps with the pixel area of ​​a photographed photovoltaic panel, the photovoltaic panel is considered defective. The type of photovoltaic panel is determined by analyzing the pixel values ​​of the defective area and combining this information with the infrared image.

[0131] Optionally, the control platform compares the previously marked defect areas with the corresponding pixel areas of the photovoltaic panels being photographed. This comparison determines which photovoltaic panels have defective areas. This step requires precise pixel coordinate matching. The control platform can be programmed with an algorithm to automatically traverse the pixel coordinates of the defective areas and check if these coordinates fall within the pixel area of ​​the photovoltaic panel being photographed. If so, the photovoltaic panel is marked as defective.

[0132] Then, the control platform combines the pixel values ​​of the defect area with the infrared image to determine the defect type. Different defect types may exhibit different characteristics in visible light and infrared images, such as color, brightness, and temperature. This step requires establishing a correspondence between defect types and image features. The control platform can train a model based on machine learning or deep learning algorithms to identify different defect types. The model can receive the pixel values ​​of the defect area and the infrared image as input and output a predicted result of the defect type.

[0133] Once the defective photovoltaic (PV) panels and their types are identified, the control platform can generate a report and send relevant information (such as PV panel number, defect type, and defect location) to maintenance personnel. Maintenance personnel can then perform further inspections and repairs based on the report. Report generation and transmission can be accomplished through the control platform's user interface or an automated email system. The report should contain clear information and images to enable maintenance personnel to quickly understand and locate the problem.

[0134] In the above technical solution, because multiple photovoltaic panels are arranged in an array, it is impossible to locate them based on the shape of the photographed object and the shapes of surrounding objects. To determine the location of the photovoltaic panels in the captured image, the shooting point of the frame is simultaneously acquired when capturing the image using visible light imaging. This allows the coordinates of the photographed geographical area to be determined based on the shooting point and the field of view of visible light. By converting the pixel coordinates in the image to a geodetic coordinate system, it is only necessary to compare the coordinates of the photographed geographical area with the photovoltaic panel layout data to determine the label of the photographed photovoltaic panel in the photographed geographical area and the corresponding pixel area. Furthermore, the defective area can be compared with the corresponding pixel area of ​​the photographed photovoltaic panel to determine which specific photovoltaic panel is defective, ultimately achieving the location of the faulty photovoltaic panel.

[0135] By utilizing the field of view and image capture point information of a visible light sensor, the above-mentioned technical solution can determine the coordinates of the geographical area captured in the visible light image. This step provides an accurate geographic reference basis for subsequent photovoltaic panel location and defect identification. Then, by combining the captured geographical area coordinates and photovoltaic panel layout data, the number of the photovoltaic panel captured in the visible light image and its corresponding pixel area can be determined. Even if the photovoltaic panels are distributed in an array, making it difficult to locate them by shape and surrounding objects, this technology can effectively identify specific photovoltaic panels.

[0136] Next, by comparing the pixel area corresponding to the photovoltaic panel being photographed with the marked defect area, defective photovoltaic panels can be quickly identified. This step greatly shortens the location time of faulty photovoltaic panels and improves operation and maintenance efficiency.

[0137] Then, by combining the pixel values ​​of the defective area with infrared image information, the defect type of the photovoltaic panel can be accurately determined. Different defect types exhibit different characteristics in visible light and infrared images; by comprehensively analyzing these characteristics, accurate identification of the defect type can be achieved.

[0138] The aforementioned technical solution enables efficient identification and location of photovoltaic panel defects through automation and intelligence, significantly reducing the workload of maintenance personnel. Furthermore, because the identification process is based on precise data and algorithms, the results are highly accurate, reducing the possibility of misjudgments and omissions. Through regular inspections and timely identification and handling of photovoltaic panel defects, this technical solution helps improve the overall reliability of photovoltaic power plants. This not only reduces power generation losses due to defects but also extends the lifespan of photovoltaic panels and lowers maintenance costs.

[0139] In one possible implementation, S402, if the defective region is the initial pixel region corresponding to the same adjacent column, the photovoltaic panel with the defect is determined based on the pixel region corresponding to the photovoltaic panel in the photograph and the defective region, specifically including:

[0140] S501. For the initial pixel region containing consecutive identical columns, determine whether the pixel values ​​in each row of the initial pixel region are the same. If not, proceed to S502; otherwise, proceed to S503.

[0141] Specifically, the analysis is performed row by row, analyzing consecutive identical columns with defects. The region containing these defective columns is designated as the initial pixel region. Each row within this initial region is analyzed to determine if the pixel values ​​are identical. If not, the row may contain defect information, and is therefore retained. Otherwise, the pixel values ​​of the column are further analyzed.

[0142] Optionally, the control platform first identifies consecutive identical columns in the visible light image that are marked as defect regions. These columns appear as adjacent regions in the image with similar pixel values ​​(such as color, brightness, etc.). Using image processing algorithms, such as edge detection and region growing, these consecutive identical columns can be automatically identified, and the initial pixel regions they occupy can be determined.

[0143] For each row of the initial pixel region, the control platform checks whether the pixel values ​​within that row are the same. If they are different, the row of pixels is retained, as different pixel values ​​may indicate the presence of a defect. If they are the same, it further determines whether the row of pixels represents standard pixel values ​​(i.e., pixel values ​​that a normal photovoltaic panel should have). If it is not a standard pixel value, the row of pixels is also retained, as it may indicate a defect. If it is a standard pixel value, the row of pixels is removed, as it is unlikely to be a defect. Standard pixel values ​​can be obtained through statistical analysis of images of normal photovoltaic panels. During comparison, the difference between pixel values ​​or a similarity metric can be used to determine whether they are the same or similar.

[0144] S502, Retain the pixels in that row in the initial pixel area.

[0145] S503. Determine whether the pixel value in the row is a normal pixel value. If not, retain the pixel value in the row in the initial pixel area. If yes, delete the pixel value in the row in the initial pixel area.

[0146] Specifically, the pixel values ​​corresponding to gate lines and interconnect lines, as well as the pixel values ​​corresponding to the semiconductor layer, are obtained and used as standard pixel values. If all pixel values ​​in a row of pixels in the initial pixel region are the same and are standard pixel values, then there is no defect information in that column of pixels, and the row of pixels is deleted. If all pixel values ​​in a row of pixels in the initial pixel region are the same but are not standard pixel values, then there is defect information in that column of pixels, and the row of pixels in the initial pixel region is retained.

[0147] S504. Determine if this is the last pixel row in the initial pixel region. If yes, output the reduced defect area. If no, update the pixel rows in the initial pixel region and return to S501.

[0148] The process involves iterating through each row of pixels in the initial pixel region, determining whether defect information exists in each row, and deleting the pixels in that row if no defect information is found. Otherwise, the pixels in that row are retained, thus obtaining the reduced defect region.

[0149] S505. For each photovoltaic panel being photographed, determine whether the reduced defect area overlaps with the corresponding pixel area of ​​the photovoltaic panel being photographed. If so, the photovoltaic panel being photographed has a defect.

[0150] Specifically, for the photovoltaic panel currently being photographed, the reduced defect area is compared with the corresponding pixel area of ​​the photovoltaic panel to determine whether the reduced defect area overlaps with the corresponding pixel area of ​​the photovoltaic panel. If so, the photovoltaic panel has a defect; otherwise, it is compared with the pixel area of ​​the next photovoltaic panel to be photographed, until a photovoltaic panel with a defect is determined.

[0151] In the above technical solution, the control platform analyzes the pixel values ​​in the consecutive identical columns. Utilizing the characteristic that most pixel values ​​in the consecutive identical columns are the same, if all pixel values ​​in a certain row are the same and are not standard pixel values, then the pixel values ​​in that row are removed to further reduce the defective area. Based on the comparison between the reduced defective area and the corresponding area of ​​each photovoltaic panel in the photograph, the defective photovoltaic panel can be accurately located. In addition, defect type identification is performed within the reduced pixel area, reducing the amount of data processing during defect type identification and improving data processing efficiency.

[0152] Optionally, the control platform compares the reduced defect area with the corresponding pixel area of ​​each photographed photovoltaic panel to check for overlap. This step can be achieved by calculating the intersection of the two areas. If the intersection is not empty, it means that the reduced defect area overlaps with a part of the photographed photovoltaic panel.

[0153] If the reduced defect area overlaps with the pixel area of ​​the photovoltaic panel in the image, the control platform determines that the photovoltaic panel has a defect. This determination is based on the previous analysis and processing results. The existence of overlap means that a pixel area matching the defect area has been found on the image of the photovoltaic panel.

[0154] Once a defective photovoltaic (PV) panel is identified, the control platform can further identify the defect type, such as cracks, stains, or shading, and generate corresponding reports or alarms so that maintenance personnel can take timely action. Through this series of steps, the control platform can more accurately locate defective PV panels, reducing false alarms and missed alarms. At the same time, by narrowing down the defect area, it also improves the efficiency and accuracy of defect type identification. Furthermore, the entire process is highly automated, reducing the workload of maintenance personnel.

[0155] In one possible implementation, S402, determining the defect type based on the pixel values ​​of the defect area and the infrared image, specifically includes:

[0156] S601. Extract connected regions with the same pixel value from the reduced defect region; for each connected region, if there is a thin strip-shaped connected region, then there is a crack defect in the connected region.

[0157] The process involves analyzing the pixel values ​​of each pixel within the reduced defect area to obtain the connected regions within that area. The shape of these connected regions is then analyzed to determine if they are thin strips; if so, a crack defect is identified within the connected region.

[0158] Optionally, within the already reduced defect area, image processing techniques (such as region growing, thresholding, etc.) can be used to extract connected regions with the same pixel values. These connected regions may consist of adjacent pixels with similar pixel values, and they may represent a specific type of defect.

[0159] Connected component extraction can be achieved by iterating through each pixel in the reduced defect region, finding adjacent pixels with the same pixel value, and then grouping them into a connected component. This process can be repeated until all pixels have been classified into a connected component, or no more connected components can be extracted.

[0160] Furthermore, for each extracted connected region, its shape is determined to be thin and elongated. If so, a crack defect is considered to exist in the connected region. Crack defects typically manifest as thin, elongated connected regions with similar pixel values. Therefore, the presence of crack defects can be determined by calculating the aspect ratio of the connected region or by using morphological operations (such as erosion and dilation).

[0161] S602. If the connected region is not a thin strip, determine the pixel region in the infrared image corresponding to the connected region. If the temperature value in the pixel region is greater than or equal to a preset threshold, then it is determined that a hot spot exists in the connected region. If the temperature value in the pixel region is less than the preset threshold, then it is determined that dirt exists in the connected region.

[0162] If the connected region is not a thin strip, it is matched with the infrared image to determine the corresponding pixel region. The temperature in this pixel region is then determined to be greater than or equal to a preset threshold. If so, a hot spot is identified in the connected region; otherwise, the region is considered dirty.

[0163] Optionally, connected regions in the visible light image can be mapped onto the infrared image to find their corresponding pixel regions. This typically requires image registration techniques to ensure accurate pixel correspondence between the visible light and infrared images. Image registration techniques can utilize methods such as feature point matching and image transformation to align the visible light and infrared images, ensuring a one-to-one correspondence between their pixels. Then, based on the position of the connected region in the visible light image, the corresponding pixel region can be found in the infrared image.

[0164] For the pixel regions corresponding to the connected regions in the infrared image, their average or highest temperature values ​​are calculated and compared with a preset threshold. Based on the comparison results, the type of defect present in the connected region is determined. If the temperature value of the pixel region is greater than or equal to the preset threshold, a hot spot defect is considered to exist in the connected region. Hot spots are usually caused by localized overheating of the photovoltaic panel, possibly due to damaged cells, poor wiring, etc. If the temperature value of the pixel region is less than the preset threshold, a dirt defect is considered to exist in the connected region. Dirt can block sunlight, causing the photovoltaic panel to malfunction and thus reducing power generation. Dirt may be caused by dust, bird droppings, leaves, or other debris covering the photovoltaic panel. Then, based on the determined defect type, a corresponding report or alarm is generated so that maintenance personnel can take timely measures for repair or cleaning. Simultaneously, the defect information can also be stored in a database for subsequent analysis and optimization.

[0165] In the above technical solution, connected regions are extracted from the defect area, and the shape of the connected regions is analyzed to determine whether crack defects exist. Combined with temperature data from infrared images, the presence of hot spots is determined. After excluding the two types of defects mentioned above, contamination can be identified. This method achieves the identification of defects, hot spots, and contamination types.

[0166] It is worth noting that by extracting connected regions with the same pixel value from the reduced defect area and introducing shape analysis techniques, we can effectively identify thin, strip-shaped connected regions, which often correspond to crack defects on photovoltaic panels. This approach can improve the accuracy and efficiency of crack identification and avoid the misjudgment problem that may be caused by similar pixel values ​​in traditional methods.

[0167] Then, by combining the temperature data from the infrared image, the pixel region corresponding to the connected region can be accurately located, and by comparing it with a preset threshold, it can be quickly determined whether a hot spot exists in that region. The introduction of infrared images means that hot spot detection no longer relies on visual observation of visible light images, but is based on objective temperature data, thereby greatly improving the accuracy and reliability of the detection.

[0168] After ruling out cracks and hot spots, we accurately identified contamination in connected regions by utilizing the characteristic that the temperature values ​​of pixel areas in infrared images are below a preset threshold. This elimination method not only simplifies the inspection process but also avoids confusion caused by the simultaneous presence of multiple defects, ensuring the accuracy of contamination identification.

[0169] By comprehensively applying the above technical solutions, a complete, rapid, and accurate identification of defects, hot spots, and contamination types on photovoltaic panels can be achieved. This comprehensive detection method not only improves detection accuracy but also significantly shortens detection time, providing strong technical support for the operation and maintenance of photovoltaic power plants. Simultaneously, the automated detection process reduces the workload of maintenance personnel and improves work efficiency. Accurate and timely identification of defects, hot spots, and contamination is crucial for the long-term stable operation of photovoltaic power plants. By promptly identifying and addressing these issues, performance degradation or damage to photovoltaic panels due to defects, hot spots, or contamination can be effectively avoided, thereby extending the service life of photovoltaic power plants and improving their power generation efficiency.

[0170] In one possible implementation, after the control platform determines the photovoltaic panel with the defect and the type of defect based on the visible light image, infrared image, and image capture point of the marked defect area in step S105, the method further includes:

[0171] S106. If the number of dirty photovoltaic panels is determined to be greater than the preset number threshold, a cleaning work order for cleaning the photovoltaic panels is generated.

[0172] The cleaning work order includes the number of photovoltaic panels to be cleaned and the required cleaning time. By inspecting the photovoltaic panels in the photovoltaic power station, the number of dirty photovoltaic panels is determined based on the visible light and infrared images obtained during the inspection.

[0173] A preset threshold can be set to half the total number of photovoltaic panels in the photovoltaic power station. If the number of dirty photovoltaic panels exceeds the preset threshold, it indicates that a large area of ​​photovoltaic panels is dirty, affecting the power generation of the entire photovoltaic power station. Therefore, once it is determined that the number of dirty photovoltaic panels exceeds the preset threshold, a cleaning work order for cleaning photovoltaic panels is generated, triggering the cleaning robot to clean the photovoltaic panels.

[0174] S107. Determine the cleaning robot to perform the cleaning operation based on the cleaning work order, the busy / idle status of the cleaning robot, and the cleaning capacity of the cleaning robot.

[0175] Specifically, the cleaning robot is selected based on its busy / idle status and the required cleaning time. The cleaning robot that can perform the cleaning operation within the required cleaning time is determined based on the cleaning capacity of the cleaning robot that can perform the cleaning operation within the required cleaning time and the number of photovoltaic panels to be cleaned, so as to ensure that the cleaning robot that performs the cleaning operation can complete the cleaning operation within the required cleaning time.

[0176] S108. The control platform sends cleaning instructions to the cleaning robot.

[0177] S109. The cleaning robot cleans the photovoltaic panels within the required cleaning time.

[0178] In the above technical solution, the control platform analyzes visible light and infrared images to determine if the number of dirty photovoltaic panels exceeds a preset threshold. Then, it automatically generates a cleaning work order for cleaning the photovoltaic panels. Based on the cleaning work order and the cleaning capabilities of the cleaning robots, the platform schedules the cleaning robots, generates cleaning instructions, and controls the cleaning robots to perform cleaning. When dirt affects the power generation capacity of the photovoltaic power station, timely cleaning restores the power generation capacity of the photovoltaic power station and improves the operating efficiency of the photovoltaic power station.

[0179] It's worth noting that the control platform can accurately identify the level of dirt on the surface of photovoltaic panels by analyzing visible light and infrared images. This not only avoids the subjectivity and inaccuracy of manual inspections but also significantly improves detection efficiency, making the operation and maintenance of photovoltaic power plants more timely and effective.

[0180] When the control platform determines that the number of dirty photovoltaic panels exceeds a preset threshold, it automatically generates a cleaning work order for the photovoltaic panels. This step simplifies the operation and maintenance process, reduces human intervention, and ensures the timeliness and accuracy of the cleaning work. At the same time, the cleaning work order clearly lists the number of photovoltaic panels to be cleaned, providing clear task guidance for subsequent cleaning operations.

[0181] Furthermore, the control platform intelligently selects cleaning robots to be used based on the cleaning work order, the busy / idle status of the cleaning robots, and their cleaning capacity, and issues cleaning instructions to them. This intelligent scheduling mechanism ensures the rational allocation and efficient use of cleaning resources, avoids the idleness or overuse of cleaning robots, and thus improves cleaning efficiency.

[0182] Furthermore, by promptly identifying dirt and generating cleaning work orders, coupled with the rapid response and efficient cleaning of cleaning robots, this technical solution can quickly restore the photovoltaic panels to a clean state before dirt affects the power generation capacity of the photovoltaic power station. This not only avoids power generation losses caused by dirt but also improves the overall operating efficiency of the photovoltaic power station.

[0183] Some embodiments of this application also provide a machine vision-based outdoor inspection method for photovoltaic power stations, which includes the following steps:

[0184] S701, the drone conducts a survey of the photovoltaic power station and sends the survey results to the control platform.

[0185] In this process, drones are operated by pilots to survey photovoltaic power plants and send the survey results to the control platform in real time.

[0186] S702. The control platform draws an electronic map of the photovoltaic power station based on the survey results and generates an inspection route based on the electronic map of the photovoltaic power station.

[0187] The control platform uses existing mapping algorithms to process the results of on-site surveys conducted by drones, creating an electronic map of the photovoltaic power station. From this map, it obtains photovoltaic panel layout data and generates inspection paths to ensure each panel is inspected while minimizing the drone's flight path. The control platform also sets parameters such as drone flight altitude and speed, and allows the drone to operate in terrain-mimicking mode. Compared to fixed-altitude flight, terrain-mimicking flight ensures clear image capture and prevents the drone from getting too close to the photovoltaic panels and damaging the machine.

[0188] The S703 drone, based on an electronic map, follows the inspection route and simulates the terrain. It uses a visible light sensor to collect visible light images of the photovoltaic panels and an infrared sensor to collect infrared light images of the photovoltaic panels.

[0189] S704 The UAV performs geometric transformation processing on the visible light image to make the frame of the photovoltaic panel parallel to the edge of the image, thus obtaining the geometrically transformed visible light image.

[0190] Geometric transformations include linear transformations and translations. Linear transformations can change the size, shape, and orientation of an object, while translations can change its position. By combining linear transformations and translations, the edges of the object can be aligned with the frame of the photovoltaic panel and the edges of the image.

[0191] Optionally, the drone first acquires a visible light image of the photovoltaic panel using a visible light sensor. Then, it processes the original visible light image using geometric transformation methods in image processing, such as rotation, translation, and scaling, so that the frame of the photovoltaic panel is parallel to the edge of the image.

[0192] The specific parameters of the geometric transformation can be determined by analyzing information such as the tilt angle and position of the photovoltaic panel's frame in the image. The transformation parameters can be set automatically or manually to ensure that the photovoltaic panel's frame is parallel to the image edges in the transformed image. The transformed image should retain all the information from the original image, only changing the spatial positions of the pixels.

[0193] The S705 UAV performs denoising processing on the geometrically transformed visible light image to obtain a denoised visible light image.

[0194] Optionally, the visible light image after geometric transformation can be denoised to remove noise interference and improve image quality. Various denoising algorithms can be used, such as mean filtering, median filtering, Gaussian filtering, and wavelet denoising, with the appropriate method selected based on the specific image and noise type. The denoising process should preserve as much detail as possible while removing noise to improve the accuracy of subsequent defect identification.

[0195] S706. The UAV extracts the distribution of pixels with the same pixel value in the visible light image, and determines whether there are defective areas in the photovoltaic panel based on the distribution of pixels with the same pixel value in the visible light image.

[0196] Optionally, the UAV extracts pixels with the same pixel value from the processed visible light image and analyzes the distribution of these pixels. This extraction can be achieved using methods such as image segmentation and clustering. The distribution analysis can include features such as the number, location, and shape of the pixels, which can then be used to determine if there are defective areas in the photovoltaic panel.

[0197] Based on the distribution of pixels with the same extracted pixel value, the distribution of pixel values ​​in each pixel column is compared to analyze whether there are any anomalies or inconsistencies, thereby determining whether there are defective areas in the photovoltaic panel. Certain thresholds or standards can be set; when the distribution of pixels exceeds these thresholds or standards, a defective area is considered to exist. The determination of defective areas can be further combined with the actual structure and functional characteristics of the photovoltaic panel, as well as common defect types, for a comprehensive judgment.

[0198] S707. When it is determined that there is no defect, the drone deletes the visible light image, infrared light image, and the image capture point.

[0199] S708. When a defect is identified, the drone will send visible light images, infrared images, and the image capture points of the marked defect area to the backend server.

[0200] S709. The backend server determines the photovoltaic panel with defects and the type of defects based on the visible light image, infrared light image, and image capture point of the marked defect area.

[0201] In the above technical solution, the drone surveys the photovoltaic power station, and the control platform processes the survey results to obtain an electronic map of the photovoltaic power station. The control platform then specifies the inspection path based on the electronic map, enabling the drone to fly along the inspection path according to the electronic map. No pilot is required to operate the drone, ensuring the reliability of the drone flight and the quality of image acquisition.

[0202] It's worth noting that the drone's initial comprehensive survey of the photovoltaic power station is crucial, as it provides detailed on-site information for subsequent image acquisition tasks. Through this survey, the drone captures key data such as the layout of the photovoltaic power station, the location of the photovoltaic panels, and potential obstacles, laying the foundation for subsequent mission planning. After the drone transmits the survey results to the control platform, the platform uses this data to create a detailed electronic map of the photovoltaic power station. This map not only accurately reflects the actual layout of the photovoltaic power station but also provides precise reference for the drone's flight path planning. The generation of the electronic map enables the drone to achieve accurate positioning even in the absence of GPS signals or with unstable signals, relying solely on map information.

[0203] Furthermore, the control platform intelligently generates the optimal inspection path based on the electronic map of the photovoltaic power station. This path fully considers the distribution of photovoltaic panels, the location of obstacles, and the flight capabilities of the drones, ensuring that the drones can complete the inspection task with the shortest path and the highest efficiency. At the same time, the generation of the inspection path also takes into account the drones' endurance and flight safety, avoiding flight accidents or mission failures caused by unreasonable paths.

[0204] Furthermore, the drone achieves autonomous flight based on the inspection path generated by the control platform. During this process, the drone does not require real-time operator intervention, significantly reducing the operator's workload and improving flight reliability. Simultaneously, because the flight path is pre-planned, the drone can fly with greater stability and smoothness, thus ensuring the quality of image acquisition. Both visible light and infrared images can be acquired with higher clarity and accuracy.

[0205] Furthermore, after the visible light sensor acquires a visible light image, the visible light image is geometrically transformed so that the frame of the photovoltaic panel in the visible light image is parallel to the image edge. This ensures that the pixels corresponding to the gate line or connecting line are located in a column or row. By comparing the pixel value distribution of each column of pixels, the defect area can be determined based on the pixel value distribution of each column of pixels, thereby improving the accuracy of defect identification.

[0206] It's worth noting that by performing geometric transformations on the visible light image, it's possible to ensure that the frame of the photovoltaic panel is parallel to the image edges. This step is crucial because it makes the gate lines or connecting lines on the photovoltaic panel appear as straight lines in the image, with corresponding pixels located in a column or row. This alignment not only simplifies subsequent image processing steps but also provides a basis for accurate pixel value comparison.

[0207] Even after geometric transformation, the visible light image may still contain noise, which can interfere with defect identification. Therefore, we performed denoising processing on the image, effectively removing random noise and interference while preserving the true information of the photovoltaic panel surface. The denoised image is clearer, providing high-quality input for subsequent pixel distribution extraction. Based on the geometric transformation and denoising processing, the UAV extracted the pixel distribution of the same pixel value in the denoised visible light image. By comparing the pixel value distribution of each pixel column, we can accurately identify defect areas on the photovoltaic panel. This pixel value distribution-based identification method is not only highly sensitive but also able to distinguish different types of defects, such as cracks, stains, and occlusions.

[0208] Furthermore, the implementation of the above technical solutions can significantly improve the accuracy and efficiency of defect identification. Geometric transformation processing ensures image alignment, denoising enhances image quality, and pixel distribution extraction enables accurate identification of defect areas. The synergistic effect of these steps allows us to quickly and accurately locate defect areas on photovoltaic panels, providing strong technical support for the operation and maintenance of photovoltaic power plants.

[0209] The control platform provided in this embodiment includes at least one processor and a memory. Optionally, the device also includes a communication component. The processor, memory, and communication component are connected via a bus.

[0210] In a specific implementation, at least one processor executes computer execution instructions stored in memory, causing at least one processor to perform the above-described method.

[0211] The specific implementation process of the processor can be found in the above method embodiments, and its implementation principle and technical effect are similar, so it will not be repeated here.

[0212] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0213] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0214] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A machine vision-based outdoor inspection method for photovoltaic power stations, characterized in that, The method is applied to a drone and a control platform, wherein the drone and the control platform are connected in communication, and the method includes: The drone flies along the inspection path, using a visible light sensor to collect visible light images of the photovoltaic panels and an infrared sensor to collect infrared images of the photovoltaic panels. The drone extracts the distribution of pixels with the same pixel value in the visible light image, and determines whether there are defective areas in the photovoltaic panel based on the distribution of pixels with the same pixel value in the visible light image; When a defect is determined to exist, the UAV sends a visible light image of the defective area, the infrared image, and the image capture point to the control platform. The control platform determines the photovoltaic panel with defects and the type of defects based on the visible light image of the marked defect area, the infrared image, and the image capture point. The drone extracts the distribution of pixels with the same pixel value in the visible light image, and determines whether there are defective areas in the photovoltaic panel based on the distribution of pixels with the same pixel value in the visible light image, specifically including: Extract the pixels from each column of the visible light image, compare the pixels in each column, and obtain the width, start column, and end column of consecutive identical columns; wherein, the consecutive identical columns refer to adjacent pixel columns with the same pixel value distribution; For each consecutive identical column, if it is determined that the width of the consecutive identical column is within a preset width set, then it is determined that there is no defect in the consecutive identical column. The preset width set includes the pixel width difference between two adjacent gate lines, the pixel width difference between the gate line and the edge of the photovoltaic module, the pixel width difference between two adjacent connecting lines, and the pixel width difference between the connecting line and the edge of the photovoltaic module. For each consecutive identical column, if it is determined that the width of the consecutive identical column is not within a preset width set, it is further determined whether the consecutive identical column contains pixels from the first column or the last column. If so, it is determined that there is no defect in the consecutive identical column; otherwise, the area corresponding to the consecutive identical column is considered a defect area; or... The drone extracts the distribution of pixels with the same pixel value in the visible light image, and determines whether there are defective areas in the photovoltaic panel based on the distribution of pixels with the same pixel value in the visible light image, specifically including: Extract connected regions with the same pixel value from the visible light image. If the connected region is a rectangular region, determine whether the size of the rectangular region is within a preset size set. If yes, there is no defect in the connected region. If no, the connected region is regarded as a defect region. The preset size set includes the size range that the gate line or connecting line should have. If the connected region is a grid-shaped region, determine whether the lengths of multiple horizontal regions in the grid-shaped region are the same, and whether the lengths of multiple vertical regions in the grid-shaped region are the same. If they are all the same, then the connected region has no defects. If the horizontal regions or vertical regions are not the same, then the connected region is regarded as a defective region.

2. The method according to claim 1, characterized in that, The control platform determines the photovoltaic panel with defects and the type of defects based on the visible light image of the marked defect area, the infrared image, and the image capture point, specifically including: The coordinates of the geographic region captured in the visible light image are determined based on the field of view of the visible light sensor and the image capture point. The number of the photovoltaic panel captured in the visible light image and the corresponding pixel area of ​​the photovoltaic panel are determined based on the coordinates of the captured geographic region and the photovoltaic panel layout data. The photovoltaic panel with defects is identified based on the pixel area and defect area corresponding to the photovoltaic panel in the photograph; the defect type is determined based on the pixel value of the defect area and the infrared image.

3. The method according to claim 1, characterized in that, Before the UAV extracts the distribution of pixels with the same pixel value in the visible light image, and determines whether there are defective areas in the photovoltaic panel based on the distribution of pixels with the same pixel value in the visible light image, the method further includes: The visible light image is subjected to geometric transformation processing to make the frame of the photovoltaic panel parallel to the edge of the image, thus obtaining a geometrically transformed visible light image; The visible light image after the geometric transformation is denoised to obtain a denoised visible light image. Accordingly, the UAV extracts the distribution of pixels with the same pixel value in the visible light image, specifically including: The drone extracts the distribution of pixels with the same pixel value in the denoised visible light image.

4. The method according to claim 1, characterized in that, After the control platform determines the photovoltaic panel with the defect and the type of defect based on the visible light image of the marked defect area, the infrared image, and the image capture point, the method further includes: If the number of dirty photovoltaic panels is determined to be greater than a preset threshold, a cleaning work order for cleaning the photovoltaic panels is generated; the cleaning work order includes the number of photovoltaic panels to be cleaned. The cleaning robot to be cleaned is determined based on the cleaning work order, the busy / idle status of the cleaning robot, and the cleaning capacity of the cleaning robot. A cleaning instruction is then issued to the cleaning robot to be cleaned, so that the cleaning robot to be cleaned can clean the photovoltaic panel.

5. The method according to claim 1, characterized in that, Before the drone flies along the inspection path and uses the visible light sensor to collect visible light images of the photovoltaic panel, and uses the infrared sensor to collect infrared images of the photovoltaic panel, the method further includes: The drone surveys the photovoltaic power station and sends the survey results to the control platform. The control platform generates an electronic map of the photovoltaic power station based on the survey results; and generates an inspection path based on the electronic map of the photovoltaic power station, enabling the UAV to perform terrain-visiting flight based on the electronic map and the inspection path.

6. A machine vision-based outdoor inspection device for photovoltaic power plants, comprising a drone and a control platform, wherein the drone and the control platform are communicatively connected, the drone is equipped with a visible light sensor and an infrared sensor, and the drone and the control platform are used to perform the method as described in any one of claims 1 to 5.

7. A control platform, characterized in that, include: processor; as well as, Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 5 by executing the executable instructions.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 5.

Images