Photovoltaic station outdoor inspection method and device based on machine vision

Abstract

The embodiment of the invention provides a photovoltaic station outdoor inspection method and device based on machine vision. The unmanned aerial vehicle flies according to the inspection path, a visible light sensor is used for collecting a visible light image of the photovoltaic panel, and an infrared sensor is used for collecting an infrared image of the photovoltaic panel. And the unmanned aerial vehicle extracts the distribution condition of the pixel points with the same pixel value in the visible light image, and determines whether the photovoltaic panel has a defect area according to the distribution condition of the pixel points with the same pixel value in the visible light image. And the unmanned aerial vehicle sends the visible light image, the infrared image and the image shooting point of the marked defect area to the control platform when determining that the defect exists. And the control platform determines the photovoltaic panel with the defect and the defect type according to the visible light image, the infrared image and the image shooting point of the marked defect area. The method does not need to use a convolutional neural network for recognition, can improve the data processing efficiency, reduces the use of hardware resources, and reduces the equipment cost.

Description

Technical Field

[0001] The present application relates to the technical field of machine vision, and in particular, to a method and device for outdoor inspection of a photovoltaic power station based on machine vision. Background Art

[0002] With the rapid development of the photovoltaic industry, the scale of photovoltaic power stations has been continuously expanding, and the difficulty of inspection work has gradually increased.

[0003] In the early stage, the maintenance and inspection of photovoltaic panels were carried out manually, and each panel was detected one by one by inspectors holding detection equipment. With the development of unmanned aerial vehicle (UAV) technology, the application of UAVs in the operation and maintenance detection of photovoltaic power stations has gradually matured. The traditional inspection of photovoltaic power stations by UAVs mainly includes: after the pilot arrives at the photovoltaic power station, the pilot surveys the photovoltaic power station and formulates a flight plan based on experience and / or survey data. After installing the UAV, the pilot controls the UAV to fly according to the flight plan. After the flight is completed, the collected images are downloaded from the UAV, and an image recognition model is used to analyze the images.

[0004] However, when using an image recognition model for image recognition, since multiple convolutions are required for feature extraction, the required calculation time is relatively long, which affects the efficiency of the image recognition model. In addition, the hardware resources of the device are also required to be relatively high when running the image recognition model, thus increasing the device cost. Summary of the Invention

[0005] Embodiments of the present application provide a method and device for outdoor inspection of a photovoltaic power station based on machine vision, so as to achieve the effects of improving the image processing efficiency and reducing the consumption of hardware resources.

[0006] Some embodiments of the present application provide a method for outdoor inspection of a photovoltaic power station based on machine vision. The method is applied to a UAV and a control platform, and the UAV and the control platform are communicatively connected. The method includes:

[0007] The UAV flies along an inspection path, uses a visible light sensor to collect visible light images of photovoltaic panels, and uses an infrared sensor to collect infrared images of photovoltaic panels;

[0008] The UAV extracts the distribution of pixel points with the same pixel value in the visible light image, and determines whether there is a defective area on the photovoltaic panel according to the distribution of pixel points with the same pixel value in the visible light image;

[0009] When the UAV determines that there is a defect, it sends the visible light image, infrared image, and image shooting point marking the defective area to the control platform;

[0010] The control platform determines the photovoltaic panel with the defect and the defect type according to the visible light image, infrared image, and image shooting point marking the defective area.

[0011] Some embodiments of the present application provide a control platform, including: a memory, a processor;

[0012] The memory stores computer-executable instructions;

[0013] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above various possible implementation manners.

[0014] An embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the above various possible implementation manners.

[0015] An embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the above various possible implementation manners. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings here are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0017] Figure 1 It is a schematic layout diagram of a photovoltaic panel;

[0018] Figure 2 It is a schematic flow diagram of a method for outdoor inspection of a photovoltaic power station based on machine vision provided by some embodiments of the present application;

[0019] Figure 3 It is a schematic appearance diagram of a photovoltaic panel provided by some embodiments of the present application;

[0020] Figure 4 It is a partial schematic diagram of a visible light image of a photovoltaic panel provided by some embodiments of the present application;

[0021] Figure 5 It is an imaging schematic diagram of a visible light sensor provided by some embodiments of the present application.

[0022] REFERENCE NUMERALS:

[0023] 10. Photovoltaic panel; 100. Photovoltaic module; 101. Gate line; 102. Connection line; X. Horizontal; Y. Vertical.

[0024] Through the above drawings, the clear embodiments of the present application have been shown, and there will be more detailed descriptions later. These drawings and text descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0026] Figure 1 A schematic diagram of the layout of a photovoltaic panel is shown in FIG. Figure 1 As shown, a large number of photovoltaic panels are arranged in arrays, usually in forests, water surfaces or residential roofs. Photovoltaic panels collect solar energy and convert it into electrical energy. In order to use photovoltaic panels to generate electricity, energy storage devices or inverters are also set up in photovoltaic stations. Part of the electricity generated by photovoltaic panels is stored in energy storage devices, or is converted into electricity after being stepped up by inverters.

[0027] After the construction of photovoltaic stations, the operation and maintenance of photovoltaic stations has received more attention. Photovoltaic panels are a large number of components in photovoltaic stations, and maintenance and inspection of photovoltaic panels is also a heavy task.

[0028] In the early days, the maintenance and inspection of photovoltaic panels was carried out manually, with inspectors using handheld inspection equipment to inspect each panel one by one. With the development of drone technology, the application of drones in the operation and maintenance of photovoltaic stations has gradually matured. The traditional drone inspection of photovoltaic stations mainly includes: after arriving at the photovoltaic station, the pilot surveys the photovoltaic station and formulates a flight plan based on experience and / or survey data. After installing the drone, the pilot controls the drone to fly according to the flight plan. After the flight, the collected images are downloaded from the drone and analyzed using an image recognition model.

[0029] However, when using image recognition models for image recognition, multiple convolutions are required for feature extraction, which takes a long time to calculate and affects the efficiency of the image recognition model. In addition, the hardware resource requirements of the device are also relatively high when running the image recognition model, which increases the cost of the device.

[0030] In view of this, the present application provides an outdoor inspection method for photovoltaic stations based on machine vision, aiming to propose a new image recognition algorithm. Since there is no need to use an image recognition model for image processing, the hardware configuration can be reduced, the equipment cost can be reduced, and the processing efficiency of the recognition algorithm can be improved.

[0031] The following uses specific embodiments to elaborate in detail on the technical solutions of this application and how the technical solutions of this application solve the above technical problems. The following several specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below in conjunction with the accompanying drawings.

[0032] Some embodiments of this application provide an outdoor inspection device for a photovoltaic power station based on machine vision. The device includes a control platform and a drone.

[0033] Control the drone to conduct aerial surveys of the entire photovoltaic area to generate an electronic map of the photovoltaic power station. The drone obtains panoramic image data of the photovoltaic power station by carrying a high-definition aerial survey camera, and makes an electronic map of the photovoltaic power station through mosaic software. By dividing the areas of different photovoltaic panels on the made electronic map and numbering the photovoltaic panels in the area, the coordinate information of the photovoltaic panels in this area is obtained, and a photovoltaic panel database is constructed.

[0034] The control platform determines the inspection time based on the weather and / or the output power of the photovoltaic power station, and generates an inspection path based on the electronic map. The control platform issues a flight instruction, and the flight instruction includes the inspection time and the inspection instruction. After receiving the flight instruction, the drone conducts autonomous cruising according to the inspection path and conducts terrain-following flight according to the flight parameters set by the control platform. An infrared sensor and a visible light sensor are carried on the drone. During the flight, visible light images and infrared images of the photovoltaic panels are collected. Figure 2 It is a schematic flowchart of a method for outdoor inspection of a photovoltaic power station based on machine vision provided by some embodiments of this application. As Figure 2 shown, this method specifically includes:

[0035] S101. The drone flies according to the inspection path, uses the visible light sensor to collect visible light images of the photovoltaic panels, and uses the infrared sensor to collect infrared images of the photovoltaic panels.

[0036] Among them, through on-site survey of the photovoltaic power station, the area to be inspected and the laying layout of the photovoltaic panels in the area to be inspected are obtained. The inspection path is formulated based on the laying layout of the photovoltaic panels in the area to be inspected. The drone flies according to the inspection path to realize the inspection of the photovoltaic panels in the area to be inspected.

[0037] The drone is equipped with a visible light sensor and an infrared sensor. The visible light sensor is used to detect dirt, cracks, etc. on the photovoltaic panels. The infrared sensor is used to detect hot spots on the photovoltaic panels.

[0038] When the drone is flying in the area to be inspected, turn on the visible light sensor and the infrared sensor. Use the visible light sensor to collect the visible light image of the photovoltaic panel, and use the infrared sensor to collect the infrared image of the photovoltaic panel. At the same image capture point, the visible light sensor captures one frame of visible light image, and the infrared sensor captures one frame of infrared image.

[0039] In short, after receiving the flight instruction sent by the control platform, the drone flies according to the preset inspection path. During the flight, the visible light sensor and the infrared sensor on the drone are started simultaneously to capture images of the photovoltaic panel respectively. The visible light sensor is used to capture the surface condition of the photovoltaic panel and generate a visible light image; the infrared sensor is used to detect the thermal radiation condition of the photovoltaic panel and generate an infrared image. These two types of image data will be used for subsequent defect detection and analysis.

[0040] S102. The drone extracts the distribution of pixel points with the same pixel value in the visible light image, and determines whether there is a defective area on the photovoltaic panel according to the distribution of pixel points with the same pixel value in the visible light image.

[0041] Optionally, the image processing module built in the drone processes the collected visible light image. First, by comparing pixel values column by column or block by block, the areas with the same and continuous pixel values in the image are extracted. These areas usually correspond to the gate lines, connection lines on the photovoltaic panel or abnormal areas formed due to defects (such as cracks, dirt). Then, compare the size and shape of these areas with the preset standard values to determine whether there is an abnormality. For example, if the width or shape of a continuous area with the same pixel value does not meet the expectation, it may be regarded as a defective area.

[0042] As shown in FIG. 3 is a schematic external view of the photovoltaic panel 10 provided by some embodiments of the present application. Figure 3 As shown, the structure of the photovoltaic panel is usually composed of a plurality of photovoltaic modules 100 arranged in an array. A transparent glass cover plate and a frame (not shown in the figure) are also laid on the photovoltaic module to play a protective role. The photovoltaic module 100 includes a semiconductor base layer, a plurality of gate lines 101 located on the surface of the semiconductor base layer, and a plurality of connection lines 102 located above the gate lines 101. The plurality of gate lines 101 extend along the longitudinal direction Y, and the plurality of connection lines 102 extend along the transverse direction X. The plurality of gate lines 101 and the plurality of connection lines 102 together divide the semiconductor base layer into a plurality of pixel regions with the same color. The plurality of gate lines 101 and the plurality of connection lines 102 are both metal conductors and also form pixel regions with the same color.

[0043] Figure 4 FIG. 18 is a partial schematic view of the visible light image of the photovoltaic panel 10 provided by some embodiments of the present application. Figure 4As shown, the pixel value of the slant-bar filled pixel points is 156, and the pixel value of the blank-filled pixel points is 230. The slant-bar filled pixel points divide the blank-filled pixel points into multiple rectangular areas. If there is dirt, cracks or hot spots on the surface of the photovoltaic module 100, the area where the blank-filled pixel points are located will be in an irregular shape or a rectangle with a size that does not conform to the specified value. By counting the distribution of the blank-filled pixel points, it is determined whether there is a defective area on the photovoltaic panel 10.

[0044] S103. When it is determined that there are no defects, the drone deletes the visible light image, the infrared image, and the image shooting point.

[0045] Among them, the drone identifies the visible light image. If it is determined that there are no defects in the visible light image, there is no need to send the visible light image, the infrared image taken at the same shooting point, and the image shooting point to the control platform. This can reduce the data transmission volume between the drone and the control platform and improve the inspection operation efficiency.

[0046] S104. When it is determined that there are defects, the drone sends the visible light image, the infrared image, and the image shooting point that mark the defective area to the control platform.

[0047] If it is determined that there are defects in the visible light image, the visible light image, the infrared image, and the image shooting point that mark the defective area are sent to the control platform. The visible light image and the infrared image that mark the defective area are images taken at the same image shooting point. After the control platform combines the infrared image and the visible light image that marks the defective area, it conducts defect determination.

[0048] Optionally, once the defective area is found, the drone automatically marks these areas and packages and sends the marked visible light image, the corresponding infrared image, and the location information (image shooting point) at the time of shooting to the control platform. This process ensures that the control platform can accurately receive all the necessary data related to the defects for further analysis and processing.

[0049] S105. The control platform determines the photovoltaic panel with defects and the type of defects based on the visible light image, the infrared image, and the image shooting point that mark the defective area.

[0050] Among them, after receiving the above data, the control platform converts the visible light image to the geodetic coordinate system based on the image shooting point and the parameters of the visible light sensor to determine the shooting geographical area. Then it compares the photovoltaic panel layout data with the shooting geographical area to further determine the photovoltaic panel with defects. And it determines whether the defect in the photovoltaic panel is a crack, dirt, or hot spot based on the pixel values in the visible light image that marks the defective area and the temperature data in the infrared image.

[0051] Optionally, after the control platform receives the data sent by the drone, it first calculates the actual geographical location corresponding to each pixel point in the image based on the image shooting point and the field of view angle of the visible light sensor. Then, in combination with the layout data of the photovoltaic field, it matches these geographical locations with the positions of the photovoltaic panels to determine which photovoltaic panel the defect appears on. Next, the control platform will further analyze the type of the defect by combining the defect area features in the visible light image and the temperature data in the infrared image. For example, different types of defects such as cracks, dirt, and hot spots can be distinguished through the shape features in the visible light image and the temperature anomalies in the infrared image.

[0052] In the above technical solution, the gate lines and connection lines are regularly arranged above the semiconductor substrate layer. In this way, the pixel values in the captured image are regularly distributed, and it is possible to determine whether there is a defective area on the photovoltaic panel through the distribution of pixel points with the same pixel value in the visible light image, without using a convolutional neural network for recognition. In this way, the data processing efficiency can be improved, the use of hardware resources can be reduced, and the equipment cost can be lowered. Due to the improvement of the data processing efficiency, there is no need to set a large storage space on the drone to store the unprocessed visible light images, and there is no need to rely on the control platform to store the unprocessed visible light images. Only the images with defects can be transmitted to the control platform, reducing the data transmission volume.

[0053] It is worth noting that traditional methods often rely on convolutional neural networks for image recognition, which requires a large amount of computing resources and time. However, the above technical solution determines the defect by directly analyzing the distribution of pixel points with the same pixel value in the visible light image, avoiding complex convolutional operations and significantly improving the data processing efficiency. Due to the improvement of the data processing speed, the drone can analyze the image immediately during flight, quickly identify and mark the defect, reducing the inspection cycle and improving the real-time performance of the inspection.

[0054] Moreover, it does not rely on high-performance computing hardware for image processing, reducing the hardware configuration requirements of the drone and the control platform, thereby reducing the hardware cost. Since there is no need to store a large amount of unprocessed image data on the drone and only the images with defects are transmitted, a large amount of storage space is saved, and the utilization rate of storage resources is improved.

[0055] In addition, by combining the visible light image and the infrared image, the defect type (such as cracks, dirt, hot spots, etc.) and location can be identified more accurately. The visible light image is used for preliminary screening of the defective area, and the infrared image provides temperature information to further confirm the defect type. By analyzing the distribution of pixel points in the visible light image, the normal photovoltaic panel structure and the defective area can be effectively distinguished, reducing false alarms; at the same time, the auxiliary verification of the infrared image also reduces the risk of missed alarms.

[0056] Furthermore, the drone autonomously flies along a preset inspection path and collects images without manual intervention, realizing the automation of the inspection process. After receiving the defect data, the control platform can automatically generate cleaning work orders or repair instructions, and intelligently dispatch cleaning robots or maintenance personnel for processing, improving the overall operation and maintenance efficiency. By promptly detecting and handling photovoltaic panel defects, it is possible to avoid a decrease in power generation or equipment damage caused by the expansion of defects. In addition, through regular inspections of the photovoltaic panels, intervention can be carried out at the initial stage of defect occurrence, extending the service life of the photovoltaic panels and reducing maintenance costs.

[0057] In a possible implementation manner, in S102, the drone extracts the distribution of pixel points with the same pixel value in the visible light image, and determines whether there is a defective area on the photovoltaic panel according to the distribution of pixel points with the same pixel value in the visible light image. Specifically, it includes:

[0058] S201, extract the pixel points of each column in the visible light image, compare the pixel points of each column, and obtain the width, starting pixel column, and ending pixel column of consecutive identical columns.

[0059] Among them, consecutive identical columns refer to pixel columns with the same pixel value distribution and adjacent to each other.

[0060] Optionally, the drone first extracts the pixel points of each column in the visible light image and compares these pixel points. Specifically, the drone divides the image into multiple columns and compares the values of adjacent pixel points column by column. When it is found that the values of a column of pixel points are all the same and are also the same as the previous column, these columns are regarded as a "consecutive identical column". This step aims to find all combinations of pixel columns with continuous and identical pixel values in the image.

[0061] For each consecutive identical column, the drone calculates its width (i.e., the number of pixel columns it contains), starting pixel column, and ending pixel column. Among them, the width refers to the number of pixel columns contained in the consecutive identical column; the starting pixel column and ending pixel column respectively identify the starting and ending positions of the consecutive identical column in the image.

[0062] Take Figure 4 the partial pixel area of the visible light image shown as an example. The partial pixel area is a 17-row × 11-column pixel array.

[0063] The pixels in the first column and the eighth column are both pixels filled with diagonal lines, representing two gate lines extending longitudinally. The pixel distributions of the pixels in the second column to the seventh column and the pixel distributions of the pixels in the ninth column to the eleventh column are the same. In both cases, the second row pixel and the sixteenth row pixel are filled with diagonal lines. The pixels in the second column to the seventh column form a continuous same column, and the width of the continuous same column is 6 pixel widths. The starting column of the continuous same column is the second column pixel, and the ending pixel column of the continuous same column is the seventh column pixel. The pixels in the ninth column to the eleventh column form a continuous same column, and the width of the continuous same column is 3 pixel widths. The starting column of the continuous same column is the ninth column pixel, and the ending pixel column of the continuous same column is the eleventh column pixel.

[0064] S202. For each continuous same column, determine whether the width of the continuous same column is within a preset width set. If so, go to S203; otherwise, go to S204.

[0065] Among them, the preset width set includes multiple standard widths. The standard width refers to the pixel width difference between two adjacent gate lines, the pixel width difference between the gate line and the edge of the photovoltaic module, the pixel width difference between two adjacent connection lines, and the pixel width difference between the connection line and the edge of the photovoltaic module.

[0066] Optionally, the drone compares the width of each continuous same column with the preset width set. Among them, the preset width set contains standard width values calculated based on the photovoltaic panel design and the image scaling ratio. If the width of the continuous same column is within the preset width set, it is considered that this column is standard and there is no defect; if it is not within the preset width set, go to the next judgment.

[0067] Obtain the distance between two adjacent gate lines on the photovoltaic module, the distance between the edge of the photovoltaic module and the gate line, the distance between two adjacent connection lines, and the distance between the connection line and the edge of the photovoltaic module. Determine the image scaling ratio according to the internal parameters and external parameters of the visible light image sensor. Calculate the standard width based on the image scaling ratio and the distance between two adjacent gate lines, and calculate the standard width based on the image scaling ratio and the distance between the edge of the photovoltaic module and the gate line. Calculate the standard width based on the image scaling ratio and the distance between two adjacent connection lines, and calculate the standard width based on the image scaling ratio and the distance between the edge of the photovoltaic module and the gate line.

[0068] If there are no cracks, dirt, or hot spots on the photovoltaic module, consecutive identical columns are separated by grid lines or connection lines, so the width of consecutive identical columns lies within a preset width set. If there are cracks, dirt, or hot spots on the photovoltaic module, consecutive identical columns are divided not only by grid lines or connection lines but also by defects such as cracks, dirt, or hot spots, so the width of consecutive identical columns is less than the standard width in the preset width set.

[0069] S203. If it is determined that the width of consecutive identical columns lies within a preset width set, it is determined that there are no defects in the consecutive identical columns.

[0070] Among them, for each consecutive identical column, compare the width of the consecutive identical column with any standard width in the preset width set to determine whether the width of the consecutive identical column lies within the preset width set. If so, it can be determined that there are no defects in the consecutive identical column.

[0071] For example: Figure 4 Take the local pixel area of the visible light image shown as an example. The consecutive identical columns formed by the pixel points from the 2nd column pixel to the 7th column pixel, and the width of the consecutive identical columns is 6 pixel widths. The pixel width between two grid lines is 6 pixel values, the pixel width between two connection lines is 4 pixel values, and the difference between the connection line and the frame of the photovoltaic panel is 11 pixel values. The width of the consecutive identical columns formed by the pixel points from the 2nd column pixel to the 7th column pixel lies within the preset width set. Then there are no defects in the consecutive identical columns formed by the pixel points from the 2nd column pixel to the 7th column pixel.

[0072] S204. For each consecutive identical column, if it is determined that the width of the consecutive identical column does not lie within the preset width set, continue to determine whether the consecutive identical column contains the first column pixel or the last column pixel. If so, enter S205; otherwise, enter S206.

[0073] Among them, for each consecutive identical column, compare the width of the consecutive identical column with any standard width in the preset width set. If it is determined that the width of the consecutive identical column does not lie within the preset width set, continue to determine whether the consecutive identical column contains the first column pixel or the last column pixel to eliminate the situation where the width is not the standard width due to incomplete shooting of the entire photovoltaic panel.

[0074] Optionally, for consecutive identical columns whose widths are not within the preset width set, the drone further determines whether they contain the first column pixel or the last column pixel of the image. Among them, since the image edge may be incomplete due to the shooting angle or cropping, consecutive identical columns containing the image edge may appear non-standard due to edge effects. If a consecutive identical column contains the first column or the last column pixel, it is regarded as a standard column and there are no defects; if not, proceed to the next step.

[0075] S205. If the first column pixel or the last column pixel is included in consecutive identical columns, it is determined that there are no defects in the consecutive identical columns.

[0076] Among them, when it is determined that the width of the consecutive identical columns does not fall within the preset width set, it can be determined that the consecutive identical columns are not divided by two adjacent gate lines, or are not divided by connection lines and borders. In order to exclude the situation caused by incomplete shooting of the photovoltaic module, it is determined whether the first column pixel or the last column pixel is included in the consecutive identical columns.

[0077] For example: Figure 4 Taking the local pixel area of the visible light image shown as an example. The consecutive identical columns formed by the pixel points of the pixels in column 9 to column 11 have a width of 3 pixel values and do not fall within the preset width set. Also, the consecutive identical columns formed by the pixel points of the pixels in column 9 to column 11 include the last column pixel. Therefore, there are no defects in the consecutive identical columns formed by the pixel points of the pixels in column 9 to column 11.

[0078] S206. If the first column pixel and the last column pixel are not included in the consecutive identical columns, the area corresponding to the consecutive identical columns is used as a defective area.

[0079] Among them, when it is determined that the width of the consecutive identical columns does not fall within the preset width set, and the first column pixel and the last column pixel are not included in the consecutive identical columns, it can be determined that the consecutive identical columns are not divided by two adjacent gate lines, or are not divided by connection lines and borders, nor are they caused by incomplete shooting of the photovoltaic panel. Therefore, the area corresponding to the consecutive identical columns is used as a defective area.

[0080] Optionally, for consecutive identical columns that are neither in the preset width set nor contain the image edge, the UAV marks the corresponding image area as a defective area. These areas are likely to have an abnormal distribution of pixel values due to cracks, dirt, or other physical damages on the surface of the photovoltaic panel. Marking these areas helps in further analysis and processing of the defect types in the subsequent stage.

[0081] In the above technical solution, since the gate lines and connection lines are distributed horizontally or vertically, and the pixel values in the area where the underlying semiconductor substrate is located are the same, the semiconductor substrate will be divided into multiple pieces, and there will be many consecutive areas with the same pixel values on the image. Taking pixel columns as the unit, two columns of pixels are compared, and adjacent pixel columns with the same pixel value distribution are extracted. In addition, according to the scaling ratio of the photovoltaic panel in the image, the standard width of the consecutive identical columns is determined. After extracting the consecutive identical columns from the image, the width of the consecutive identical columns is compared with the standard width to determine whether there are defects. Since the comparison is carried out column by column, the data processing efficiency can be improved. And the situation where the width is not the standard width caused by incomplete shooting of the photovoltaic panel is excluded, improving the defect recognition accuracy.

[0082] It should be noted that traditional methods may require pixel-by-pixel analysis of the entire image, while the above solution compares in units of pixel columns, greatly reducing the amount of data to be processed, thereby improving the efficiency of data processing and defect recognition. Moreover, by calculating the widths of consecutive identical columns and comparing them with a preset set of widths, columns that do not conform to the specified width can be quickly located, and then the defect area can be determined, realizing fast defect recognition.

[0083] In addition, in the above solution, the situation that the image edge may be incomplete due to the shooting angle or cropping is also considered. By determining whether consecutive identical columns contain the first column pixels or the last column pixels, the influence of the edge effect on defect recognition is effectively excluded, improving the accuracy of recognition. And, by determining the specified width of consecutive identical columns according to the scaling ratio of the photovoltaic panel in the image and comparing it with the actual width, this dynamic adjustment based on the actual situation further improves the accuracy of defect recognition.

[0084] Furthermore, this solution adopts a simple pixel column comparison algorithm, without the need for complex image recognition or deep learning algorithms, reducing the demand for the hardware resources of the drone and the control platform. Since only the defective image area needs to be transmitted instead of the entire image, the amount of data transmission is greatly reduced, reducing the demand for communication bandwidth.

[0085] Moreover, regardless of whether the gate lines and connection lines of the photovoltaic panel are distributed horizontally or vertically, this solution can effectively extract consecutive identical columns for comparison, so it is applicable to the defect detection of different types of photovoltaic panels. The specified width can also be flexibly adjusted according to the actual size of the photovoltaic panel and the image scaling ratio, making the solution more adaptable. And through regular inspections and fast defect recognition, defects in the photovoltaic panel can be discovered in time, avoiding the reduction of power generation or equipment damage caused by the expansion of defects.

[0086] In a possible implementation manner, in S102, the drone extracts the distribution of pixel points with the same pixel value in the visible light image, and determines whether there is a defective area on the photovoltaic panel according to the distribution of pixel points with the same pixel value in the visible light image, specifically including:

[0087] S301, extract the connected regions with the same pixel value in the visible light image, and determine whether the connected region is a rectangular region. If so, go to S302, otherwise, go to S305.

[0088] In the photovoltaic module, multiple gate lines extend longitudinally, and multiple connection lines extend horizontally. The multiple gate lines and multiple connection lines form a grid pattern, forming a grid-shaped connected region in the visible light image. The horizontally extending connection lines and the longitudinally extending gate lines divide the semiconductor substrate into multiple regions, forming multiple rectangular connected regions in the visible light image.

[0089] Compare the pixel values of each pixel point in the visible light image. If the pixel points with the same pixel value are continuously arranged to form a connected region without interruption, this region is regarded as the connected region with the same pixel value.

[0090] Optionally, the drone first collects the visible light image of the photovoltaic panel through the visible light sensor carried by it. Subsequently, an image processing algorithm is used to analyze the image and extract the connected regions with the same pixel value. These connected regions may be formed because the pixel values of the gate lines, connection lines, and semiconductor base layers on the surface of the photovoltaic panel are the same. The extraction of the connected regions can be achieved through image segmentation algorithms such as threshold segmentation and region growing. The algorithm will traverse the entire image and classify the pixel points with the same pixel value and adjacent to each other into the same connected region.

[0091] Take Figure 4 the local pixel area of the visible light image shown as an example. The local pixel area is a 17-row × 11-column pixel array. The pixel points filled with upward slashes in the 2nd row and the 16th row are the connection lines extending horizontally. The pixel points filled with upward slashes in the 1st column and the 8th column are the gate lines extending vertically.

[0092] The pixel points filled with upward slashes in the 2nd row, the pixel points filled with upward slashes in the 16th row, the pixel points filled with upward slashes in the 1st column, and the pixel points filled with upward slashes in the 8th column form a cross-shaped connected region.

[0093] The rectangular region formed by the 3rd-row pixel in the 2nd column, the 15th-row pixel in the 2nd column, the 3rd-row pixel in the 7th column, and the 15th-row pixel in the 7th column as the four vertices is a connected region filled entirely with white.

[0094] S302. If the connected region is a rectangular region, determine whether the size of the rectangular region is within the preset size set. If so, enter S303; otherwise, enter S304.

[0095] Optionally, for each extracted connected region, the drone first determines whether it is a rectangular region. If it is a rectangular region, further measure its size (such as length and width), and compare these sizes with the preset size set. The judgment of the rectangular region can be made by checking whether the boundary of the connected region is composed of straight lines and whether the opposite sides are parallel and of equal length. The preset size set is determined based on the design and manufacturing specifications of the photovoltaic panel and includes the size range that the normal surface structure of the photovoltaic panel (such as gate lines, connection lines, etc.) should have. If the size of the rectangular region is within the preset size set, it is considered that the region is normal and there are no defects; if the size is not within the preset size set, the region is marked as a defective region.

[0096] Determine the scaling ratio of the image according to the internal parameters and external parameters of the visible light image sensor.

[0097] For the rectangular region divided by the gate lines and connection lines, obtain the distance between two adjacent gate lines on the photovoltaic module and the distance between two adjacent connection lines on the photovoltaic module. Calculate the standard width of the rectangular region based on the scaling ratio of the image and the distance between two adjacent gate lines. Obtain the standard length of the rectangular region based on the scaling ratio of the image and the distance between two adjacent connection lines.

[0098] For the rectangular region divided by the gate lines, connection lines, and the longitudinal edge of the photovoltaic module, calculate the standard width of the rectangular region based on the scaling ratio of the image and the distance between the gate line and the longitudinal edge of the photovoltaic module, and obtain the standard length of the rectangular region based on the scaling ratio of the image and the distance between two adjacent connection lines.

[0099] For the rectangular region divided by the gate lines, connection lines, and the transverse edge of the photovoltaic module, calculate the standard width of the rectangular region based on the scaling ratio of the image and the distance between two gate lines, and obtain the standard length of the rectangular region based on the scaling ratio of the image and the distance between the connection line and the transverse edge of the photovoltaic module.

[0100] For the rectangular region divided by the gate lines, connection lines, the transverse edge, and the longitudinal edge of the photovoltaic module, calculate the standard width of the rectangular region based on the scaling ratio of the image and the distance between the gate line and the longitudinal edge of the photovoltaic module, and obtain the standard length of the rectangular region based on the scaling ratio of the image and the distance between the connection line and the transverse edge of the photovoltaic module.

[0101] The preset size set includes the standard sizes of multiple rectangular regions, and the standard size of a rectangular region includes the standard width and the standard length of the rectangular region.

[0102] If the connected region is a rectangular region, compare the size of the connected region with each standard size in the preset size set, and then determine whether the size of the rectangular region is within the preset size set.

[0103] S303. If it is determined that the size of the rectangular region is within the preset size set, there are no defects in the connected region.

[0104] Among them, if it is determined that the size of the rectangular region is within the preset size set, it means that the connected region is divided by the gate lines and connection lines. Therefore, it is determined that there are no defects in this connected region.

[0105] S304. If it is determined that the size of the rectangular region is not within the preset size set, regard the connected region as a defective region.

[0106] Among them, if it is determined that the size of the rectangular region is not within the preset size set, it means that the connected region is divided by dirt, cracks, or hot spots. Therefore, it is determined that there are defects in this connected region.

[0107] S305. If the connected region is a cross-shaped region, determine whether the lengths of multiple horizontal regions in the cross-shaped region are the same, and whether the lengths of multiple vertical regions in the cross-shaped region are the same. If both are the same, proceed to S306; otherwise, proceed to S307.

[0108] Among them, the cross-shaped region includes connection lines arranged horizontally and gate lines arranged vertically. The lengths of all gate lines are the same, and the lengths of all connection lines are the same. When determining that the connected region is a cross-shaped region, determine whether the lengths of multiple horizontal regions in the cross-shaped region are the same, and whether the lengths of multiple vertical regions in the cross-shaped region are the same, so as to determine whether there are any missing gate lines and connection lines.

[0109] If the lengths of all horizontal regions are the same, then there is no missing connection line. If the lengths of the horizontal regions are not the same, then the short connection line is missing. If the lengths of all vertical regions are the same, then there is no missing gate line; otherwise, the short gate line is missing.

[0110] Optionally, for a non-rectangular connected region distributed in a cross shape, the drone needs to determine whether the lengths of its multiple horizontal regions and vertical regions are the same. Among them, the determination of the cross-shaped region can be made by checking whether the connected region is composed of multiple intersecting straight lines (or approximately straight lines), and these straight lines divide the region into multiple small rectangles or squares. Subsequently, measure the lengths of these small rectangles or squares, and compare whether the lengths of the horizontal and vertical regions are consistent. If the lengths of all horizontal regions in the cross-shaped region are the same, and the lengths of all vertical regions are also the same, then the region is considered normal and there are no defects; if the lengths of the horizontal or vertical regions are not the same, then the region is marked as a defective region.

[0111] S306. If both the horizontal region and the vertical region are the same, the connected region has no defects.

[0112] Among them, if both the horizontal region and the vertical region are the same, it means that neither the connection lines arranged horizontally nor the gate lines arranged vertically are missing, so there are no defects in the cross-shaped connected region.

[0113] S307. If the horizontal region or the vertical region is not the same, then regard the connected region as a defective region.

[0114] Among them, if the horizontal regions are not the same, then the connection lines arranged horizontally are defective, and the short connection line is missing. If the vertical regions are not the same, then the gate lines arranged vertically are defective, and the short gate line is missing. If the horizontal region or the vertical region is not the same, then regard the connected region as a defective region. Further, regard the short horizontal region or the short vertical region as a defective region.

[0115] Optionally, for the connected regions marked as defective areas, the drone records information such as their positions and sizes in the visible light image, and transmits this information together with the image to the control platform. The marking of defective areas can be achieved by adding annotations on the image, changing pixel colors or values, etc. The data transmitted to the control platform can include metadata such as the compressed version of the image, the coordinates and sizes of the defective areas. After receiving the data, the control platform can further analyze the type and severity of the defects, and formulate corresponding repair or maintenance plans accordingly.

[0116] In the above technical solution, since the gate lines and connection lines are distributed horizontally or vertically, because the pixel values in the area where the underlying semiconductor layer is located are the same, the semiconductor layer will be divided into multiple pieces, and there will be many rectangular connected regions with the same pixel values and cross-shaped connected regions on the image. By extracting the connected regions in the image and determining whether the connected regions are rectangular regions or cross-shaped regions, if neither is the case, it indicates that there are defects in this area. If it is a rectangular region or a cross-shaped region, then continue to determine whether the region size is the standard width to further eliminate misjudged results, thus improving the recognition accuracy. Since defect recognition is carried out according to connected regions, the connected regions are usually relatively small, and the defective areas can be located more accurately.

[0117] It should be noted that by extracting the connected regions with the same pixel values in the visible light image, the technical solution directly focuses on the key information in the image, that is, the structural features on the surface of the photovoltaic panel. This avoids pixel-by-pixel analysis of the entire image and greatly improves the extraction efficiency of defect information.

[0118] For rectangular connected regions, by determining whether their sizes are within the preset size set, the technical solution can accurately identify regions with abnormal sizes, and these regions are likely to have defects.

[0119] For cross-shaped connected regions, by comparing whether the lengths of multiple horizontal and vertical regions are the same, the technical solution can further screen out regions with irregular structures, which are also potential defective areas.

[0120] It can be seen that the above technical solution not only considers the shape of the connected regions (rectangular or cross-shaped), but also combines the judgment of size or length. This multi-dimensional analysis effectively reduces misjudgments caused by a single feature.

[0121] Moreover, by setting the preset size set and the standard width, the above technical solution can also eliminate slight pixel value changes caused by factors such as the image shooting angle and lighting conditions, thereby further improving the recognition accuracy.

[0122] In addition, since defect identification is carried out according to connected regions, and the connected regions are usually small and concentrated, the technical solution can more accurately locate the defect region, providing accurate guidance for subsequent repair and maintenance.

[0123] In a possible implementation manner, in S105, the control platform determines the photovoltaic panel with a defect and the defect type according to the visible light image, the infrared image, and the image capture point of the marked defect region, specifically including:

[0124] S401. Determine the coordinates of the geographical region captured in the visible light image according to the field of view angle of the visible light sensor and the image capture point. According to the coordinates of the captured geographical region and the photovoltaic panel layout data, determine the number of the photovoltaic panel captured in the visible light image and the pixel region corresponding to the captured photovoltaic panel.

[0125] Among them, Figure 5 is an imaging schematic diagram of the visible light sensor provided in some embodiments of the present application. As Figure 5 shown, obtain the height information of the photovoltaic panel, obtain the height difference △H between the photovoltaic panel and the image capture point based on the height information of the photovoltaic panel and the height information of the image capture point, and calculate the coordinates of the captured geographical region based on the height difference △H between the photovoltaic panel and the image capture point, the longitude and latitude coordinates of the image capture point, and the field of view angles (θ1 and θ2) of the visible light sensor.

[0126] The photovoltaic panel layout data includes the longitude and latitude of each photovoltaic panel. Compare the coordinates of the captured geographical region with the photovoltaic panel layout data to determine the area occupied by the photovoltaic panel in the captured geographical region. Use the number of the photovoltaic panel in the captured geographical region as the number of the photovoltaic panel captured in the visible light image, and determine the pixel region corresponding to the captured photovoltaic panel according to the area occupied by the photovoltaic panel in the captured geographical region.

[0127] Optionally, the control platform first receives the visible light image transmitted by the drone and the corresponding image capture point information. Using the field of view angle of the visible light sensor (i.e., the angle of the image range that the sensor can capture) and the geographical location information of the image capture point (such as longitude, latitude, height, etc.), determine the coordinate range of the geographical region captured in the visible light image through geometric calculation. This step involves the application of a geographic information system, and it is necessary to convert the three-dimensional coordinates (if height information is included) of the image capture point into two-dimensional plane coordinates (longitude and latitude), and combine the field of view angle of the sensor to calculate the geographical region range covered by the image.

[0128] Furthermore, the control platform can compare the calculated geographical area coordinates with the photovoltaic panel layout data of the photovoltaic power station. The photovoltaic panel layout data includes the specific position information (such as longitude and latitude) and numbers of each photovoltaic panel. Through the comparison, the control platform can determine which photovoltaic panels are captured in the visible light image and calculate the corresponding pixel areas of these photovoltaic panels in the image. This step requires accurate photovoltaic panel layout data as a support.

[0129] S402. Determine the photovoltaic panels with defects based on the pixel areas corresponding to the captured photovoltaic panels and the defect areas. Determine the defect types according to the pixel values of the defect areas and the infrared images.

[0130] Among them, compare the defect area with the pixel areas corresponding to the captured photovoltaic panels. If the defect area coincides with the pixel area corresponding to a certain captured photovoltaic panel, it is determined that the photovoltaic panel has a defect. By analyzing the pixel values of the defect area and combining with the infrared image, determine the type of the photovoltaic panel.

[0131] Optionally, the control platform compares the previously marked defect areas with the pixel areas corresponding to the captured photovoltaic panels. Through the comparison, it can be determined which photovoltaic panels have defect areas. This step requires accurate pixel coordinate matching. The control platform can write an algorithm to automatically traverse the pixel coordinates of the defect areas and check whether these coordinates fall within the pixel areas of the captured photovoltaic panels. If so, mark the photovoltaic panel as having a defect.

[0132] Then, the control platform combines the pixel values of the defect areas and the infrared images to determine the defect types. Different defect types may exhibit different characteristics in the visible light image and the infrared image, such as color, brightness, temperature, etc. This step requires establishing the corresponding relationship between the defect types and the image characteristics. The control platform can train a model to identify different defect types based on machine learning or deep learning algorithms. The model can receive the pixel values of the defect areas and the infrared images as inputs and output the prediction results of the defect types.

[0133] Once the photovoltaic panels with defects and the defect types are determined, the control platform can generate a report and send the relevant information (such as photovoltaic panel numbers, defect types, defect positions, etc.) to the operation and maintenance personnel. The operation and maintenance personnel can carry out further inspection and repair work according to the report. The report generation and sending can be completed through the user interface of the control platform or the automated email system. The report should contain clear information and images so that the operation and maintenance personnel can quickly understand and locate the problem.

[0134] In the above technical solution, since multiple photovoltaic panels are arranged in an array, it is impossible to locate according to the shape of the object being photographed and the shapes of surrounding objects. In order to determine the position of the photovoltaic panels in the photographed image, when using a visible light image to take a picture, the shooting point of this frame of image is collected at the same time. In this way, the coordinates of the geographical area being photographed can be determined based on the shooting point and the field of view angle of the visible light. The pixel coordinates in the image are converted into the geodetic coordinate system. In this way, only by comparing the coordinates of the photographed geographical area with the photovoltaic panel layout data, the label of the photovoltaic panel captured in the photographed geographical area and the pixel area corresponding to the captured photovoltaic panel can be determined. Furthermore, the defective area can be compared with the pixel area corresponding to the captured photovoltaic panel to determine which specific photovoltaic panel among the captured photovoltaic panels has a defect, and finally the positioning of the faulty photovoltaic panel can be achieved.

[0135] By utilizing the field of view angle of the visible light sensor and the image shooting point information, the above technical solution can determine the coordinates of the geographical area photographed in the visible light image. This step provides an accurate geographical reference basis for subsequent photovoltaic panel positioning and defect identification. Then, by combining the coordinates of the photographed geographical area and the photovoltaic panel layout data, the number of the photovoltaic panel captured in the visible light image and the corresponding pixel area can be determined. Even if the photovoltaic panels are arranged in an array and it is difficult to locate them by shape and surrounding objects, this technology can effectively identify the specific photovoltaic panels.

[0136] Next, by comparing the pixel area corresponding to the captured photovoltaic panel with the marked defective area, the photovoltaic panel with a defect can be quickly determined. This step greatly shortens the positioning time of the faulty photovoltaic panel and improves the operation and maintenance efficiency.

[0137] Then, by combining the pixel values of the defective area and the infrared image information, the type of defect of the photovoltaic panel can be accurately judged. Different types of defects exhibit different characteristics in visible light and infrared images. By comprehensively analyzing these characteristics, accurate identification of the defect type can be achieved.

[0138] The above technical solution can realize the efficient identification and positioning of photovoltaic panel defects in an automated and intelligent manner, greatly reducing the workload of operation and maintenance personnel. At the same time, since the identification process is based on accurate data and algorithms, the identification results have high accuracy, reducing the possibility of misjudgment and missed judgment. By regularly inspecting and timely identifying and handling the defects of photovoltaic panels, the technical solution helps to improve the overall reliability of photovoltaic power stations. This can not only reduce the power generation loss caused by defects, but also extend the service life of photovoltaic panels and reduce the operation and maintenance costs.

[0139] In a possible implementation manner, S402, when the defective area is the initial pixel area corresponding to the same adjacent columns, determining the photovoltaic panel with a defect according to the pixel area corresponding to the captured photovoltaic panel and the defective area specifically includes:

[0140] S501. For the initial pixel region where consecutive identical columns are located, determine whether the pixel values in each row of the initial pixel region are the same. If not, proceed to S502; otherwise, proceed to S503.

[0141] Among them, analyze the consecutive identical columns with defects row by row, take the region where the consecutive identical columns with defects are located as the initial pixel region, analyze the pixels in each row of the initial pixel region, and determine whether the pixel values in each pixel of each row are the same. If not, the pixels in this row may contain defective information, so retain the pixels in this row. Otherwise, further analyze the pixel values of this column.

[0142] Optionally, the control platform first identifies the consecutive identical columns marked as defective regions in the visible light image. These columns appear as adjacent regions with similar pixel values (such as color, brightness, etc.) in the image. Using image processing algorithms, such as edge detection, region growing, etc., these consecutive identical columns can be automatically identified, and the initial pixel region they occupy can be determined.

[0143] For each row of the initial pixel region, the control platform checks whether the pixel values in this row are the same. If they are different, retain the pixels in this row because different pixel values may indicate the presence of defects. If they are the same, further determine whether the pixels in this row are standard pixel values (i.e., the pixel values that a normal photovoltaic panel should have). If they are not standard pixel values, also retain the pixels in this row because it may indicate a type of defect. If they are standard pixel values, remove the pixels in this row because it is less likely to be the location of the defect. Among them, the standard pixel values can be obtained by statistically analyzing the images of normal photovoltaic panels. When comparing, the difference or similarity measure between pixel values can be used to determine whether they are the same or similar.

[0144] S502. Retain the pixels in this row of the initial pixel region.

[0145] S503. Determine whether the pixels in this row are standard pixel values. If not, retain the pixels in this row of the initial pixel region; if so, delete the pixels in this row of the initial pixel region.

[0146] Among them, obtain the pixel values corresponding to the gate lines and connection lines, as well as the pixel values corresponding to the semiconductor layer, and use the above pixel values as the standard pixel values. If all the pixel values of a certain row of pixels in the initial pixel region are the same and are standard pixel values, there is no defective information in this column of pixels, and the pixels in this row are deleted. If all the pixel values of a certain row of pixels in the initial pixel region are the same but are not standard pixel values, there is defective information in this column of pixels, and the pixels in this row of the initial pixel region are retained.

[0147] S504. Determine whether it is the last pixel row in the initial pixel region. If so, output the shrunk defective region. If not, update the pixel row in the initial pixel region and return to S501.

[0148] Among them, by traversing the pixels in each row of the initial pixel region, it is judged whether there is defective information in the pixels of each row. If not, the pixels of that row are deleted. Otherwise, the pixels of that row are retained, and thus the shrunk defective region is obtained.

[0149] S505. For each photographed photovoltaic panel, determine whether the shrunk defective region overlaps with the pixel region corresponding to the photographed photovoltaic panel. If so, the photographed photovoltaic panel has a defect.

[0150] Among them, for the currently photographed photovoltaic panel, the shrunk defective region is compared with the pixel region corresponding to the photographed photovoltaic panel to determine whether the shrunk defective region overlaps with the pixel region corresponding to the photographed photovoltaic panel. If so, the photographed photovoltaic panel has a defect. If not, it is compared with the pixel region corresponding to the next photographed photovoltaic panel until the photographed photovoltaic panel with a defect is determined.

[0151] In the above technical solution, the control platform analyzes the pixel values in the continuous same columns, utilizes the feature that most of the pixel values in the continuous same columns are the same. If the pixel values in a certain row are all the same and are not the standard pixel values, then the pixel values of that row are removed, further shrinking the defective region. Based on the comparison between the shrunk defective region and the regions corresponding to each photographed photovoltaic panel, the photovoltaic panel with a defect can be accurately located. In addition, defect type recognition is performed within the shrunk pixel region, reducing the amount of data processing during defect type recognition and improving the data processing efficiency.

[0152] Optionally, the control platform compares the shrunk defective region with the pixel region corresponding to each photographed photovoltaic panel to check whether there is an overlapping part between them. This step can be achieved by calculating the intersection of the two regions. If the intersection is not empty, it means that the shrunk defective region overlaps with a certain part of the photographed photovoltaic panel.

[0153] If the shrunk defective region overlaps with the pixel region of the photographed photovoltaic panel, the control platform determines that the photovoltaic panel has a defect. This determination is based on the previous analysis and processing results. The existence of the overlap means that a pixel region matching the defective region has been found on the image of the photovoltaic panel.

[0154] Once a defective photovoltaic panel is identified, the control platform can further identify the type of defect, such as cracks, stains, occlusion, etc., and generate corresponding reports or alarms so that the operation and maintenance personnel can take measures in a timely manner. Through this series of steps, the control platform can more accurately locate the defective photovoltaic panels, reducing false alarms and missed alarms. At the same time, since the scope of the defect area is reduced, the efficiency and accuracy of defect type identification are also improved. In addition, the entire processing process has a high degree of automation, reducing the workload of the operation and maintenance personnel.

[0155] In a possible implementation manner, S402. Determine the defect type according to the pixel values of the defect area and the infrared image, specifically including:

[0156] S601. Extract the connected regions with the same pixel values in the reduced defect area; for each connected region, if there is a connected region in the shape of a thin strip, there is a crack defect in the connected region.

[0157] Among them, analyze the pixel values of each pixel point in the reduced defect area to obtain the connected regions in the reduced defect area. Analyze the shape of the connected region to determine whether the connected region is in the shape of a thin strip. If so, it is determined that there is a crack defect in the connected region.

[0158] Optionally, in the already reduced defect area, use image processing techniques (such as region growing, threshold segmentation, etc.) to extract the connected regions with the same pixel values. These connected regions may be composed of pixel points with similar and adjacent pixel values, and they may represent a specific defect.

[0159] The extraction of the connected region can be achieved by traversing each pixel point in the reduced defect area, finding the pixel points with the same and adjacent pixel values as it, and then combining them into a connected region. This process can be repeated until all pixel points are classified into a certain connected region or no more connected regions can be extracted.

[0160] Furthermore, for each extracted connected region, judge whether its shape is in the shape of a thin strip. If so, it is considered that there is a crack defect in the connected region. Among them, the crack defect usually appears as a slender and connected region with similar pixel values. Therefore, the shape characteristics can be detected by calculating the aspect ratio of the connected region or using morphological operations (such as erosion, dilation, etc.) to determine whether there is a crack defect.

[0161] S602. If the connected region is not in the shape of a thin strip, determine the pixel region corresponding to the connected region in the infrared image. If the temperature value in the pixel region is greater than or equal to the preset threshold, it is determined that there is a hot spot in the connected region. If the temperature value in the pixel region is less than the preset threshold, it is determined that there is dirt in the connected region.

[0162] Among them, if the excluded connected region is not in a thin strip shape, the connected region is matched with the infrared image to determine the pixel region corresponding to the connected region in the infrared image. Then, it is judged whether the temperature in this pixel region is greater than or equal to a preset threshold. If so, it is determined that there is a hot spot in the connected region. Otherwise, there is dirt in the connected region.

[0163] Optionally, the connected region in the visible light image is mapped onto the infrared image to find the corresponding pixel region. This usually needs to be achieved through image registration technology to ensure that the pixel points in the visible light image and the infrared image can correspond accurately. Among them, image registration technology can use methods such as feature point matching and image transformation to align the visible light image and the infrared image so that the pixel points in them can correspond one by one. Then, according to the position of the connected region in the visible light image, the corresponding pixel region can be found in the infrared image.

[0164] For the pixel region corresponding to the connected region in the infrared image, calculate its average temperature value or the highest temperature value, and compare it with the preset threshold. According to the comparison result, determine the type of defect existing in the connected region. If the temperature value of the pixel region is greater than or equal to the preset threshold, it is considered that there is a hot spot defect in the connected region. Hot spots are usually caused by local overheating of the photovoltaic panel, which may be due to reasons such as damaged solar cells or poor wiring. If the temperature value of the pixel region is less than the preset threshold, it is considered that there is a dirt defect in the connected region. Dirt will block sunlight, causing the photovoltaic panel to not work properly, thereby reducing power generation. Dirt may be caused by debris such as dust, bird droppings, and leaves covering the photovoltaic panel. Then, according to the determined defect type, generate a corresponding report or alarm so that the operation and maintenance personnel can take measures to repair or clean it in time. At the same time, the defect information can also be stored in the database for subsequent analysis and optimization.

[0165] In the above technical solution, connected regions are extracted in the defect region, and the shape of the connected regions is analyzed to determine whether there are crack defects. And combined with the temperature data of the infrared image to determine whether there are hot spots. After excluding the above two types of defects, the dirt phenomenon can be determined, and the identification of defect, hot spot, and dirt categories is realized in the above way.

[0166] It is worth noting that by extracting the connected regions with the same pixel value in the reduced defect region and introducing shape analysis technology, we can effectively identify the connected regions in the shape of thin strips, which often correspond to the crack defects on the photovoltaic panel. This method can improve the accuracy and efficiency of crack identification and avoid the misjudgment problem that may occur due to similar pixel values in traditional methods.

[0167] Then, by combining the temperature data of the infrared image, it is possible to accurately locate the pixel region corresponding to the connected region, and by comparing it with a preset threshold, quickly determine whether there is a hot spot in this region. The introduction of the infrared image makes the detection of hot spots no longer rely on the naked-eye observation of the visible light image, but is based on objective temperature data, thus greatly improving the accuracy and reliability of the detection.

[0168] After excluding crack defects and hot spot phenomena, we utilize the characteristic that the temperature value of the pixel region in the infrared image is less than the preset threshold to accurately determine the dirt phenomenon existing in the connected region. The application of this exclusion method not only simplifies the detection process but also avoids the problem of recognition confusion caused by the simultaneous existence of multiple defects, ensuring the accuracy of dirt recognition.

[0169] Through the comprehensive application of the above technical solutions, it is possible to achieve a comprehensive, rapid, and accurate identification of defects, hot spots, and dirt categories on the photovoltaic panel. This comprehensive detection method not only improves the detection accuracy but also greatly shortens the detection time, providing strong technical support for the operation and maintenance work of the photovoltaic power station. At the same time, the automated detection process also reduces the workload of the operation and maintenance personnel and improves work efficiency. Accurate and timely identification of defects, hot spots, and dirt is crucial for the long-term stable operation of the photovoltaic power station. By promptly discovering and handling these problems, it is possible to effectively avoid the performance degradation or damage of the photovoltaic panel caused by defects, hot spots, or dirt, thereby extending the service life of the photovoltaic power station and improving its power generation efficiency.

[0170] In a possible implementation manner, after S105, the control platform determines the photovoltaic panel with defects and the defect type based on the visible light image, infrared image, and image capture point of the marked defect region, the method further includes:

[0171] S106. If it is determined that the number of photovoltaic panels with dirt is greater than the preset number threshold, generate a cleaning work order for cleaning the photovoltaic panels.

[0172] Among them, the cleaning work order includes the number of photovoltaic panels to be cleaned and the required cleaning time. By inspecting the photovoltaic panels in the photovoltaic power station, determine the number of photovoltaic panels with dirt based on the visible light image and infrared image obtained from the inspection.

[0173] The preset number threshold can be set to half of the total number of photovoltaic panels in the photovoltaic power station. If the number of photovoltaic panels with dirt is greater than the preset number threshold, it means that a large area of photovoltaic panels is dirty, affecting the power generation power of the entire photovoltaic power station. Therefore, after it is determined that the number of photovoltaic panels with dirt is greater than the preset number threshold, generate a cleaning work order for cleaning the photovoltaic panels and trigger the cleaning robot to clean the photovoltaic panels.

[0174] S107. Determine the cleaning robot for performing the cleaning operation according to the cleaning work order, the busy or idle status of the cleaning robot, and the cleaning capacity of the cleaning robot.

[0175] Among them, select the cleaning robot that can perform the cleaning operation within the required cleaning time according to the busy or idle status of the cleaning robot and the required cleaning time, and determine the cleaning robot for performing the cleaning operation according to the cleaning capacity of the cleaning robot that can perform the cleaning operation within the cleaning time and the number of photovoltaic panels to be cleaned, so as to ensure that the cleaning robot for performing the cleaning operation can complete the cleaning operation within the required cleaning time.

[0176] S108. The control platform sends a cleaning instruction to the cleaning robot.

[0177] S109. The cleaning robot cleans the photovoltaic panels at the required cleaning time.

[0178] In the above technical solution, after the control platform analyzes the visible light image and the infrared image and determines that the number of soiled photovoltaic panels is greater than the preset number threshold, it automatically generates a cleaning work order for cleaning the photovoltaic panels, schedules the cleaning robot based on the cleaning work order and the cleaning capacity of the cleaning robot, generates a cleaning instruction, controls the cleaning robot to perform the cleaning, and in time cleans after the soiling affects the power generation capacity of the photovoltaic power station, restores the power generation capacity of the photovoltaic power station, and improves the operation efficiency of the photovoltaic power station.

[0179] It should be noted that the control platform can accurately identify the soiling condition on the surface of the photovoltaic panels by analyzing the visible light image and the infrared image. This not only avoids the subjectivity and inaccuracy of manual inspection, but also significantly improves the detection efficiency, making the operation and maintenance work of the photovoltaic power station more timely and effective.

[0180] When the control platform determines that the number of soiled photovoltaic panels exceeds the preset number threshold, it will automatically generate a cleaning work order for cleaning the photovoltaic panels. This step simplifies the operation and maintenance process, reduces human intervention, and ensures the timeliness and accuracy of the cleaning work. At the same time, the number of photovoltaic panels to be cleaned is clearly listed in the cleaning work order, providing clear task guidance for the subsequent cleaning operation.

[0181] Moreover, the control platform intelligently selects the cleaning robot to be operated according to the cleaning work order, the busy or idle status of the cleaning robot, and the cleaning capacity, and sends a cleaning instruction to it. This intelligent scheduling mechanism ensures the reasonable allocation and efficient utilization of cleaning resources, avoids the idleness or overuse of cleaning robots, and thus improves the cleaning efficiency.

[0182] In addition, by promptly identifying dirt and generating cleaning work orders, and combined with the rapid response and efficient cleaning of cleaning robots, this technical solution can quickly restore the cleanliness of photovoltaic panels before the dirt affects the power generation capacity of the photovoltaic power station. This not only avoids power generation losses caused by dirt but also improves the overall operating efficiency of the photovoltaic power station.

[0183] Some embodiments of this application also provide a method for outdoor inspection of a photovoltaic power station based on machine vision. The method includes the following steps:

[0184] S701. A drone surveys the photovoltaic power station and sends the survey results to the control platform.

[0185] Among them, a drone operator controls the drone to survey the photovoltaic power station and sends the survey results to the control platform in real time.

[0186] S702. The control platform draws an electronic map of the photovoltaic power station based on the survey results and generates an inspection path according to the electronic map of the photovoltaic power station.

[0187] Among them, the control platform uses existing map drawing algorithms to process the results of on-site surveys by the drone to draw an electronic map of the photovoltaic power station. Obtain the photovoltaic panel layout data from the electronic map of the photovoltaic power station, and generate an inspection path according to the photovoltaic panel layout data to ensure that each photovoltaic panel can be inspected and the drone flight path is the shortest. The control platform also sets parameters such as the flight altitude and speed of the drone, and can set the drone to work in the terrain imitation mode. Compared with fixed-altitude flight, terrain imitation flight can ensure clear images are captured and can also prevent the drone from being too close to the photovoltaic panel and damaging the machine.

[0188] S703. The drone performs terrain imitation flight along the inspection path based on the electronic map, uses a visible light sensor to collect visible light images of the photovoltaic panels, and uses an infrared sensor to collect infrared light images of the photovoltaic panels.

[0189] S704. The drone performs geometric transformation processing on the visible light images to make the frame of the photovoltaic panel parallel to the image edge, and obtains the geometrically transformed visible light images.

[0190] Among them, the geometric transformation includes linear transformation and translation. Linear transformation can change the size, shape, and direction of an object, while translation can change the position of an object. By combining linear transformation and translation, the object edge can be made parallel to the frame of the photovoltaic panel and the image edge.

[0191] Optionally, the drone first collects a visible light image of the photovoltaic panel through a visible light sensor, and then processes the original visible light image using geometric transformation methods in image processing technology, such as rotation, translation, and scaling, so that the border of the photovoltaic panel is parallel to the edge of the image.

[0192] The specific parameters of the geometric transformation can be determined by analyzing the tilt angle, position and other information of the photovoltaic panel frame in the image. The transformation parameters can be set automatically or manually to ensure that the photovoltaic panel frame in the transformed image is parallel to the edge of the image. The transformed image should retain all the information in the original image, but only change the spatial position of the pixel points.

[0193] S705. The UAV performs denoising on the visible light image after the geometric transformation to obtain a denoised visible light image.

[0194] Optionally, the visible light image after geometric transformation is subjected to denoising to remove noise interference in the image and improve image quality. Among them, a variety of denoising algorithms can be used, such as mean filtering, median filtering, Gaussian filtering, wavelet denoising, etc., and a suitable denoising method is selected according to the specific situation of the image and the type of noise. The denoising process should retain the detailed information in the image as much as possible, while removing noise to improve the accuracy of subsequent defect recognition.

[0195] S706. The drone extracts the distribution of pixel points with the same pixel value in the visible light image, and determines whether there is a defective area in the photovoltaic panel according to the distribution of pixel points with the same pixel value in the visible light image.

[0196] Optionally, the drone extracts pixels with the same pixel value in the processed visible light image and analyzes the distribution of these pixel points. Among them, image segmentation, clustering and other methods can be used to extract pixels with the same pixel value. The analysis of the distribution may include the number, position, shape and other features of the pixel points, so that it can be determined whether there is a defective area in the photovoltaic panel based on these features.

[0197] Based on the distribution of extracted pixels with the same pixel value, by comparing the pixel value distribution of each pixel column, it is analyzed whether there is an abnormality or inconsistency, so as to determine whether there is a defective area in the photovoltaic panel. Certain thresholds or standards can be set, and when the distribution of pixel points exceeds these thresholds or standards, it is considered that there is a defective area. Among them, the determination of the defective area can be further combined with the actual structure and functional characteristics of the photovoltaic panel, as well as the common defect types for comprehensive judgment.

[0198] S707: When it is determined that there is no defect, the drone deletes the visible light image, the infrared light image, and the image shooting point.

[0199] S708. When a defect is determined to exist, the drone will send the visible light image, infrared light image, and image capture point of the defect area to the background server.

[0200] S709. The background server determines the photovoltaic panel with the defect and the defect type based on the visible light image, infrared light image, and image capture point of the marked defect area.

[0201] In the above technical solution, the drone surveys the photovoltaic power station. After the control platform processes the survey results, an electronic map of the photovoltaic power station is obtained. The control platform designates an inspection path based on the electronic map, enabling the drone to fly along the inspection path based on the electronic map without the need for a drone operator to operate the drone, ensuring the flight reliability of the drone and the quality of image acquisition.

[0202] It should be noted that the drone first conducts a comprehensive survey of the photovoltaic power station. This step is crucial as it provides detailed on-site information for subsequent image acquisition tasks. Through the survey, the drone can capture key data such as the layout of the photovoltaic power station, the location of photovoltaic panels, and possible obstacles, laying the foundation for subsequent task planning. After the drone sends the survey results to the control platform, the control platform uses this data to draw a detailed electronic map of the photovoltaic power station. This map not only accurately reflects the actual layout of the photovoltaic power station but also provides an accurate reference for the flight path planning of the drone. The generation of the electronic map enables the drone to rely on the map information for precise positioning even in the absence of a GPS signal or when the signal is unstable.

[0203] Moreover, the control platform intelligently generates the optimal inspection path based on the electronic map of the photovoltaic power station. This path fully considers the distribution of photovoltaic panels, the location of obstacles, and the flight capabilities of the drone, ensuring that the drone can complete the inspection task with the shortest path and the highest efficiency. At the same time, the generation of the inspection path also takes into account the battery life and flight safety of the drone, avoiding flight accidents or mission failures caused by an unreasonable path.

[0204] In addition, the drone realizes autonomous flight based on the inspection path generated by the control platform. During this process, the drone does not require real-time operation by a drone operator, greatly reducing the workload of the drone operator and improving flight reliability. At the same time, since the flight path is pre-planned, the drone can fly in a more stable and steady posture, thus ensuring the quality of image acquisition. Whether it is a visible light image or an infrared image, it can be captured with higher clarity and accuracy.

[0205] In addition, after the visible light sensor captures the visible light image, by performing geometric transformation on the visible light image, the frame of the photovoltaic panel in the visible light image is made parallel to the image edge, so as to ensure that the pixels corresponding to the gate lines or connection lines are located in one column or one row. In this way, the defect area can be determined based on the pixel value distribution of each pixel column by comparing the pixel value distributions of each pixel column, improving the accuracy of defect recognition.

[0206] It should be noted that by performing geometric transformation processing on the visible light image, it can be ensured that the frame of the photovoltaic panel is parallel to the image edge. This processing step is crucial because it makes the gate lines or connection lines on the photovoltaic panel appear as straight lines in the image, and the corresponding pixels are located in one column or one row. This alignment not only simplifies the subsequent image processing steps but also provides a basis for accurate pixel value comparison.

[0207] The visible light image after geometric transformation may still contain noise, which will interfere with the recognition of defects. Therefore, we perform denoising processing on the image, effectively removing the random noise and interference in the image and retaining the real information on the surface of the photovoltaic panel. The denoised image is clearer, providing high-quality input for the subsequent extraction of the pixel point distribution. Based on the geometric transformation and denoising processing, the drone extracts the pixel point distribution of the same pixel values in the denoised visible light image. By comparing the pixel value distributions of each pixel column, we can accurately identify the defect area on the photovoltaic panel. This recognition method based on pixel value distribution is not only highly sensitive but also can distinguish different types of defects, such as cracks, stains, occlusions, etc.

[0208] In addition, the implementation of the above technical solution can significantly improve the accuracy and efficiency of defect recognition. Geometric transformation processing ensures the alignment of the image, denoising processing enhances the image quality, and the extraction of the pixel point distribution realizes the accurate recognition of the defect area. The synergistic effect of these steps enables us to quickly and accurately locate the defect area on the photovoltaic panel, providing strong technical support for the operation and maintenance work of the photovoltaic power station.

[0209] The control platform provided in this embodiment includes: at least one processor and a memory. Optionally, the device further includes a communication component. Among them, the processor, the memory, and the communication component are connected through a bus.

[0210] In the specific implementation process, at least one processor executes the computer execution instructions stored in the memory, so that at least one processor executes the above method.

[0211] For the specific implementation process of the processor, reference can be made to the above method embodiment, and its implementation principle and technical effects are similar. Therefore, they will not be elaborated here in this embodiment.

[0212] The present application also provides a computer program product, including a computer program which, when executed by a processor, implements the above method.

[0213] The present application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above method.

[0214] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include known common knowledge or conventional technical means in the technical field not disclosed by the present invention. It is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A photovoltaic station outdoor inspection method based on machine vision, characterized in that: The method is applied to a drone and a control platform, wherein the drone and the control platform are in communication connection, and the method comprises: The drone flies along the inspection route, uses a visible light sensor to collect visible light images of the photovoltaic panels, and uses an infrared sensor to collect infrared images of the photovoltaic panels; The drone extracts the distribution of pixel points with the same pixel value in the visible light image, and determines whether there is a defective area in the photovoltaic panel according to the distribution of pixel points with the same pixel value in the visible light image; When the UAV determines that there is a defect, the UAV sends the visible light image of the marked defect area, the infrared image, and the image shooting point to the control platform; The control platform determines the photovoltaic panel where the defect occurs and the defect type according to the visible light image of the marked defect area, the infrared image and the image shooting point.

2. The method according to claim 1, characterized in that The drone extracts the distribution of pixels with the same pixel value in the visible light image, and determines whether there is a defective area in the photovoltaic panel according to the distribution of pixels with the same pixel value in the visible light image, specifically including: Extracting pixel points in each column of the visible light image, comparing the pixel points in each column, and obtaining the width, starting pixel column, and ending pixel column of consecutive identical columns; wherein the consecutive identical columns refer to adjacent pixel columns having the same pixel value distribution; For each continuous identical column, if it is determined that the width of the continuous identical column is within a preset width set, it is determined that there is no defect in the continuous identical column; For each consecutive identical column, if it is determined that the width of the consecutive identical column is not within the preset width set, continue to determine whether the consecutive identical column contains the first column of pixels or the last column of pixels. If so, determine that there is no defect in the consecutive identical column; if not, take the area corresponding to the consecutive identical column as a defective area.

3. The method according to claim 1, characterized in that The drone extracts the distribution of pixels with the same pixel value in the visible light image, and determines whether there is a defective area in the photovoltaic panel according to the distribution of pixels with the same pixel value in the visible light image, specifically including: Extracting a connected area with the same pixel value in the visible light image, if the connected area is a rectangular area, determining whether the size of the rectangular area is within a preset size set, if so, there is no defect in the connected area, if not, treating the connected area as a defect area; If the connected area is a tic-tac-toe area, determine whether the lengths of multiple horizontal areas in the tic-tac-toe area are the same, and whether the lengths of multiple vertical areas in the tic-tac-toe area are the same. If they are the same, there is no defect in the connected area. If the horizontal areas or the vertical areas are not the same, the connected area is regarded as a defective area.

4. The method according to any one of claims 1 to 3, characterized in that The control platform determines the photovoltaic panel with defects and the defect type according to the visible light image of the marked defect area, the infrared image and the image shooting point, specifically including: Determine the coordinates of the geographical area captured in the visible light image according to the field of view of the visible light sensor and the image capturing point, and determine the serial number of the photovoltaic panel captured in the visible light image and the pixel area corresponding to the photovoltaic panel captured according to the coordinates of the captured geographical area and the photovoltaic panel layout data; The photovoltaic panel with defects is determined according to the pixel area and defect area corresponding to the photographed photovoltaic panel; and the defect type is determined according to the pixel value of the defect area and the infrared image.

5. The method according to any one of claims 1 to 3, characterized in that: Before the drone extracts the distribution of pixels with the same pixel value in the visible light image and determines whether there is a defective area in the photovoltaic panel according to the distribution of pixels with the same pixel value in the visible light image, the method further includes: Performing geometric transformation processing on the visible light image so that the frame of the photovoltaic panel and the edge of the image are parallel to obtain a visible light image after geometric transformation; Performing denoising processing on the visible light image after the geometric transformation to obtain a denoised visible light image; Accordingly, the drone extracts the distribution of pixels with the same pixel value in the visible light image, specifically including: The drone extracts the distribution of pixels with the same pixel value in the denoised visible light image.

6. The method according to any one of claims 1 to 3, characterized in that: After the control platform determines the photovoltaic panel where the defect occurs and the defect type according to the visible light image of the marked defect area, the infrared image and the image shooting point, the method further includes: If it is determined that the number of dirty photovoltaic panels is greater than a preset number threshold, a cleaning work order for cleaning the photovoltaic panels is generated; the cleaning work order includes the number of photovoltaic panels to be cleaned; The cleaning robot to be operated is determined according to the cleaning work order, the busy or idle state of the cleaning robot and the cleaning capacity of the cleaning robot, and a cleaning instruction is issued to the cleaning robot to be operated so that the cleaning robot to be operated cleans the photovoltaic panel.

7. The method according to any one of claims 1 to 3, characterized in that Before the drone flies along the inspection route and uses the visible light sensor to collect visible light images of the photovoltaic panels and uses the infrared sensor to collect infrared images of the photovoltaic panels, the method further includes: The drone surveys the photovoltaic station and sends the survey result to the control platform; The control platform draws an electronic map of the photovoltaic station according to the survey results; and generates an inspection path according to the electronic map of the photovoltaic station, so that the drone performs terrain inspection flight according to the inspection path based on the electronic map.

8. A machine vision-based outdoor inspection device for photovoltaic stations, comprising a drone and a control platform, wherein the drone and the control platform are communicatively connected, the drone is equipped with a visible light sensor and an infrared sensor, and the drone and the control platform are used to execute the method described in any one of claims 1 to 7.

9. A control platform, characterized in that: include: processor; as well as, A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images