Photovoltaic panel defect detection method, system, device and computer-readable storage medium based on dual-light fusion

Abstract

The present invention provides a photovoltaic panel defect detection method, system, device and computer-readable storage medium based on dual-light fusion. Through the technical solution proposed in this application, it is possible to simultaneously use visible light images and thermal imaging images acquired by drones to realize high-speed automated inspection of photovoltaic panel installation areas. Specifically, the dual-light images are detected separately through an improved target detection algorithm, and the results of the defect detection algorithm based on visible light images are combined with the hot spot temperature and average temperature of the target component, so that photovoltaic panel defects can be detected in a timely manner. In the process of dual-light image fusion, the shooting deviation of the dual-light camera is overcome, and the precise positioning of the defective panel is achieved. It can automatically generate an automated customized report containing defect detection information, defect positioning information and image fusion information, which is suitable for the refined operation and maintenance needs of photovoltaic power stations and has promotional value.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and analysis, and in particular, discloses a photovoltaic panel defect detection method, system, device and computer-readable storage medium based on dual-light fusion. Background Art

[0002] With the development of the national "dual carbon" energy strategy and the need to adapt to the digital transformation of energy, photovoltaic power plants have entered a period of refined operation and maintenance, moving from a peak construction phase. Existing technologies typically rely on manual inspections to record photovoltaic power plant operating data. However, given the large footprint of photovoltaic power plants, manual inspections are not only labor-intensive but also lack the ability to guarantee data accuracy. Manual, 24 / 7 data collection and recording is tedious and inefficient, necessitating the adoption of automated methods such as drone inspections as an effective alternative.

[0003] Traditional automated drone inspections often rely solely on visible light image analysis or thermal imaging. Understandably, analyzing only visible light images is ineffective at detecting faults within the photovoltaic panels, while analyzing only thermal images fails to reveal actual obstructions, cracks, or attachments on the panels, failing to meet the demands of refined maintenance. Summary of the Invention

[0004] In order to solve the above-mentioned problems existing in the prior art, the present invention provides a photovoltaic panel defect detection method, system, device and computer-readable storage medium based on dual-light fusion.

[0005] In a first aspect of the present application, a photovoltaic panel defect detection method based on dual-light fusion is provided, which is used to detect and locate defective photovoltaic panels based on thermal imaging image data and visible light image data acquired by drone photography;

[0006] The photovoltaic panel defect detection method includes:

[0007] Acquiring thermal imaging image data and visible light image data;

[0008] Performing thermal imaging defect detection on the thermal imaging image data to obtain a corresponding first detection result, and performing visible light defect detection on the visible light image data to obtain a corresponding second detection result;

[0009] Performing image distortion correction on the thermal imaging image data and the visible light image data according to preset parameters to obtain corresponding correction parameters;

[0010] Performing a dual-light fusion operation on the thermal imaging image data and the visible light image data according to the first detection result, the second detection result, and the correction parameter to obtain defect distribution of the photovoltaic panel;

[0011] According to the defect distribution of the photovoltaic panels, the actual defect distribution position of the defective photovoltaic panels is obtained;

[0012] A defect detection report associated with the defective photovoltaic panel is generated based on the first detection result, the second detection result, and the actual defect distribution location.

[0013] In a possible implementation of the first aspect, the process of performing thermal imaging defect detection on the thermal imaging image data to obtain a corresponding first detection result further includes:

[0014] In the thermal imaging image data, the image mask of each photovoltaic panel is obtained based on the yolact algorithm;

[0015] The maximum thermal imaging area corresponding to each image mask is binarized to obtain the corresponding recognition area;

[0016] In the identification area, obtaining the minimum bounding rectangle corresponding to the photovoltaic panel, and using the minimum bounding rectangle as the detection and judgment area corresponding to the photovoltaic panel;

[0017] In the detection and judgment area, it is determined whether the photovoltaic panel has defects to generate a first detection result.

[0018] In a possible implementation of the first aspect above, the first detection result includes that the photovoltaic panel is in a normal operating condition and that an internal component of the photovoltaic panel has an abnormal temperature condition.

[0019] In a possible implementation of the first aspect, the process of performing visible light defect detection on visible light image data to obtain a corresponding second detection result further includes:

[0020] Perform image segmentation on the visible light image data to obtain a detection image that meets the pixel requirements of a preset target detection network;

[0021] Whether there are defects in the detection image is detected according to a preset target detection network to generate a second detection result.

[0022] In a possible implementation of the first aspect above, the second detection result includes that the photovoltaic panel is in a normal operating condition and that an abnormal operating condition occurs on the surface of the photovoltaic panel.

[0023] In a possible implementation of the first aspect, in the process of performing image distortion correction on the thermal imaging image data and the visible light image data according to preset parameters, the process further includes:

[0024] Obtaining initial registration parameters of the thermal imaging image data and the visible light image data based on the preset marked thermal imaging image and the marked visible light image;

[0025] Obtaining the actual distance deviation corresponding to the visible light image data based on the latitude and longitude information corresponding to the marked thermal imaging image and the marked visible light image;

[0026] According to the camera imaging principle, the actual distance deviation is mapped to the initial registration parameters to obtain the correction parameters.

[0027] In a possible implementation of the first aspect, the process of obtaining the actual defect distribution position of the defective photovoltaic panel according to the defect distribution of the photovoltaic panel further includes:

[0028] Obtain the flight attitude data of the drone during the shooting process and the actual positioning information of the drone during the shooting process;

[0029] Obtaining a mapping relationship between image pixel coordinates in the visible light image data and geographic information coordinates based on the flight attitude data and actual positioning information;

[0030] According to the first detection result and the second detection result, when the photovoltaic panel has an abnormal working condition, the geographic information coordinates corresponding to the image pixel coordinates of the photovoltaic panel are obtained, and the geographic information coordinates are used as the actual defect distribution position.

[0031] A second aspect of the present application provides a photovoltaic panel defect detection system based on dual-light fusion, which is applied to the photovoltaic panel defect detection method based on dual-light fusion provided in the first aspect, and is used to detect and locate defective photovoltaic panels based on thermal imaging image data and visible light image data captured by drones;

[0032] The photovoltaic panel defect detection system includes:

[0033] an acquisition unit, configured to acquire thermal imaging image data and visible light image data;

[0034] a detection unit, configured to perform thermal imaging defect detection on the thermal imaging image data to obtain a corresponding first detection result, and to perform visible light defect detection on the visible light image to obtain a corresponding second detection result;

[0035] a calibration unit, configured to perform image distortion correction on the thermal imaging image data and the visible light image data according to preset parameters to obtain a corrected first thermal imaging image and a first visible light image;

[0036] a dual-light fusion unit, configured to perform a dual-light fusion operation based on the first detection result, the second detection result, the first thermal imaging image, and the first visible light image to obtain defect distribution of the photovoltaic panel;

[0037] A positioning unit, configured to obtain an actual defect distribution position of a defective photovoltaic panel based on the defect distribution of the photovoltaic panel;

[0038] The detection generating unit is configured to generate a defect detection report associated with the photovoltaic panel having defects based on the first detection result, the second detection result, and the actual defect distribution location.

[0039] The third aspect of the present application provides a photovoltaic panel defect detection device based on dual-light fusion, comprising:

[0040] Memory for storing computer programs;

[0041] A processor is used to implement the photovoltaic panel defect detection method based on dual-light fusion provided by the first aspect when executing a computer program.

[0042] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the photovoltaic panel defect detection method based on dual-light fusion provided in the first aspect.

[0043] Compared with the prior art, this application has the following beneficial effects:

[0044] Through the technical solution proposed in this application, it is possible to simultaneously use visible light images and thermal imaging images acquired by drones to realize high-speed automated inspection of photovoltaic panel installation areas. Specifically, the dual-light images are detected separately through an improved target detection algorithm, and the results of the defect detection algorithm based on visible light images are combined with the hot spot temperature of the target component and the average temperature of the target string, so that most photovoltaic panel defects can be detected in a timely manner. In the process of dual-light image fusion, the shooting deviation of the dual-light camera is overcome, so that the targets to be detected in the multi-source images can be mapped to each other through coordinate transformation, and at the same time, the precise positioning of the defective panels is achieved. It can automatically generate an automated customized report containing defect detection information, defect positioning information and image fusion information, which is suitable for the refined operation and maintenance needs of photovoltaic power stations and has promotional value. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0046] Figure 1 According to an embodiment of the present application, a schematic flow chart of a photovoltaic panel defect detection method based on dual-light fusion is shown;

[0047] Figure 2 According to an embodiment of the present application, a schematic diagram of a process for performing image distortion correction on thermal imaging image data and visible light image data according to preset parameters is shown;

[0048] Figure 3 According to an embodiment of the present application, a schematic diagram of a process for obtaining the actual defect distribution position of a defective photovoltaic panel based on the defect distribution of the photovoltaic panel is shown;

[0049] Figure 4 According to an embodiment of the present application, an abstract schematic diagram of a drone photography process is shown;

[0050] Figure 5 According to an embodiment of the present application, a schematic structural diagram of a photovoltaic panel defect detection system based on dual-light fusion is shown;

[0051] Figure 6 According to an embodiment of the present application, a schematic structural diagram of a photovoltaic panel defect detection device based on dual-light fusion is shown;

[0052] Figure 7 According to an embodiment of the present application, a structural schematic diagram of a computer-readable storage medium is shown. DETAILED DESCRIPTION

[0053] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0054] As used herein, the term "including" and its variations represent open inclusion, i.e., "including but not limited to." Unless otherwise stated, the term "or" means "and / or." The term "based on" means "based at least in part on." The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment." The term "another embodiment" means "at least one additional embodiment." The terms "first," "second," etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0055] To address the practical issues of high manual inspection costs and low efficiency, as well as the relatively limited drone inspection methods, existing in the existing technology, this application provides a photovoltaic panel defect detection method, system, device, and computer-readable storage medium based on dual-light fusion. The technical solution provided by this application enables a comprehensive analysis of the impact of policy issuance on industrial development over a certain period of time, using both policy impact scores and sentiment scores after policy issuance. The technical solution provided by this application will be explained and illustrated in conjunction with examples.

[0056] In some embodiments of the present application, Figure 1A photovoltaic panel defect detection method based on dual-light fusion is described, which is used to detect and locate defective photovoltaic panels based on thermal imaging and visible light image data captured by a drone. In the above embodiment, dual-light image acquisition can be performed using a quadrotor drone equipped with RTK equipment and a dual-light camera, but this is not limited here.

[0057] like Figure 1 As shown, the photovoltaic panel defect detection method may include:

[0058] Step 101: Acquire thermal imaging image data and visible light image data. The thermal imaging image data and visible light image data can be acquired by a dual-light camera carried by a drone, which is not limited here.

[0059] Step 102: Perform thermal imaging defect detection on the thermal imaging image data to obtain a corresponding first detection result, and perform visible light defect detection on the visible light image data to obtain a corresponding second detection result. The specific implementation of thermal imaging defect detection and visible light defect detection will be described in detail later.

[0060] Step 103: Perform image distortion correction on the thermal imaging image data and the visible light image data according to preset parameters to obtain corresponding correction parameters. The specific implementation of image distortion correction will be described in detail later.

[0061] Step 104: performing a dual-light fusion operation on the thermal imaging image data and the visible light image data according to the first detection result, the second detection result, and the correction parameter to obtain the defect distribution of the photovoltaic panel.

[0062] Step 105: According to the defect distribution of the photovoltaic panel, the actual defect distribution position of the defective photovoltaic panel is obtained. The specific implementation method of locating the actual defect distribution position will be described in detail later.

[0063] Step 106: Generate a defect detection report for the defective photovoltaic panel based on the first and second detection results and the actual defect distribution locations. It is understood that the defect detection report can be automatically generated based on dual-light defect detection information, image fusion information, and defective panel ID location information. Users can also customize the presentation of the defect detection report based on their actual needs, which is not limited here.

[0064] It can be understood that through the above steps 101 to 106, accurate detection and positioning of photovoltaic panel defects can be achieved. The specific implementation process of the above steps 101 to 106 will be further explained and illustrated below.

[0065] In some embodiments of the present application, during the specific implementation of the aforementioned step 102, the process of performing thermal imaging defect detection on the thermal imaging image data to obtain the corresponding first detection result may further include the following steps:

[0066] Step 102a: Obtain an image mask of each photovoltaic panel based on the yolact algorithm in the thermal imaging image data.

[0067] It is understandable that since drones need to take photos while flying during the inspection process, they will inevitably capture photovoltaic panel strings or components in an oblique direction. These targets have multiple angles of arbitrary rotation and are densely arranged. Traditional target detection methods often use a method of marking with a rectangular frame parallel to the edge of the image, which easily leads to the addition of redundant background within the target detection frame, making the feature expression information redundant. The superposition of such target frames has a great impact on accuracy and affects generalization ability. In order to overcome the above problems, the specific implementation of the above embodiment can use an instance segmentation method to detect strings and faulty components.

[0068] Specifically, in the above step 102a, the yolact algorithm can be used in the thermal imaging defect detection process to add a mask branch to the existing one-stage yolo target detection model, and split the instance segmentation task into two parallel subtasks: the first parallel subtask generates several prototype masks for each image through a Protonet network; the second parallel subtask predicts several linear combination coefficients for each instance, and finally generates an instance mask through linear combination, that is, the above-mentioned image mask.

[0069] Step 102b: performing binarization processing on the maximum thermal imaging area corresponding to each image mask to obtain the corresponding recognition area.

[0070] Step 102c: Obtain a minimum bounding rectangle corresponding to the photovoltaic panel in the identification area, and use the minimum bounding rectangle as the detection and judgment area corresponding to the photovoltaic panel.

[0071] It can be understood that through the above steps 102b to 102c, the feature expression of irregular rectangular frames in target detection can be achieved, thereby achieving effective identification and acquisition of the corresponding photovoltaic panel areas in the drone thermal imaging image without excessive redundant information.

[0072] Step 102d: Determine whether the photovoltaic panel has defects in the detection and judgment area to generate a first detection result. The first detection result may include whether the photovoltaic panel is in a normal working condition or whether the temperature of the photovoltaic panel internal components is abnormal.

[0073] It is understood that during defect detection using thermal imaging images, defective areas can be detected based on the distribution of hot spots and the average temperature of the photovoltaic panel group. If a photovoltaic panel appears significantly darker than other photovoltaic panels in the thermal image or a dark hot spot appears, this indicates that the photovoltaic panel has a potential defect based on the thermal imaging image analysis. Those skilled in the art may also use other analysis methods to determine whether a defect exists, which is not limited here.

[0074] In some embodiments of the present application, during the specific implementation of the aforementioned step 102, the process of performing visible light defect detection on the visible light image data to obtain the corresponding second detection result may further include the following steps:

[0075] Step 102A: Perform image segmentation on the visible light image data to obtain a detection image that meets the pixel requirements of a preset target detection network.

[0076] It is understandable that the visible light images taken by drones are often high-resolution, and a single photovoltaic panel only occupies a small part of the pixel area in the large-size image. If the entire visible light image is directly input into the target detection network, it is easy for the network to learn the feature information of the target. For example, in a specific embodiment of the present application, the visible light image from the drone's perspective is an image with a resolution of 5184*3880. If it is directly input into a target detection network, such as the network of the YOLO target detection series, taking the image of the network input 608*608 as an example, the YOLO target detection series uses 5 downsamplings, so the final feature map sizes are 19*19, 38*38, and 76*76 pixels respectively, where the largest feature map is responsible for detecting small targets, and corresponds to the network input size 608*608, and then 608*608 is corresponding to a visible light image with a resolution of 5184*3880. Taking the longest side as an example, 5184 / 608*8=68, that is, if the width or height of the target in the original image is less than 68 pixels, it is difficult for the network to learn the feature information of the target.

[0077] To overcome this problem, the image can be segmented before input into the model through step 102A. The segmented images are then fed into the object detection network, and each image is then detected. This significantly reduces the minimum target pixel limit. For example, the image in the above embodiment can be segmented into 608*608 images and fed into the 608*608 network. This will allow the features of small objects larger than 8 pixels in the original image to be learned.

[0078] Step 102B: Detect whether there are defects in the detection image using a preset target detection network to generate a second detection result, wherein the second detection result may include whether the photovoltaic panel is in a normal working condition or an abnormal working condition occurs on the surface of the photovoltaic panel.

[0079] It is understandable that those skilled in the art can select a suitable target detection algorithm to detect whether there is any abnormality on the surface of the photovoltaic panel according to actual conditions, and no limitation is made here.

[0080] Based on the explanation of the aforementioned embodiments, the photovoltaic panel defect detection method provided in this application is optimized for both visible light and thermal imaging. Since visible light and thermal imaging images observe different types of defects, thermal imaging can, on the one hand, detect hot spot effects caused by abnormal temperatures inside photovoltaic modules. On the other hand, visible light images capture more complex image information, making it easier to observe specific visible causes of defects, such as cracks, dirt, and obstructions caused by leaves, weeds, and other foreign objects in the photovoltaic panel. Therefore, it is necessary to fuse the defect information detected in the two images.

[0081] Understandably, due to differences in imaging principles, camera parameters, exposure time, and image size between visible light and thermal cameras, the size and field of view of visible light and thermal images differ significantly. Visible light images are approximately 5184 x 3880 pixels, while thermal images are approximately 640 x 512 pixels. This difference in imaging mechanisms leads to different exposure times, which are further magnified by the faster flight. Consequently, in two images taken simultaneously, the visible light image may not fully encompass the thermal image, resulting in objects in the thermal image being invisible in the visible light image.

[0082] In order to overcome the above-mentioned defects of dual-light fusion, in some embodiments of the present application, Figure 2 A schematic diagram of a process for performing image distortion correction on thermal imaging image data and visible light image data according to preset parameters is shown, which may specifically include:

[0083] Step 201: Acquire initial registration parameters of the thermal imaging image data and the visible light image data according to the preset marked thermal imaging image and the marked visible light image.

[0084] It is understood that in the above embodiment, when initializing the drone, a set of visible light and thermal images can be collected as the preset labeled thermal image and the labeled visible light image, respectively, denoted as Figure A and Figure B. Three pairs of corresponding points are manually found in Figures A and B, and the parameter matrix of the transformation relationship between the two images is obtained through a radial transformation. A radial transformation is the process of converting one two-dimensional coordinate system to another. The relative positions and attributes of the coordinate points do not change during the transformation. As a linear transformation, the process only involves rotation and translation.

[0085] In the above embodiment, the radiation variation matrix may be:

[0086]

[0087] Among them, (x, y) is the original coordinate, (x', y') is the transformed coordinate. In the final parameter matrix, , Indicates the stretching of the image; , Indicates the rotation of the image; Indicates the offset in the x-axis direction. Indicates the offset in the y-axis direction.

[0088] It is understandable that simply using the parameters obtained by radial changes to solve the corresponding coordinates is biased and has low adaptability. In order to reduce errors and enhance adaptability, the optimal parameters can be solved based on the clustering state of the selected points, and the parameters obtained by the radial transformation can be corrected. Since the matrix point pairs after transformation show a certain degree of clustering, by observing the distribution of the matrix parameters, it is believed that the more clustered the place, the greater the possibility of the optimal solution. In the above embodiment, the cluster center can be solved by calculation, and the optimal transformation matrix can be obtained by the square method. At this point, the initial alignment of the drone is achieved through two preset images, and the alignment parameter matrix is obtained. The subsequent images are aligned through the alignment parameter matrix to obtain the coordinate mapping estimated in the visible light image of the thermal imaging target.

[0089] It is understandable that due to the different exposure times of the visible light camera and thermal imaging camera in the dual-light camera, as well as the uncertainty of automatic exposure, there is a deviation between the estimated target position and the actual position. Therefore, for the same drone, there are differences between the registration parameters of different groups of photos. This difference mainly exists in the x and y axes of the pixel coordinate system, that is, , , , Almost unchanged, , An offset occurs.

[0090] In order to overcome the above problem, the following steps 202 to 203 are used to correct the UAV registration parameter matrix:

[0091] Step 202: Obtain the actual distance deviation corresponding to the visible light image data according to the latitude and longitude information corresponding to the marked thermal imaging image and the marked visible light image.

[0092] Step 203: According to the camera imaging principle, the actual distance deviation is mapped to the initial registration parameters to obtain correction parameters.

[0093] In steps 202 and 203 above, the latitude and longitude information of the initial two preset marker images (visible light image A and infrared thermal image B) are first recorded. The spherical coordinates of the longitude and latitude are converted into plane coordinates, recorded as (Lat, Lng), (Lat2, Lng2), respectively. Lat and Lat2 are converted from longitude, and Lng and Lng2 are converted from latitude. A set of infrared thermal and visible light images are randomly selected, and their changed longitude and latitude plane coordinates are recorded as (Lat1, Lng1) and (Lat3, Lng3). Using the longitude and latitude changes of the two thermal images as a reference, the actual distance deviation produced by the visible light image is obtained and recorded as:

[0094]

[0095]

[0096] in, is the actual distance deviation produced on the longitude of the visible light image, is the actual distance deviation generated in the visible light image dimension.

[0097] In this embodiment, based on the actual distance deviation in the visible light image, this deviation is mapped to the pixel coordinate system. This technical solution achieves precise mapping through the camera imaging principle, which requires the application of intrinsic and extrinsic parameters of camera calibration, which are not limited here. Thus, a point in the world coordinate system can be mapped to the pixel coordinate system. That is, the actual distance deviation generated in the world coordinate system can be mapped to the pixel coordinate system, completing the correction of pixel deviation.

[0098] In some embodiments of the present application, Figure 3 A schematic diagram of a process for obtaining the actual defect distribution position of a defective photovoltaic panel based on the defect distribution of the photovoltaic panel is shown, which may specifically include:

[0099] Step 301: Acquire the flight attitude data of the drone during the shooting process and the actual positioning information of the drone during the shooting process.

[0100] Step 302: According to the flight attitude data and the actual positioning information, a mapping relationship between the image pixel coordinates in the visible light image data and the geographic information coordinates is obtained.

[0101] Step 303: Based on the first detection result and the second detection result, when the photovoltaic panel has an abnormal working condition, obtain the geographic information coordinates corresponding to the image pixel coordinates of the photovoltaic panel, and use the geographic information coordinates as the actual defect distribution location.

[0102] In the implementation of the above embodiment, the drone first captures an orthophoto with RTK positioning enabled. The PV panels are then vectorized on the orthophoto to establish a mapping relationship between the PV panel identification data and geographic coordinates. The PV panel detection results return the pixel coordinates of the corresponding component. These coordinates are then projected onto map coordinates based on the camera coordinates and the drone's posture. The corresponding PV panel is then searched for using these map coordinates.

[0103] Specifically, Figure 4 An abstract schematic diagram of the drone shooting process is shown. Figure 4 As shown in the figure, for a camera on a drone with a center point (u0, v0), a focal length of f (units uniformly in millimeters), and a pixel size of p (units uniformly in millimeters), when it is located at the geodetic coordinates (X0, Y0), height H, yaw angle α, pitch angle β, and roll angle γ, the pixel coordinates of the polygonal area points captured by it are (uk, vk), where k is a non-zero natural number. Then, for one point (u, v), its actual geodetic coordinates are calculated as follows:

[0104] Taking the center of the photo as the origin, the camera vector (dx, dy, f) is:

[0105]

[0106] The unit vector of this vector for:

[0107]

[0108] The rotated vector The solution is:

[0109]

[0110] The attitude angle parameters in the rotation vector are α yaw angle, β pitch angle, and γ roll angle, then The cosine of the angle between the ground normal and the ground normal can be , and then get the pixel from the camera center point O The vector to its ground projection point A is:

[0111]

[0112] The vector from point A to the camera's vertical projection point B on the ground ,Right now:

[0113]

[0114]

[0115]

[0116] From this, we can get the geodetic coordinates X, Y of point A, and then we can get the polygon points of the geodetic coordinates, whose area S is:

[0117]

[0118] The geographic information mapping that can be achieved according to the above method can present the actual defect distribution position corresponding to the faulty panel in the visible light image and obtain the corresponding actual geographic coordinates.

[0119] In some embodiments of the present application, Figure 5 A photovoltaic panel defect detection system based on dual-light fusion is shown, which is applied to the photovoltaic panel defect detection method based on dual-light fusion provided in the previous embodiment. This photovoltaic panel defect detection system is also used to detect and locate defective photovoltaic panels based on thermal imaging data and visible light image data captured by drones.

[0120] like Figure 5 As shown, the photovoltaic panel defect detection system may include:

[0121] The acquisition unit 001 is used to acquire thermal imaging image data and visible light image data.

[0122] The detection unit 002 is configured to perform thermal imaging defect detection on the thermal imaging image data to obtain a corresponding first detection result, and to perform visible light defect detection on the visible light image to obtain a corresponding second detection result.

[0123] The calibration unit 003 is configured to perform image distortion correction on the thermal imaging image data and the visible light image data according to preset parameters to obtain a corrected first thermal imaging image and a first visible light image.

[0124] The dual-light fusion unit 004 is used to perform a dual-light fusion operation based on the first detection result, the second detection result, the first thermal imaging image and the first visible light image to obtain the defect distribution of the photovoltaic panel.

[0125] The positioning unit 005 is used to obtain the actual defect distribution position of the defective photovoltaic panel according to the defect distribution of the photovoltaic panel.

[0126] The detection generation unit 006 is used to generate a defect detection report associated with the defective photovoltaic panel based on the first detection result, the second detection result and the actual defect distribution location.

[0127] It can be understood that the functions performed by the above-mentioned acquisition unit 001 to the detection generation unit 006 are consistent with the specific implementation contents of the above-mentioned steps 101 to 106, and are not repeated here.

[0128] In some embodiments of the present application, a photovoltaic panel defect detection device based on dual-light fusion is also provided, which may include:

[0129] Memory for storing computer programs;

[0130] A processor is used to implement the steps of the photovoltaic panel defect detection method described in the technical solution of this application when executing a computer program.

[0131] It is understandable that the photovoltaic panel defect detection device based on dual-light fusion can be a microcomputer integrated with memory and processor, and is applied in the aforementioned embodiment by being mounted on a drone equipped with RTK equipment, which is not limited here.

[0132] It is understood that various aspects of the technical solution of the present application can be implemented as a system, method, or program product. Therefore, various aspects of the technical solution of the present application can be specifically implemented in the following forms, namely, a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, which can be collectively referred to as "circuit", "module" or "platform" herein.

[0133] Figure 6 According to some embodiments of the present application, a schematic diagram of the structure of a photovoltaic panel defect detection device based on dual light fusion is shown. Figure 6 The electronic device 600 implemented according to the implementation in this embodiment is described in detail. Figure 6 The electronic device 600 shown is merely an example and should not limit the functions and scope of use of any embodiment of the technical solution of the present application.

[0134] like Figure 6 As shown, electronic device 600 is implemented as a general-purpose computing device. The components of electronic device 600 may include, but are not limited to, at least one processing unit 610, at least one storage unit 620, a bus 630 connecting various platform components (including storage unit 620 and processing unit 610), and a display unit 640.

[0135] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the implementation steps described in the image stitching method area in the embodiment. For example, the processing unit 610 can execute the following steps: Figures 1 to 5 Follow the steps shown in .

[0136] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access unit (RAM) 6201 and / or a cache storage unit 6202 , and may further include a read-only storage unit (ROM) 6203 .

[0137] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0138] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0139] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 through the bus 630. It should be understood that although Figure 6 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0140] In some embodiments of the present application, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the relevant steps of the photovoltaic panel defect detection method based on dual-light fusion provided in the above embodiments can be implemented.

[0141] Although this embodiment does not list other specific implementation methods in detail, in some possible implementation methods, the various aspects described in the technical solution of this application can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps described in the image stitching method area of the technical solution of this application according to the implementation methods in various embodiments of the technical solution of this application.

[0142] Figure 7 According to some embodiments of the present application, a schematic diagram of the structure of a computer-readable storage medium is shown. Figure 7 , which describes a program product 800 for implementing the above method according to an embodiment of the technical solution of the present application. The program product 800 may be a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. Of course, the program product produced according to this embodiment is not limited thereto. In the technical solution of the present application, the readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0143] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0144] Computer-readable storage media may include a data signal propagated in baseband or as a carrier wave region, wherein readable program code is carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0145] The program code for performing the operations of the technical solutions of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as C or similar programming languages. The program code can be executed entirely on the user computing device, locally on the user device, as a separate software package, locally on the user computing device and locally on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0146] To sum up, through the technical solution proposed in this application, it is possible to use computer programs to realize batch collection, analysis and evaluation of policy information issued by government units, automatically obtain the degree of support for the industry to which each issued policy belongs, save a lot of manpower and time costs, and objectively quantify and conveniently obtain the support or pressure of each issued policy on the industry. At the same time, it is convenient for enterprises or financial support institutions to adjust their subsequent development layout in related industries in a timely manner, and has promotional value.

[0147] The above description is only a description of the preferred embodiment of the technical solution of this application, and does not limit the scope of the technical solution of this application. Any changes and modifications made by ordinary technicians in the field of the technical solution of this application based on the above disclosure shall fall within the scope of protection of the claims.

Claims

1. A photovoltaic panel defect detection method based on dual-light fusion, characterized in that: Used to detect and locate defective photovoltaic panels based on thermal imaging image data and visible light image data obtained by drone photography; The photovoltaic panel defect detection method comprises: acquiring the thermal imaging image data and the visible light image data; performing a thermal imaging defect detection on the thermal imaging image data to obtain a corresponding first detection result, and performing a visible light defect detection on the visible light image data to obtain a corresponding second detection result; performing image distortion correction on the thermal imaging image data and the visible light image data according to preset parameters to obtain corresponding correction parameters; performing a dual-light fusion operation on the thermal imaging image data and the visible light image data according to the first detection result, the second detection result, and the correction parameter to obtain defect distribution of the photovoltaic panel; According to the defect distribution of the photovoltaic panel, an actual defect distribution position of the photovoltaic panel with defects is obtained; generating a defect detection report associated with the photovoltaic panel having defects based on the first detection result, the second detection result, and the actual defect distribution location; The process of performing image distortion correction on the thermal imaging image data and the visible light image data according to preset parameters further includes: According to the preset marked thermal imaging image and the marked visible light image, obtaining initial registration parameters of the thermal imaging image data and the visible light image data; Obtaining an actual distance deviation corresponding to the visible light image data according to the latitude and longitude information corresponding to the marked thermal imaging image and the marked visible light image; According to the camera imaging principle, the actual distance deviation is mapped to the initial registration parameter to obtain the correction parameter; wherein, according to the latitude and longitude information corresponding to the marked thermal imaging image and the marked visible light image, an actual distance deviation corresponding to the visible light image data is obtained; Record the latitude and longitude information of the marked visible light image and the marked thermal image, convert the longitude and latitude spherical coordinates into plane coordinates, recorded as (Lat, Lng), (Lat2, Lng2), respectively. Record a set of infrared thermal images and visible light images, and record the changed longitude and latitude plane coordinates as (Lat1, Lng1) and (Lat3, Lng3). Based on the longitude and latitude changes of the two thermal images, obtain the actual distance deviation generated by the visible light image, recorded as: in, is the actual distance deviation produced on the longitude of the visible light image, is the actual distance deviation produced in the latitude of the visible light image.

2. The photovoltaic panel defect detection method based on dual-light fusion according to claim 1, characterized in that: The process of performing thermal imaging defect detection on the thermal imaging image data to obtain a corresponding first detection result further includes: Acquiring an image mask of each photovoltaic panel based on a yolact algorithm from the thermal imaging image data; performing binarization processing on the maximum thermal imaging area corresponding to each of the image masks to obtain a corresponding recognition area; In the identification area, obtaining a minimum bounding rectangle corresponding to the photovoltaic panel, and using the minimum bounding rectangle as a detection and judgment area corresponding to the photovoltaic panel; In the detection and judgment area, it is determined whether the photovoltaic panel has defects to generate the first detection result.

3. The photovoltaic panel defect detection method based on dual-light fusion according to claim 1 or 2, characterized in that: The first detection result includes that the photovoltaic panel is in a normal working condition and that the internal components of the photovoltaic panel have an abnormal temperature working condition.

4. The photovoltaic panel defect detection method based on dual-light fusion according to claim 1, characterized in that: The process of performing visible light defect detection on the visible light image data to obtain a corresponding second detection result further includes: Performing image segmentation on the visible light image data to obtain a detection image that meets pixel requirements of a preset target detection network; Whether there are defects in the detection image is detected according to the preset target detection network to generate the second detection result.

5. The photovoltaic panel defect detection method based on dual-light fusion according to claim 1 or 4, characterized in that: The second detection result includes whether the photovoltaic panel is in a normal working condition and whether an abnormal working condition occurs on the surface of the photovoltaic panel.

6. The photovoltaic panel defect detection method based on dual-light fusion according to claim 1, characterized in that: The process of obtaining the actual defect distribution position of the photovoltaic panel having defects according to the defect distribution of the photovoltaic panel further includes: Acquiring flight attitude data of the drone during the filming process and actual positioning information of the drone during the filming process; Acquire a mapping relationship between image pixel coordinates in the visible light image data and geographic information coordinates according to the flight attitude data and the actual positioning information; According to the first detection result and the second detection result, when the photovoltaic panel has an abnormal working condition, the geographic information coordinates corresponding to the image pixel coordinates of the photovoltaic panel are obtained, and the geographic information coordinates are used as the actual defect distribution position.

7. A photovoltaic panel defect detection system based on dual-light fusion, characterized in that: Applied to the photovoltaic panel defect detection method based on dual-light fusion as described in any one of claims 1 to 6, for detecting and locating defective photovoltaic panels based on thermal imaging image data and visible light image data acquired by drone photography; The photovoltaic panel defect detection system includes: an acquisition unit, configured to acquire the thermal imaging image data and the visible light image data; a detection unit, configured to perform a thermal imaging defect detection on the thermal imaging image data to obtain a corresponding first detection result, and to perform a visible light defect detection on the visible light image to obtain a corresponding second detection result; a calibration unit, configured to perform image distortion correction on the thermal imaging image data and the visible light image data according to preset parameters to obtain a corrected first thermal imaging image and a first visible light image; a dual-light fusion unit, configured to perform a dual-light fusion operation based on the first detection result, the second detection result, the first thermal imaging image, and the first visible light image to obtain defect distribution of the photovoltaic panel; a positioning unit, configured to obtain an actual defect distribution position of the photovoltaic panel having defects according to the defect distribution of the photovoltaic panel; A detection generating unit is configured to generate a defect detection report associated with the photovoltaic panel having defects based on the first detection result, the second detection result, and the actual defect distribution location.

8. A photovoltaic panel defect detection device based on dual-light fusion, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the photovoltaic panel defect detection method based on dual-light fusion as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the photovoltaic panel defect detection method based on dual-light fusion according to any one of claims 1 to 6 is implemented.

Images