Photovoltaic unit defect positioning method, device, storage medium and electronic equipment

Abstract

The present invention provides a photovoltaic unit defect location method, device, storage medium, and electronic device. The method obtains infrared and visible light images of all photovoltaic units according to a preset drone route, and uses the infrared images to obtain photovoltaic defects, thereby obtaining a visible light image mapped with the photovoltaic defects to obtain a target image. This improves the efficiency of photovoltaic unit defect inspections and effectively reduces the number of manual inspections. At the same time, by splicing the target image mapped with photovoltaic defects with other visible light images without photovoltaic defects taken by the drone to obtain a panoramic image, the panoramic image is globally marked. This allows the drone to accurately locate photovoltaic defects in the photovoltaic field area when they are detected, solving the problem of traditional drone inspection positioning being limited by the drone's field of view, and greatly improving the efficiency of photovoltaic unit operation and maintenance.

Description

Technical Field

[0001] The present invention relates to the technical field of photovoltaic unit maintenance, and in particular to a photovoltaic unit defect locating method, device, storage medium and electronic equipment. Background Art

[0002] Traditional photovoltaic panel inspections rely primarily on manual inspections, which are extremely inefficient and prone to delays in detecting defects due to their complexity. Furthermore, photovoltaic power plants are often built in complex terrain, such as on water or mountains, and their scale presents significant challenges for manual inspections. To address these shortcomings of traditional inspections, some methods have emerged that utilize drones for photovoltaic panel inspections. These drones use infrared and color cameras to collect photovoltaic data and determine if a panel is defective. These methods rely on the distinct bright spots that appear on infrared images of different photovoltaic defects. However, the infrared image cannot be taken too close or too far from the panel. If too far, the infrared image will not show the defective bright spot; if too close, the photovoltaic area cannot be identified, making it difficult to locate the defect. Furthermore, after detecting a photovoltaic defect in a single image, it is necessary to pinpoint the specific photovoltaic panel within the field. Currently, no effective technical solution exists to address this issue. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a photovoltaic unit defect positioning method, device, storage medium and electronic equipment to improve the inspection efficiency of photovoltaic panels while realizing the global positioning of photovoltaic unit defects in the photovoltaic field area.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0005] A photovoltaic unit defect locating method, comprising:

[0006] Obtain infrared and visible light images of all photovoltaic units along the drone's preset route;

[0007] Acquire a photovoltaic defect through the infrared light image, and acquire a visible light image mapped with the photovoltaic defect as a target image;

[0008] Performing panoramic stitching on the target image and the visible light image without photovoltaic defects mapped therein to obtain a panoramic image;

[0009] The panoramic image is globally marked according to the preset route of the drone, and the photovoltaic defect is located according to the global position mark.

[0010] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0011] A photovoltaic unit defect locating device, comprising:

[0012] An acquisition module is used to acquire infrared and visible light images of all photovoltaic units along the preset route of the drone;

[0013] a mapping module, configured to acquire photovoltaic defects through the infrared light image, and acquire a visible light image mapped with the photovoltaic defects as a target image;

[0014] a stitching module, configured to perform panoramic stitching on the target image and the visible light image without photovoltaic defects mapped therein to obtain a panoramic image;

[0015] A positioning module is used to mark the global position of the panoramic image according to the preset route of the drone, and to locate the photovoltaic defect according to the global position mark.

[0016] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0017] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the various steps of the above-mentioned photovoltaic unit defect locating method.

[0018] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0019] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, each step of the above-mentioned photovoltaic unit defect location method is implemented.

[0020] The beneficial effects of the present invention are as follows: infrared and visible light images of all photovoltaic units are acquired according to the preset route of the drone, and photovoltaic defects are acquired through the infrared images, thereby acquiring visible light images with the photovoltaic defects mapped thereon, and obtaining target images, thereby improving the efficiency of photovoltaic unit defect inspections and effectively reducing the number of manual inspections. At the same time, after obtaining a panoramic image by splicing the target image with the photovoltaic defects mapped thereon with other visible light images without photovoltaic defects taken by the drone, the panoramic image is globally marked, so that when the drone detects photovoltaic defects, it can accurately locate the photovoltaic defects in the photovoltaic field area, solving the problem that traditional drone inspection positioning is limited by the drone's field of view, and greatly improving the efficiency of photovoltaic unit operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flowchart of the steps of a photovoltaic unit defect location method disclosed in the present invention;

[0022] Figure 2 A comparison chart of the calibrated photovoltaic defect image and the model detection and recognition results provided by the present invention;

[0023] Figure 3 A schematic diagram of a photovoltaic defect after registration of the infrared image and the visible light image provided by the present invention;

[0024] Figure 4 A schematic diagram of the manifold splicing method provided by the present invention;

[0025] Figure 5 A flow chart of the panoramic image stitching process provided by the present invention;

[0026] Figure 6 A schematic diagram of a panoramic image with a marked route provided by the present invention;

[0027] Figure 7 A schematic diagram of a global position marker provided by the present invention;

[0028] Figure 8 This is a schematic structural diagram of a photovoltaic unit defect locating device disclosed in the present invention;

[0029] Figure 9 This is a schematic structural diagram of an electronic device disclosed in the present invention;

[0030] Description of labels:

[0031] 101. Acquisition module; 102. Mapping module; 103. Stitching module; 104. Positioning module; 201. Memory; 202. Processor. DETAILED DESCRIPTION

[0032] To illustrate the technical content, achieved objectives and effects of the present invention in detail, the following description is given in conjunction with the embodiments and accompanying drawings.

[0033] Please refer to Figure 1 , an embodiment of the present invention provides a photovoltaic unit defect location method, comprising:

[0034] Obtain infrared and visible light images of all photovoltaic units along the drone's preset route;

[0035] Acquire a photovoltaic defect through the infrared light image, and acquire a visible light image mapped with the photovoltaic defect as a target image;

[0036] Performing panoramic stitching on the target image and the visible light image without photovoltaic defects mapped therein to obtain a panoramic image;

[0037] The panoramic image is globally marked according to the preset route of the drone, and the photovoltaic defect is located according to the global position mark.

[0038] As can be seen from the above description, the beneficial effects of the present invention are: infrared and visible light images of all photovoltaic units are obtained according to the preset route of the drone, and visible light images with photovoltaic defects mapped are obtained through infrared images to obtain target images, thereby improving the efficiency of photovoltaic unit defect inspections and effectively reducing the number of manual inspections. At the same time, by panoramically stitching the target image with photovoltaic defects and other visible light images without photovoltaic defects taken by the drone, after obtaining a panoramic image, the panoramic image is globally marked, so that when the drone detects photovoltaic defects, it can accurately locate the photovoltaic defects in the photovoltaic field area, solving the problem that traditional drone inspection positioning is limited by the drone's field of view, and greatly improving the operation and maintenance efficiency of photovoltaic units.

[0039] Furthermore, the panorama stitching of the target image and the visible light image without the photovoltaic defect mapped therein to obtain the panoramic image includes:

[0040] Eliminating the target image and the photovoltaic unit area in the visible light image that is not mapped with photovoltaic defects to obtain a corresponding background image;

[0041] Selecting reference background images in sequence according to the preset route of the drone, and extracting feature points of the reference background image and adjacent background images respectively; the adjacent background images are background images that are adjacent to the reference background image in the order of shooting by the drone;

[0042] Matching feature points of the reference background image with adjacent background images, and calculating an offset of the adjacent background images based on the matching results of the feature points;

[0043] The target image and the visible light image without photovoltaic defects mapped therein are stitched together using a manifold stitching method according to the offset to obtain a panoramic image.

[0044] As can be seen from the above description, by panoramically stitching the target image with the photovoltaic defect mapped and the visible light image without the photovoltaic defect, the position of the photovoltaic defect within the entire photovoltaic field can be determined. In other words, the visible light image without the photovoltaic defect provides a reference position, achieving global positioning of the photovoltaic defect. At the same time, by removing the photovoltaic cell area from the image and retaining only the background area, the interference factors caused by the high similarity of the photovoltaic cells in the image are avoided. This improves the accuracy of the subsequent feature point matching between the reference background image and the adjacent background images, thereby improving the accuracy of the offset and ensuring the accuracy of the panoramic stitching.

[0045] Furthermore, the calculation of the offset of the adjacent background image according to the matching result of the feature points is specifically as follows:

[0046] Determining whether the number of feature point matches is greater than or equal to a preset number, and if so, calculating a homography matrix of the adjacent background image, and calculating an offset of the adjacent background image according to the homography matrix;

[0047] Otherwise, obtain the offset and longitude and latitude differences of other adjacent background images, and calculate the longitude and latitude differences of the adjacent background images, and calculate the offset of the adjacent background image based on the offset and longitude and latitude differences of the other adjacent background images and the longitude and latitude differences of the adjacent background images.

[0048] As can be seen from the above description, when the number of feature point matches is greater than or equal to the preset number, it indicates a high degree of match between the reference background image and the adjacent background image, and the corresponding offset can be directly calculated using the homography matrix. When the number of feature point matches is less than the preset number, it indicates a low degree of match between the two. In this case, the offset of the current adjacent background image can be inferred from other adjacent background images with known offsets, thereby ensuring the accuracy of the offset calculation and avoiding the accumulation of matching errors.

[0049] Furthermore, the target image and the visible light image without photovoltaic defects mapped therein are stitched together using a manifold stitching method according to the offset to obtain a panoramic image:

[0050] Calculating a global starting point of each of the target images and each of the visible light images without photovoltaic defects mapped therein according to the offset;

[0051] According to the global starting point, each of the target images and each of the visible light images without photovoltaic defects mapped therein are manifold stitched in the order of shooting by the drone to obtain a panoramic image.

[0052] From the above description, it can be seen that panoramic stitching through offset combined with manifold stitching can reduce the problem of stitching error accumulation, and its stitching order is similar to the order of drone shooting, which helps to achieve panoramic stitching of large quantities of images, avoids panoramic image deformation, and facilitates the global position marking of panoramic images through the preset drone route in the later stage.

[0053] Furthermore, the globally marking the panoramic image according to the preset route of the drone includes:

[0054] Acquire the target image and the center point of the visible light image without photovoltaic defects mapped in the panoramic image;

[0055] Connecting the center points in sequence according to the preset route of the drone to obtain a marked route of the panoramic image;

[0056] dividing the target image and the visible light image without photovoltaic defects mapped in the panoramic image into row images and column images according to the marked route;

[0057] Sequence marking is performed on the row images and the column images respectively to obtain a global position mark of the panoramic image.

[0058] From the above description, it can be seen that in order to accurately describe the position of photovoltaic defects in the panoramic image, it is necessary to first mark the flight route of the drone in the panoramic image, so that the sorting method of each image is similar to the flight route of the drone. At the same time, after determining the position of each image in the panoramic image according to the marked route, it is divided into row images and column images for sequence marking, so that each image is used as a reference coordinate system to accurately describe the position of photovoltaic defects in the overall photovoltaic field, thereby facilitating rapid positioning by operation and maintenance personnel.

[0059] Furthermore, the step of obtaining photovoltaic defects through the infrared image and obtaining a visible light image mapped with the photovoltaic defects as a target image includes:

[0060] Inputting the infrared image into the optimized YOLOv5s model for recognition to obtain an infrared image marked with photovoltaic defects;

[0061] The infrared image marked with the photovoltaic defect is registered with the visible light image based on sparse features to obtain a visible light image mapped with the photovoltaic defect, which is used as the target image.

[0062] As can be seen from the above description, using the latest YOLOv5s model to recognize infrared images can effectively improve the accuracy of photovoltaic defect detection. Furthermore, the infrared images captured by drones have low resolution, while visible light images have high resolution. There is a certain deviation in the field of view between the two images. Furthermore, infrared images are false-color rendered, making feature point matching with visible light images difficult. Therefore, sparse feature registration is used to ensure accurate mapping.

[0063] Furthermore, the calculation of the offset of the adjacent background image based on the offset and longitude and latitude difference of the other adjacent background images and the longitude and latitude difference of the adjacent background image is specifically as follows:

[0064] Calculate the proportional relationship between the offset and the latitude and longitude difference of the other adjacent background images:

[0065]

[0066] The offset of the adjacent background image is calculated according to the proportional relationship and the latitude and longitude difference of the adjacent background image:

[0067] offset=[abs(lng i -lng i+1 ),abs(lat i -lat i+1 )]·R T ;

[0068] Among them, offset[0] represents the offset of other adjacent background images in longitude; offset[1] represents the offset of other adjacent background images in latitude; abs(lng i-1 -lng i ) represents the longitude difference of other adjacent background images; abs(lat i-1 -lat i ) represents the latitude difference of other adjacent background images; abs(lng i -lng i+1 ) represents the longitude difference between adjacent background images; abs(lat i -lat i+1 ) represents the latitude difference between adjacent background images; R T The transposed matrix representing the proportional relationship.

[0069] From the above description, it can be seen that the offset of other adjacent background images and the proportional relationship corresponding to the longitude and latitude difference are first used, and then the offset of the current adjacent background image is inferred through the longitude and latitude difference, so as to avoid the error caused by the offset calculated when the feature point matching accuracy is low, and ensure the integrity and accuracy of the subsequent panoramic stitching.

[0070] Please refer to Figure 8 Another embodiment of the present invention provides a photovoltaic unit defect locating device, comprising:

[0071] An acquisition module is used to acquire infrared and visible light images of all photovoltaic units along the preset route of the drone;

[0072] a mapping module, configured to acquire photovoltaic defects through the infrared light image, and acquire a visible light image mapped with the photovoltaic defects as a target image;

[0073] a stitching module, configured to perform panoramic stitching on the target image and the visible light image without photovoltaic defects mapped therein to obtain a panoramic image;

[0074] A positioning module is used to mark the global position of the panoramic image according to the preset route of the drone, and to locate the photovoltaic defect according to the global position mark.

[0075] As can be seen from the above description, the beneficial effects of the present invention are: infrared and visible light images of all photovoltaic units are obtained according to the preset route of the drone, and visible light images with photovoltaic defects mapped are obtained through infrared images to obtain target images, thereby improving the efficiency of photovoltaic unit defect inspections and effectively reducing the number of manual inspections. At the same time, by panoramically stitching the target image with photovoltaic defects and other visible light images without photovoltaic defects taken by the drone, after obtaining a panoramic image, the panoramic image is globally marked, so that when the drone detects photovoltaic defects, it can accurately locate the photovoltaic defects in the photovoltaic field area, solving the problem that traditional drone inspection positioning is limited by the drone's field of view, and greatly improving the operation and maintenance efficiency of photovoltaic units.

[0076] Another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements the steps of the above-mentioned photovoltaic unit defect location method.

[0077] Please refer to Figure 9 Another embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, each step in the above-mentioned photovoltaic unit defect location method is implemented.

[0078] As can be seen from the above description, the beneficial effects of the present invention are: infrared and visible light images of all photovoltaic units are obtained according to the preset route of the drone, and visible light images with photovoltaic defects mapped are obtained through infrared images to obtain target images, thereby improving the efficiency of photovoltaic unit defect inspections and effectively reducing the number of manual inspections. At the same time, by panoramically stitching the target image with photovoltaic defects and other visible light images without photovoltaic defects taken by the drone, after obtaining a panoramic image, the panoramic image is globally marked, so that when the drone detects photovoltaic defects, it can accurately locate the photovoltaic defects in the photovoltaic field area, solving the problem that traditional drone inspection positioning is limited by the drone's field of view, and greatly improving the operation and maintenance efficiency of photovoltaic units.

[0079] The embodiments of the present invention provide a photovoltaic unit defect location method, device, storage medium, and electronic device, which can be used for photovoltaic unit operation and maintenance in photovoltaic power plants, improve photovoltaic defect inspection efficiency, reduce the number of manual inspections, and accurately locate the location coordinates of photovoltaic defects in real time. The following is an illustration of the method and device using a specific embodiment:

[0080] Please refer to Figures 1 to 7 , embodiment 1 of the present invention is:

[0081] A photovoltaic unit defect locating method, comprising:

[0082] S1. Obtain infrared and visible light images of all photovoltaic units along the preset route of the drone.

[0083] In some embodiments, the drone's preset route includes multiple collection points, distributed according to the coverage area of ​​the photovoltaic cells. The drone simultaneously collects infrared and visible light images at each collection point to ensure that the drone can capture infrared and visible light images of all photovoltaic cells along the preset route. It should be noted that both the infrared and visible light images are orthophoto images.

[0084] S2, obtaining photovoltaic defects through the infrared image, and obtaining a visible light image mapped with the photovoltaic defects as a target image. Specifically, S2 includes:

[0085] S21. Input the infrared image into the optimized YOLOv5s model for recognition to obtain an infrared image marked with photovoltaic defects.

[0086] In some embodiments, the S21 is specifically:

[0087] Step A1: Use labelme (a deep learning image labeling software) to calibrate known photovoltaic defects on the infrared image to obtain a training image set.

[0088] Step A2: Perform data augmentation on the training image set. In this way, the training dataset of photovoltaic defects is expanded and the training accuracy of the YOLOv5s model is improved.

[0089] The step A2 is specifically as follows:

[0090] Step A21: Perform color inversion enhancement on all infrared images in the training image set; in this way, the contour information of the photovoltaic unit area is highlighted.

[0091] Step A22: Randomly generate infrared images of different resolutions from all the infrared images in the training image set; in this way, the problem of inconsistent resolution of images taken at different heights is solved.

[0092] Step A23: Randomly adjust all infrared images in the training image set to image saturations of different scales; in this way, the problem that images taken at different heights exhibit different saturations is solved.

[0093] Step A24: Perform multi-scale exposure on all infrared images in the training image set; in this way, the edges and grayscale transition parts of the images are enhanced.

[0094] Step A25: Randomly adjust the brightness values ​​of all infrared images in the training image set; in this way, images under different brightness can be better recognized.

[0095] Step A3: Train the YOLOv5s model using the image set obtained in steps A21-A25, and continuously optimize the YOLOv5s model based on the loss function to obtain an optimized YOLOv5s model. This solves the sample imbalance between defect categories and improves detection accuracy. Figure 2 Part of the detection results of the YOLOv5s model, Figure 2 The images in the figure are all infrared images, and the areas with white spots in the images are photovoltaic defects.

[0096] S22 . Registering the infrared image marked with the photovoltaic defect with the visible light image based on sparse features to obtain a visible light image mapped with the photovoltaic defect, and using the image as a target image.

[0097] In some embodiments, the S22 is specifically:

[0098] Step B1, extracting photovoltaic unit areas in the infrared image and the visible light image respectively;

[0099] Step B2: Binarize the photovoltaic unit area in the infrared image and the photovoltaic unit area in the visible light image to extract edge information;

[0100] Step B3: aligning the infrared image and the optical image based on the edge information of the infrared image and the edge information of the optical image based on the sparse features;

[0101] Step B4: Map the photovoltaic defects detected in the infrared image to the photovoltaic unit area of ​​the successfully registered visible light image to obtain the target image. Figure 3 The photovoltaic defect corresponding to the infrared image and the visible light image is registered; Figure 3 In the first row of images in the mid-infrared image, the area where the light spot occurs is circled with a square because the light spot is not obvious in the black and white image; in the visible light image, the area where the photovoltaic defect is located is circled with a box diagram because the area where the photovoltaic defect is located is not obvious in the black and white image using color calibration.

[0102] S3, performing panoramic stitching on the target image and the visible light image without photovoltaic defects to obtain a panoramic image. Figure 5 As shown, specifically, the S3 includes:

[0103] S31 , eliminating the target image and the photovoltaic unit area in the visible light image that is not mapped with photovoltaic defects to obtain a corresponding background image. Figure 5The image sequence to be stitched includes the target image and the visible light image without photovoltaic defects mapped therein.

[0104] In an optional embodiment, the target image and the non-photovoltaic unit area in the visible light image where no photovoltaic defects are mapped may be extracted to obtain a corresponding background image.

[0105] S32. Select reference background images in sequence according to the preset route of the drone, and extract feature points of the reference background image and adjacent background images respectively; the adjacent background images are background images that are adjacent to the reference background image in the order of shooting by the drone, that is, the images that are located before and after the reference background image in the order of shooting by the drone are adjacent background images.

[0106] It should be noted that the selected reference background image is used to determine the adjacent background image for which the offset needs to be calculated.

[0107] In some embodiments, feature points of the reference background image and adjacent background images are extracted using a SIFT (Scale-invariant Feature Transform) algorithm. Because some images only have a small portion of background area left after the photovoltaic cell area is removed, resulting in a lack of feature points and inability to accurately match, it is necessary to construct images at multiple scales (e.g., 2880x1560, 1920x1280, and 1280x960) to extract as many feature points as possible and improve matching accuracy.

[0108] S33, matching the reference background image with the adjacent background image for feature points, and calculating the offset of the adjacent background image based on the matching results of the feature points. Specifically, in S33: calculating the offset of the adjacent background image based on the matching results of the feature points is specifically as follows:

[0109] S331, determining whether the number of feature point matches is greater than or equal to a preset number, if so, executing step S332; otherwise, executing step S333;

[0110] In some embodiments, the preset number is 4.

[0111] S332: Calculate the homography matrix of the adjacent background image, and calculate the offset of the adjacent background image according to the homography matrix.

[0112] In some embodiments, assume that its homography matrix H is as follows:

[0113]

[0114] Then the offset of the adjacent background image is offset=[h 13 , h 23 ].

[0115] S333. Obtain the offset and longitude and latitude differences of other adjacent background images, and calculate the longitude and latitude differences of the adjacent background images. Calculate the offset of the adjacent background image based on the offset and longitude and latitude differences of the other adjacent background images and the longitude and latitude differences of the adjacent background images.

[0116] Specifically, in S333, the offset of the adjacent background image is calculated based on the offset and longitude and latitude difference of the other adjacent background images and the longitude and latitude difference of the adjacent background image as follows:

[0117] S3331. Calculate the proportional relationship between the offset and the longitude and latitude difference of the other adjacent background images:

[0118]

[0119] S3332: Calculate the offset of the adjacent background images according to the proportional relationship and the longitude and latitude differences of the adjacent background images:

[0120] offset=[abs(lng i -lng i+1 ),abs(lat i -lat i+1 )]·R T ;

[0121] Among them, offset[0] represents the offset of other adjacent background images in longitude; offset[1] represents the offset of other adjacent background images in latitude; abs(lng i-1 -lng i ) represents the longitude difference of other adjacent background images; abs(lat i-1 -lat i ) represents the latitude difference of other adjacent background images; abs(lng i -lng i+1 ) represents the longitude difference between adjacent background images; abs(lat i -lat i+1 ) represents the latitude difference between adjacent background images; R T The transposed matrix representing the proportional relationship.

[0122] It should be noted that the other adjacent background images here are background images that are located before the current adjacent background image in the shooting order.

[0123] It should be noted that, by executing step S32 and step S33, the offset of each image to be stitched is obtained, and after the offset is stored, step S34 is executed.

[0124] S34, stitching the target image and the visible light image without photovoltaic defects mapped therein by a manifold stitching method according to the offset to obtain a panoramic image. The S34 is specifically as follows:

[0125] S341 : Calculate a global starting point of each of the target images and each of the visible light images not mapped with photovoltaic defects according to the offset.

[0126] In some embodiments, the S341 is specifically:

[0127]

[0128] Among them, start_point is the global starting point, i is the shooting order of the image, and offset is the offset.

[0129] The global starting point of the first image i=0 is [0,0], and the global starting point of subsequent images i>0 is the accumulation of the offsets of all previous images.

[0130] S342: Based on the global starting point, each of the target images and each of the visible light images without photovoltaic defects mapped therein are subjected to manifold stitching in the order in which the drone was photographed, to obtain a panoramic image. Figure 4 As shown, images 1-4 are all visible light images.

[0131] It should be noted that stitching the target image with all visible light images that do not map photovoltaic defects is a large-scale photovoltaic image stitching process. Compared with image stitching in other scenarios, it is more difficult because photovoltaic panels have highly similar features in the image, so the feature points within the photovoltaic panel range will seriously interfere with the overall feature matching results, resulting in the inability to accurately calculate the homography matrix between adjacent images. Therefore, if the traditional affine transformation stitching method is used for the stitching of large-scale photovoltaic images, the errors will continue to accumulate, resulting in serious deformation of the overall image.

[0132] S4. Marking the global position of the panoramic image according to the preset route of the drone, and locating the photovoltaic defect according to the global position mark.

[0133] Specifically, in S4: marking the global position of the panoramic image according to the preset route of the drone includes:

[0134] S41 : Acquire the target image and the center point of the visible light image without photovoltaic defects mapped therein in the panoramic image.

[0135] S42: Connect the central points in sequence according to the preset route of the drone to obtain a marked route of the panoramic image.

[0136] It should be noted that the center point of the image can be used as the collection point of each image, so that the collection points and the preset route of the drone can be connected to obtain a marked route.

[0137] S43 . Divide the target image and the visible light image without photovoltaic defects mapped in the panoramic image into row images and column images according to the marked route.

[0138] In some embodiments, a row of images parallel to the long line marking the route is called a row image, and a row of images perpendicular to the long line marking the route is called a column image. Figure 6 As shown, Figure 6 In the figure, images {1,2,3}, {4,5,6}, and {7,8,9} are all row images, and images {1,6,7}, {2,5,8}, and {3,4,9} are all column images.

[0139] S44 , performing sequence marking on the row images and column images respectively to obtain a global position mark of the panoramic image.

[0140] In some embodiments, the sequence marking of row images and column images is to mark the image number in the row / column. i,n Sequence labeling is performed, where i represents the number of rows and n represents the number of images in the i-th row, such as Figure 7 As shown, Figure 7 There are 7 rows of images in the middle row, H1-H7, where each image in the row is marked sequentially in H1; the row images in the marked route are marked with V i,n Perform sequence labeling, where i represents the number of columns and n represents the image number in the i-th column. Figure 7 The sequence labels of the column images are not shown.

[0141] Please refer to Figure 8 , the second embodiment of the present invention is:

[0142] A photovoltaic unit defect locating device, comprising:

[0143] The acquisition module 101 is used to acquire infrared images and visible light images of all photovoltaic units along the preset route of the drone;

[0144] A mapping module 102 is configured to obtain photovoltaic defects through the infrared image and obtain a visible light image mapped with the photovoltaic defects as a target image;

[0145] A stitching module 103 is configured to perform panoramic stitching on the target image and the visible light image without photovoltaic defects mapped therein to obtain a panoramic image;

[0146] The positioning module 104 is configured to mark the global position of the panoramic image according to the preset route of the drone, and to locate the photovoltaic defect according to the global position mark.

[0147] In this embodiment, the device can implement each step of the photovoltaic unit defect positioning method described in the first embodiment through the acquisition module, the mapping module, the splicing module and the positioning module.

[0148] The third embodiment of the present invention is:

[0149] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a photovoltaic unit defect location method described in embodiment 1.

[0150] Please refer to Figure 9 , the fourth embodiment of the present invention is:

[0151] An electronic device includes a memory 201, a processor 202, and a computer program stored in the memory 201 and executable on the processor 202. When the processor 202 executes the computer program, each step of a photovoltaic unit defect location method described in Example 1 is implemented.

[0152] In summary, the present invention provides a photovoltaic unit defect positioning method, device, storage medium and electronic device, which obtains infrared light images and visible light images of all photovoltaic units according to the preset route of the drone, and inputs the infrared light image into the YOLOv5s model for detection to obtain photovoltaic defects, thereby obtaining a visible light image mapped with the photovoltaic defects, and obtaining a target image. The drone inspection replaces the manual inspection, and the single inspection area is large, which improves the efficiency of photovoltaic unit defect inspection. At the same time, by manifold splicing the target image mapped with photovoltaic defects with other visible light images without photovoltaic defects taken by the drone, a panoramic image of the entire photovoltaic field is obtained, which solves the problem of error accumulation and deformation caused by large-scale image splicing; at the same time, the panoramic image is globally marked according to the drone route, so that when the drone detects photovoltaic defects, it can use the image taken under the preset drone route as a coordinate reference to accurately locate the photovoltaic defects in the photovoltaic field area, solving the problem that traditional drone inspection positioning is limited by the drone's field of view, and greatly improving the operation and maintenance efficiency of photovoltaic units.

[0153] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's description and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A photovoltaic unit defect location method, characterized in that: include: Obtain infrared and visible light images of all photovoltaic units along the drone's preset route; Acquire a photovoltaic defect through the infrared light image, and acquire a visible light image mapped with the photovoltaic defect as a target image; Performing panoramic stitching on the target image and the visible light image without photovoltaic defects mapped therein to obtain a panoramic image; Performing a global position mark on the panoramic image according to a preset route of the drone, and locating the photovoltaic defect according to the global position mark; The panoramic stitching of the target image and the visible light image without photovoltaic defects mapped therein to obtain a panoramic image comprises: Eliminating the target image and the photovoltaic unit area in the visible light image that is not mapped with photovoltaic defects to obtain a corresponding background image; Selecting reference background images in sequence according to the preset route of the drone, and extracting feature points of the reference background image and adjacent background images respectively; the adjacent background images are background images that are adjacent to the reference background image in the order of shooting by the drone; Matching feature points of the reference background image with adjacent background images, and calculating an offset of the adjacent background images based on the matching results of the feature points; splicing the target image and the visible light image without photovoltaic defects mapped therein by a manifold stitching method according to the offset to obtain a panoramic image; The performing global position marking on the panoramic image according to the preset route of the drone includes: Acquire the target image and the center point of the visible light image without photovoltaic defects mapped in the panoramic image; Connecting the center points in sequence according to the preset route of the drone to obtain a marked route of the panoramic image; dividing the target image and the visible light image without photovoltaic defects mapped in the panoramic image into row images and column images according to the marked route; Sequence marking is performed on the row images and the column images respectively to obtain a global position mark of the panoramic image.

2. A photovoltaic unit defect location method according to claim 1, characterized in that: The calculation of the offset of the adjacent background image according to the matching result of the feature points is specifically as follows: Determining whether the number of feature point matches is greater than or equal to a preset number, and if so, calculating a homography matrix of the adjacent background image, and calculating an offset of the adjacent background image according to the homography matrix; Otherwise, obtain the offset and longitude and latitude differences of other adjacent background images, and calculate the longitude and latitude differences of the adjacent background images, and calculate the offset of the adjacent background image based on the offset and longitude and latitude differences of the other adjacent background images and the longitude and latitude differences of the adjacent background images.

3. A photovoltaic unit defect location method according to claim 1, characterized in that: The target image and the visible light image without photovoltaic defects mapped therein are stitched together using a manifold stitching method according to the offset to obtain a panoramic image. Specifically, the panoramic image is: Calculating a global starting point of each of the target images and each of the visible light images without photovoltaic defects mapped therein according to the offset; According to the global starting point, each of the target images and each of the visible light images without photovoltaic defects mapped therein are manifold stitched in the order of shooting by the drone to obtain a panoramic image.

4. A photovoltaic unit defect location method according to claim 1, characterized in that: The step of acquiring photovoltaic defects through the infrared light image and acquiring a visible light image mapped with the photovoltaic defects as a target image includes: Inputting the infrared image into the optimized YOLOv5s model for recognition to obtain an infrared image marked with photovoltaic defects; The infrared image marked with the photovoltaic defect is registered with the visible light image based on sparse features to obtain a visible light image mapped with the photovoltaic defect, which is used as the target image.

5. A photovoltaic unit defect location method according to claim 2, characterized in that: The method of calculating the offset of the adjacent background image according to the offset and longitude and latitude difference of the other adjacent background images and the longitude and latitude difference of the adjacent background image is specifically as follows: Calculate the proportional relationship between the offset and the latitude and longitude difference of the other adjacent background images: ； The offset of the adjacent background image is calculated according to the proportional relationship and the latitude and longitude difference of the adjacent background image: ； Among them, offset[0] represents the offset of other adjacent background images in longitude; offset[1] represents the offset of other adjacent background images in latitude; abs(lng i-1 -lng i ) represents the longitude difference of other adjacent background images; abs(lat i-1 -lat i ) represents the latitude difference of other adjacent background images; abs(lng i -lng i+1 ) represents the longitude difference between adjacent background images; abs(lat i -lat i+1 ) represents the latitude difference between adjacent background images; R T The transposed matrix representing the proportional relationship.

6. A photovoltaic unit defect locating device, characterized in that: include: An acquisition module is used to acquire infrared and visible light images of all photovoltaic units along the preset route of the drone; a mapping module, configured to acquire photovoltaic defects through the infrared light image, and acquire a visible light image mapped with the photovoltaic defects as a target image; a stitching module, configured to perform panoramic stitching on the target image and the visible light image without photovoltaic defects mapped therein to obtain a panoramic image; a positioning module, configured to mark the global position of the panoramic image according to a preset route of the drone, and to locate the photovoltaic defect according to the global position mark; The panoramic stitching of the target image and the visible light image without photovoltaic defects mapped therein to obtain a panoramic image comprises: Eliminating the target image and the photovoltaic unit area in the visible light image that is not mapped with photovoltaic defects to obtain a corresponding background image; Selecting reference background images in sequence according to the preset route of the drone, and extracting feature points of the reference background image and adjacent background images respectively; the adjacent background images are background images that are adjacent to the reference background image in the order of shooting by the drone; Matching feature points of the reference background image with adjacent background images, and calculating an offset of the adjacent background images based on the matching results of the feature points; splicing the target image and the visible light image without photovoltaic defects mapped therein by a manifold stitching method according to the offset to obtain a panoramic image; The performing global position marking on the panoramic image according to the preset route of the drone includes: Acquire the target image and the center point of the visible light image without photovoltaic defects mapped in the panoramic image; Connecting the center points in sequence according to the preset route of the drone to obtain a marked route of the panoramic image; dividing the target image and the visible light image without photovoltaic defects mapped in the panoramic image into row images and column images according to the marked route; Sequence marking is performed on the row images and the column images respectively to obtain a global position mark of the panoramic image.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the photovoltaic unit defect location method according to any one of claims 1 to 5 is implemented.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, each step of the photovoltaic unit defect location method according to any one of claims 1 to 5 is implemented.

Images