Photovoltaic module hyperspectral image registration splicing method, system and device

Abstract

The invention discloses a photovoltaic module hyperspectral image registration splicing method, system and device. The method comprises the following steps: acquiring a hyperspectral image of a to-be-detected region and a high-resolution image of the same region; the acquired hyperspectral image and the high-resolution image are preprocessed; performing feature extraction and descriptor generation; preliminarily matching the feature points extracted from the hyperspectral image and the high-resolution image; performing accurate matching on the preliminarily matched feature points, calculating a spatial transformation model from the hyperspectral image to the high-resolution image, and performing registration on the hyperspectral image by using the transformation model obtained by calculation; and carrying out sequence splicing and fusion on the plurality of hyperspectral images and the registered image. According to the invention, the high-resolution image-assisted photovoltaic module hyperspectral image registration splicing reduces mismatching, reduces the number of times of repeated calculation and adjustment of an algorithm, reduces the consumption of calculation resources, and efficiently completes the registration splicing task under the condition of not increasing the hardware cost.

Description

Technical Field

[0001] The present invention relates to the technical field of optical image processing, and more specifically, to a method, system and device for registering and splicing hyperspectral images of photovoltaic modules. Background Art

[0002] With the continuous increase in the global demand for clean energy, photovoltaic power generation, as a sustainable energy solution, is rapidly expanding in scale and application scope. In the photovoltaic power generation industry, efficient monitoring and management of photovoltaic modules are crucial, which is directly related to power generation efficiency and economic benefits. Most traditional photovoltaic surface detection methods rely on manual inspections or single optical imaging technologies. Currently, in the operation and maintenance of photovoltaic power plants, there is already an intelligent inspection method that uses drones equipped with visible light and infrared image sensors to obtain visible light and infrared image data of photovoltaic modules, and combines computer vision technology to implement a technical solution for fault detection of photovoltaic modules.

[0003] Hyperspectral imaging technology has brought new technical means for photovoltaic detection. It can capture rich spectral information of photovoltaic modules and shows great potential in the field of photovoltaic module detection. It can be used to identify component defects, dirt layer shielding, aging degree, and evaluate their power generation performance, etc. The hyperspectral images collected by drones equipped with imaging spectrometers have high resolution in the spectral dimension, but their spatial resolution is relatively insufficient, making it difficult to clearly present key spatial detail information of photovoltaic modules, such as the fine gaps between solar cells, edge details, and slight surface textures. During the registration process, the lack of these precise spatial detail information makes it difficult to locate accurate corresponding points, resulting in limited registration accuracy and possible misalignment. At the same time, photovoltaic modules have highly repetitive texture features, which are extremely prone to false matching. The above factors lead to poor registration and splicing effects of hyperspectral images collected by drones equipped with imaging spectrometers in large-scale and multi-scale scenarios in practical applications, making it difficult to obtain complete and well-aligned whole image data, thereby affecting subsequent data analysis and evaluation of photovoltaic modules, resulting in problems such as misjudgment of component defects and deviation of performance evaluation, and having an adverse impact on the overall operation and maintenance decision-making of photovoltaic power plants.

[0004] Therefore, a new solution is needed to solve the above problems. Summary of the Invention

[0005] The purpose of the present invention is to overcome the deficiencies of the above-mentioned prior art and provide a method, system and device for registration and stitching of hyperspectral images of photovoltaic components, which utilize the rich detail information provided by high-resolution images to assist in guiding the registration of hyperspectral images of photovoltaic components, reduce the dependence on feature points and mismatching, effectively solve the problem of repeated textures of photovoltaic components, improve the registration accuracy and reliability, and facilitate the accurate defect detection and contamination layer monitoring of photovoltaic components in the intelligent operation and maintenance of large-scale photovoltaic power stations, and avoid abnormal data fluctuations, system failures or false alarms caused by inaccurate registration.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A photovoltaic module hyperspectral image registration and splicing method, the method comprising:

[0008] Step S1, using an imager mounted on a drone to obtain a hyperspectral image of the area to be measured and a high-resolution image of the same area;

[0009] Step S2, preprocessing the acquired hyperspectral image and high-resolution image, including reflectance correction and geometric correction, calculating the signal-to-noise ratio of each band of the hyperspectral image and selecting the band with the highest signal-to-noise ratio;

[0010] Step S3, extracting features and generating descriptors for the selected hyperspectral image and high-resolution image;

[0011] Step S4, performing preliminary matching of feature points extracted from the hyperspectral image and the high-resolution image;

[0012] Step S5, accurately matching the initially matched feature points, calculating a spatial transformation model from the hyperspectral image to the high-resolution image, and registering the hyperspectral image using the calculated transformation model;

[0013] Step S6, sequentially stitching and fusing the multiple hyperspectral images and the registered images to obtain the whole image result of the area to be measured.

[0014] Furthermore, in step S2, the calculation method of reflectivity correction is:

[0015]

[0016] In the formula, For the wavelength The reflectivity at For the wavelength The radiant brightness at is the dark current response obtained by measuring the black calibration plate, is the radiant brightness measured for a white reference plate, is the known reflectivity value of the white reference plate.

[0017] Further, in step S2, the signal-to-noise ratio is calculated using the local variance method to estimate the signal-to-noise ratio, including the following steps:

[0018] Step S201, use the Canny operator to extract the edges of the image, and the result is a binary image, where the edge value is 1 and the non-edge value is 0;

[0019] Step S202, divide the image into blocks according to the specified sub-block size, and count whether each sub-block contains an edge value. If so, exclude this sub-block and do not refer to the subsequent signal-to-noise ratio estimation;

[0020] Step S203, use the local variance method to estimate the noise value;

[0021] Step S204, count the pixel average value of the image after excluding the edge blocks as the signal estimation value. The calculation formula for the image signal-to-noise ratio is:

[0022]

[0023] In the formula, is the image signal-to-noise ratio; is the image mean; is the noise estimation value.

[0024] Further, in step S203, the method of using the local variance method to estimate the noise value is: calculate the local standard deviation and mean of each sub-block, divide 50 intervals between the minimum value of the local standard deviation and 1.2 times the average value, and allocate each sub-block to the corresponding interval according to the standard deviation size, so as to calculate the obtained histogram, and count the interval with the most sub-blocks according to the histogram, and calculate the average value of the standard deviation within this interval as the noise estimation value.

[0025] Further, in step S3, the feature extraction uses the SIFT algorithm, including the following steps:

[0026] Step S301, through the two-dimensional Gaussian convolution kernel process the image to be registered to generate a scale space , and the calculation method is as follows;

[0027]

[0028]

[0029] In the formula, represents the horizontal and vertical coordinates of the pixel points in the image; is the scale factor of the Gaussian convolution kernel;

[0030] Step S302: Construct a Gaussian pyramid through the scale space. Subtract two adjacent images in the same group of the Gaussian pyramid to obtain the Gaussian difference pyramid. Compare the pixel value of this point with the pixel values of 8 surrounding pixels at the same scale and 9 points in the adjacent upper and lower scales, a total of 26 points, so as to complete the feature point detection;

[0031] Step S303: Establish the main direction for the feature points. The main direction of the feature points is obtained by statistically calculating the gradient direction and gradient amplitude histogram of all pixels within 1.5 times its scale. For each pixel in the image, its gradient amplitude and gradient direction are as follows:

[0032]

[0033]

[0034] In the formula, L is the spatial image closest to the scale of the feature point;

[0035] Step S304: After obtaining the main direction of each feature point, first rotate the main direction of the feature point to the horizontal axis to obtain rotational invariance, and then take a neighborhood window centered on the feature point, divide it into sub-regions, calculate the gradient amplitude and direction of the pixel points in each sub-region, construct and combine the gradient direction histogram to obtain the feature descriptor.

[0036] Furthermore, in step S4, the preliminary matching algorithm adopts the nearest neighbor distance ratio algorithm, including the following steps:

[0037] Step S401: Store the feature point set and of the image to be registered obtained from step S3, the feature descriptor set and ;

[0038] Step S402: Calculate the Euclidean distance of the feature descriptor set of the image to be registered:

[0039]

[0040] In the formula, is the value of the nth dimension in the feature vector of the ith feature point of the reference image, is the value of the nth dimension in the feature vector of the jth feature point of the image to be registered; i and j are the serial numbers of the feature points, used to identify different feature points, and n is the dimension serial number of the feature vector, with a total of 128 dimensions;

[0041] Step S403: Calculate the Euclidean distance between the nearest neighbor points on the two images to be registered The Euclidean distance between the point and the next nearest neighbor The distance ratio, by comparing which with a preset threshold to judge whether the descriptor matches. The calculation method of the final matching point pair is as follows:

[0042]

[0043] In the formula, r is the feature point of the reference image, and are the nearest neighbor point and the next nearest neighbor point; represents the distance ratio threshold.

[0044] Furthermore, in step S5, the accurate matching and transformation model calculation method adopts the Random Sample Consensus algorithm, including the following steps:

[0045] Step S501, extract a minimum sample set from the set of matching point pairs after preliminary matching, then the coordinate point pairs of the image to be registered and The projective transformation can be represented by the hypothetical matrix H, and the calculation method is as follows:

[0046]

[0047]

[0048] In the formula, is the scale factor of the i-th matching point pair, ~ are all constants;

[0049] Step S502, substitute the point pairs into the homography transformation equation, expand to get two equations, substitute multiple points to get a system of equations, and solve the hypothetical matrix H by the least squares method;

[0050] Step S503, transform all the matching points with the obtained hypothetical matrix H, compare the error between the transformed points and the actual matching points with the threshold. If it is less than the threshold, it is an inlier; the error calculation method is as follows:

[0051]

[0052] Step S504, repeat the process of randomly sampling, model estimation and inlier calculation iteratively, and select the matching points with the largest number of inliers. Finally, obtain the inlier set with a confidence level of p and the corresponding optimal transformation model ; the iteration times calculation method is as follows:

[0053]

[0054] Wherein, p is the confidence level, w and m are the inlier ratio and the minimum number of samples respectively.

[0055] Further, in step S6, the image fusion stitching algorithm uses a dynamic programming algorithm to search for an optimal stitching line with the minimum error in the overlapping region, including the following steps:

[0056] Step S601, calculate the weight of the search edge, and the calculation method is as follows:

[0057]

[0058] Wherein, e is the weight of the search edge, and are the overlapping regions of the images to be registered;

[0059] Step S602, apply dynamic programming to find the minimum cost path, traverse the overlapping region from top to bottom and calculate the cumulative minimum squared difference E, and the calculation method is as follows:

[0060]

[0061] Wherein, u is the row coordinate and v is the column coordinate;

[0062] Step S603, the optimal path is obtained by backtracking from the bottom to the top to find the path with the minimum cost; then the overlapping region is divided into two parts, with the optimal seam as the boundary, to obtain the images and the corresponding binary stitching template and ;

[0063] Step S604, perform image sequence stitching, and use the optimal transformation model to calculate the images to be registered and the fusion model based on the optimal stitching line and , take and the intersection of as the stitching template for the image to be stitched onto the panoramic image, where the first image is the image after registration and stitching of the high-resolution image and the first hyperspectral image;

[0064] Step S605, iteratively execute step S604 until all hyperspectral images in the image sequence are stitched into the panoramic image to obtain the final panoramic image.

[0065] The present invention also provides a registration and stitching system for photovoltaic module hyperspectral images, and the system includes:

[0066] An image acquisition unit for photographing the photovoltaic module to be detected to obtain a hyperspectral image and a high-resolution image;

[0067] A preprocessing unit for preprocessing the hyperspectral image by reflectance correction and geometric correction;

[0068] A band selection unit for selecting the band with the optimal hyperspectral signal-to-noise ratio of the photovoltaic module for subsequent registration;

[0069] A feature extraction unit for extracting features from the high-resolution image and the hyperspectral image to be registered;

[0070] An image registration unit for matching feature points and calculating a spatial transformation model to achieve the registration of the high-resolution image and the hyperspectral image;

[0071] An image fusion and stitching unit for performing sequential stitching and fusion on the registered images to obtain a panoramic image.

[0072] The present invention also provides a photovoltaic module hyperspectral image registration and stitching device, including:

[0073] A memory for storing computer programs and data;

[0074] A processor for implementing the steps of the above-mentioned photovoltaic module hyperspectral image registration and stitching method when executing the computer program.

[0075] The beneficial effects of the present invention are:

[0076] 1. In the present invention, the high-resolution image, with its rich detail information, provides a large number of reliable feature points for registration, assisting the algorithm to accurately identify and match, effectively reducing the mis-matching caused by the repetitive texture of the photovoltaic module, and greatly improving the registration accuracy; the accurate registration makes the data such as temperature and spectral reflectance reflecting the state of the photovoltaic module in the hyperspectral image more reliable, providing a credible basis for subsequent defect detection, performance evaluation, etc., and laying a solid data foundation for operation and maintenance decisions.

[0077] 2. In the present invention, the registration and stitching of the photovoltaic module hyperspectral image assisted by the high-resolution image reduces mis-matching, reduces the number of repeated calculations and adjustments of the algorithm, reduces the consumption of computing resources, and efficiently completes the registration and stitching task without increasing the hardware cost.

[0078] 3. With the assistance of high-resolution images, the hyperspectral images of photovoltaic modules registered and stitched by the present invention can not only capture rich spectral information of photovoltaic modules, but also achieve accurate image registration and stitching at large scales and multiple scales. It can be used for accurate defect detection, dirt layer monitoring, intelligent operation and maintenance, etc. of photovoltaic modules in the operation and maintenance monitoring of large-scale photovoltaic power stations, avoiding data abnormal fluctuations, system failures or false alarms caused by inaccurate registration, and providing extensive technical support for the development of the photovoltaic industry. Description of the Drawings

[0079] Figure 1 It is a flowchart of a method for registering and stitching hyperspectral images of photovoltaic modules in this embodiment;

[0080] Figure 2 It is a structural framework diagram of a system for registering and stitching hyperspectral images of photovoltaic modules in this embodiment;

[0081] Figure 3 It is a structural framework diagram of a device for registering and stitching hyperspectral images of photovoltaic modules in this embodiment.

[0082] Reference Signs: Image Acquisition Unit 1, Preprocessing Unit 2, Band Selection Unit 3, Feature Extraction Unit 4, Image Registration Unit 5, Image Fusion and Stitching Unit 6, Memory 7, Processor 8. Detailed Embodiments

[0083] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0084] Embodiment: A method for registering and stitching hyperspectral images of photovoltaic modules, as Figure 1 shown, the method includes:

[0085] Step S1, using a drone to carry an imager to obtain hyperspectral images of the area to be measured and high-resolution images of the same area;

[0086] Step S2, preprocessing the obtained hyperspectral images and high-resolution images, including reflectance correction, geometric correction, calculating the signal-to-noise ratio of each band of the hyperspectral image and selecting the band with the highest signal-to-noise ratio for subsequent feature matching;

[0087] Among them, the calculation method of reflectance correction is:

[0088]

[0089] In the formula, is the reflectance at wavelength , is the radiance (obtained from the sensor) at wavelength , is the dark current response obtained by measuring a black calibration plate, is the radiance obtained by measuring a white reference plate, is the known reflectance value of the white reference plate, usually 1 or a known high reflectance value.

[0090] The signal-to-noise ratio is calculated using the local variance method to estimate the signal-to-noise ratio, including the following steps:

[0091] Step S201, use the Canny operator to extract edges from the image, and the result is a binary image with edge values of 1 and non-edge values of 0;

[0092] Step S202, divide the image into blocks according to the specified sub-block size (4×4), count whether each sub-block contains an edge value, and if so, exclude the sub-block and do not participate in the subsequent signal-to-noise ratio estimation;

[0093] Step S203, use the local variance method to estimate the noise value; specifically:

[0094] Calculate the local standard deviation and mean of each sub-block, divide 50 intervals between the minimum local standard deviation and 1.2 times the average value, and assign each sub-block to the corresponding interval according to the standard deviation size to obtain a histogram. Calculate the average value of the standard deviation within the interval with the most sub-blocks as the noise estimate value;

[0095] Step S204, count the average value of the pixels in the image after excluding the edge blocks as the signal estimate value. The calculation formula for the image signal-to-noise ratio is:

[0096]

[0097] In the formula, is the image signal-to-noise ratio; is the image mean; is the noise estimate value.

[0098] Step S3, perform feature extraction and descriptor generation on the selected hyperspectral image and high-resolution image;

[0099] Among them, feature extraction uses the SIFT algorithm, including the following steps:

[0100] Step S301, use the two-dimensional Gaussian convolution kernel to process the image to be registered to generate a scale space , and the calculation method is as follows;

[0101]

[0102]

[0103] In the formula, represent the horizontal and vertical coordinates of the pixel points in the image; is the scale factor of the Gaussian convolution kernel;

[0104] Step S302: Construct a Gaussian pyramid through the scale space. Subtract the adjacent two layers of images in the same group in the Gaussian pyramid to obtain the Gaussian difference pyramid. Compare the pixel value of this point with the pixel values of 8 surrounding pixels at the same scale and 9 points in the adjacent upper and lower scales, a total of 26 points, so as to complete the feature point detection;

[0105] Step S303: In order to make the feature points have rotation invariance, establish the main direction for the feature points. The main direction of the feature points is obtained by statistically analyzing the gradient direction and gradient amplitude histogram of all pixels within 1.5 times of its scale range of the feature points. For each pixel in the image, its gradient amplitude and gradient direction are as follows:

[0106]

[0107]

[0108] In the formula, L is the spatial image closest to the scale of the feature point; according to the fact that the pixels closer to the feature point contribute more to the direction of the feature point, perform Gaussian filtering on the pixels within 1.5 times range of the feature point, and then generate a histogram according to the gradient amplitude and gradient direction of the pixels within this range. Among them, the horizontal direction has 36 intervals with 10 degrees as 1 interval, and the vertical direction is the cumulative value of the gradients in each angle interval. Take the corresponding direction of the histogram value of each feature point as the main direction of the feature point, and in order to improve the matching accuracy and stability of the feature points, take the direction with the amplitude greater than 80% of the peak value in the histogram as the secondary direction of the feature point;

[0109] Step S304: After obtaining the main direction of each feature point, first rotate the main direction of the feature point to the horizontal axis to obtain rotation invariance, and then take a neighborhood window centered on the feature point, divide it into sub-regions, calculate the gradient amplitude and direction of the pixel points in each sub-region, construct and combine the gradient direction histogram to obtain the feature descriptor.

[0110] Step S4: Perform preliminary matching on the feature points extracted from the hyperspectral image and the high-resolution image;

[0111] Among them, the preliminary matching algorithm adopts the nearest neighbor distance ratio algorithm, including the following steps:

[0112] Step S401: Store the set of feature points of the image to be registered obtained from Step S3 and , the set of feature descriptors and (each 128 - dimensional);

[0113] Step S402: Calculate the Euclidean distance of the set of feature descriptors of the image to be registered :

[0114]

[0115] In the formula, is the value of the n - th dimension in the feature vector of the i - th feature point of the reference image, is the value of the n - th dimension in the feature vector of the j - th feature point of the image to be registered; i and j are the serial numbers of the feature points, used to identify different feature points, n is the dimension serial number of the feature vector, with a total of 128 dimensions, that is, n ranges from 1 to 128; by summing the squares of the differences of the 128 - dimensional corresponding values and then taking the square root, the Euclidean distance between two feature descriptors is obtained, which measures their similarity. The smaller the distance, the more similar the two feature descriptors are;

[0116] Step S403: Calculate the Euclidean distance between the nearest - neighbor points and the second - nearest - neighbor points on two images to be registered and the Euclidean distance between the second - nearest - neighbor points The distance ratio is used to judge whether the descriptors match by comparing this ratio with a preset threshold. The calculation method of the final matching point pairs is as follows:

[0117]

[0118] In the formula, r is the feature point of the reference image, and are the nearest - neighbor point and the second - nearest - neighbor point; represents the distance - ratio threshold, set to 0.4 - 0.8.

[0119] Step S5: Perform precise matching on the preliminarily matched feature points, and calculate the spatial transformation model from the hyperspectral image to the high - resolution image, and register the hyperspectral image using the calculated transformation model;

[0120] Among them, the precise matching and the calculation method of the transformation model adopt the Random Sample Consensus (RANSAC) algorithm, including the following steps:

[0121] Step S501: Extract a minimum sample set from the set of matching point pairs after preliminary matching. Then, for the coordinate point pairs of the image to be registered and The projection transformation can be represented by the assumed matrix H, which is calculated as follows:

[0122]

[0123]

[0124] In the formula, is the scale factor of the i-th matching point pair, ~ are all constants;

[0125] Step S502, bringing the point pair into the homography transformation equation, expanding to obtain two equations, substituting multiple points into the equation system, and solving the assumption matrix H by the least squares method;

[0126] Step S503, using the obtained hypothesis matrix H to change all matching points, and by calculating the error between the transformed point and the actual matching point and comparing it with the threshold, if it is less than the threshold, it is an inlier point; the error calculation method is as follows:

[0127]

[0128] Step S504, repeat the process of iterative random sampling, model estimation and calculation of inliers, and select the matching point with the largest number of inliers, and finally obtain the inlier set with a confidence level of p and the corresponding optimal transformation model ; The number of iterations is calculated as follows:

[0129]

[0130] Where p is the confidence level (usually 0.995), w and m are the proportion of internal points and the minimum number of samples, respectively.

[0131] Step S6, sequentially stitching and fusing the multiple hyperspectral images with the registered images to obtain the whole image result of the area to be measured;

[0132] The image fusion stitching algorithm uses a dynamic programming algorithm to search for an optimal stitching line with the minimum error in the overlapping area, including the following steps:

[0133] Step S601, calculate the weight of the search edge, the calculation method is as follows:

[0134]

[0135] In the formula, e is the weight of the search edge, and is the overlapping area of ​​the images to be registered;

[0136] Step S602: Apply dynamic programming to find the minimum-cost path, traverse the overlapping region from top to bottom, and calculate the cumulative minimum squared difference E. The calculation method is as follows:

[0137]

[0138] In the formula, u is the row coordinate and v is the column coordinate;

[0139] Step S603: The optimal path is obtained by backtracking from the bottom to the top to find the path with the minimum cost; then the overlapping region is divided into two parts with the optimal seam as the boundary to obtain the image and the corresponding binary splicing template and ;

[0140] Step S604: Perform image sequence splicing, and use the optimal transformation model in Step S504 and to calculate the fusion model and of the image to be registered and based on the best stitching line. Take the intersection of as the splicing template for splicing the image

[0141] into the panoramic image. The first image is the image after registration and splicing of the high-resolution image and the first hyperspectral image;

[0142] In the technical solution provided by the embodiment of the present application, by using the rich detail information provided by the high-resolution image to assist in guiding the registration of the hyperspectral image of the photovoltaic module, the dependence on feature points and false matching are reduced, the problem of repeated textures of the photovoltaic module can be effectively solved, the registration accuracy and reliability are improved, which is conducive to realizing accurate defect detection, dirt layer monitoring, etc. of the photovoltaic module in the intelligent operation and maintenance of large-scale photovoltaic power stations, and avoiding data abnormal fluctuations, system failures or false alarms caused by inaccurate registration.

[0143] See Figure 2 which is a schematic structural diagram of a hyperspectral image registration and splicing system for photovoltaic modules provided by the embodiment of the present application; the system includes an image acquisition unit 1, a preprocessing unit 2, a band selection unit 3, a feature extraction unit 4, an image registration unit 5, and an image fusion and splicing unit 6;

[0144] Among them, the image acquisition unit 1 is used to capture hyperspectral images and high-resolution images of the photovoltaic module to be detected;

[0145] A preprocessing unit 2 for preprocessing the hyperspectral image by reflectance correction and geometric correction;

[0146] A band selection unit 3 for selecting the band with the optimal hyperspectral signal-to-noise ratio of the photovoltaic module for subsequent registration;

[0147] A feature extraction unit 4 for extracting features from the high-resolution image and the hyperspectral image to be registered;

[0148] An image registration unit 5 for matching feature points and calculating a spatial transformation model to achieve the registration of the high-resolution image and the hyperspectral image;

[0149] An image fusion and stitching unit 6 for performing sequential stitching and fusion on the registered images to obtain a panoramic image.

[0150] In the technical solution provided by the embodiment of the present application, first, the hyperspectral images and high-resolution images of all photovoltaic modules to be detected are acquired; the hyperspectral images and high-resolution images are preprocessed, including reflectance correction and geometric correction; the signal-to-noise ratio of each band of the hyperspectral image is calculated and the band with the highest signal-to-noise ratio is selected; a feature extraction algorithm is used to extract the features of the hyperspectral image and the high-resolution image; the features extracted from the high-resolution image are matched with the features extracted from the hyperspectral image; according to the matched feature pairs, a spatial transformation model from the hyperspectral image to the high-resolution image is calculated; the calculated transformation model is used to register the hyperspectral image; finally, multiple images are sequentially fused and stitched.

[0151] See Figure 3 , which is a device for registering and stitching the hyperspectral image of a photovoltaic module provided by the embodiment of the present application; the device includes a memory 7 and a processor 8;

[0152] Among them, the memory 7 is used to store computer programs and data; the processor 8 is used to implement the steps of the above-mentioned method for registering and stitching the hyperspectral image of a photovoltaic module when executing the computer program.

[0153] The above are only the preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and retouches should also be regarded as the protection scope of the present invention.

Claims

1. A photovoltaic module hyperspectral image registration and splicing method, characterized in that: The method comprises: Step S1, using an imager mounted on a drone to obtain a hyperspectral image of the area to be measured and a high-resolution image of the same area; Step S2, preprocessing the acquired hyperspectral image and high-resolution image, including reflectance correction and geometric correction, calculating the signal-to-noise ratio of each band of the hyperspectral image and selecting the band with the highest signal-to-noise ratio; Step S3, extracting features and generating descriptors for the selected hyperspectral image and high-resolution image; Step S4, performing preliminary matching of feature points extracted from the hyperspectral image and the high-resolution image; Step S5, accurately matching the initially matched feature points, calculating a spatial transformation model from the hyperspectral image to the high-resolution image, and registering the hyperspectral image using the calculated transformation model; Step S6, sequentially stitching and fusing the multiple hyperspectral images and the registered images to obtain the whole image result of the area to be measured.

2. A photovoltaic module hyperspectral image registration and stitching method according to claim 1, characterized in that: In step S2, the calculation method of reflectivity correction is: In the formula, For the wavelength The reflectivity at For the wavelength The radiant brightness at is the dark current response obtained by measuring the black calibration plate, is the radiant brightness measured for a white reference plate, is the known reflectivity value of the white reference plate.

3. A photovoltaic module hyperspectral image registration and stitching method according to claim 1, characterized in that: In step S2, the calculation of the signal-to-noise ratio adopts the local variance method to estimate the signal-to-noise ratio, including the following steps: Step S201, using the Canny operator to extract the edge of the image, the result is a binary image, the edge value is 1, and the non-edge value is 0; Step S202, dividing the image into blocks according to the specified sub-block size, counting whether each sub-block contains an edge value, and if so, removing the sub-block and not referring to the subsequent signal-to-noise ratio estimation; Step S203, estimating the noise value using the local variance method; Step S204, the average pixel value of the image after removing the edge blocks is counted as the signal estimation value, and the calculation formula of the image signal-to-noise ratio is: In the formula, is the image signal-to-noise ratio; is the image mean; is the noise estimate.

4. A photovoltaic module hyperspectral image registration and stitching method according to claim 3, characterized in that: In step S203, the method of estimating the noise value using the local variance method is as follows: calculate the local standard deviation and mean of each sub-block, divide 50 intervals between the minimum local standard deviation and 1.2 times the mean value, and place each sub-block into the corresponding interval according to the size of the standard deviation, thereby calculating a histogram, and according to the histogram, count the interval containing the most sub-blocks, and calculate the average value of the standard deviation in the interval as the noise estimation value.

5. A photovoltaic module hyperspectral image registration and stitching method according to claim 1, characterized in that: In step S3, feature extraction uses SIFT algorithm, including the following steps: Step S301, using a two-dimensional Gaussian convolution kernel Treat the registered image Processing to generate scale space , the calculation method is as follows: In the formula, Represents the horizontal and vertical coordinates of the pixel points in the image; is the scale factor of the Gaussian convolution kernel; Step S302, constructing a Gaussian pyramid in scale space, subtracting two adjacent layers of images in the same group in the Gaussian pyramid to obtain a Gaussian difference pyramid, and comparing the pixel values ​​of the point with the surrounding 8 pixel points of the same scale and the adjacent 9 points of the upper and lower scales, a total of 26 points, to complete feature point detection; Step S303: establish the main direction of the feature point. The main direction of the feature point is obtained by counting the gradient direction and gradient amplitude histogram of all pixels within 1.5 times the scale of the feature point. For each pixel in the image, its gradient amplitude and the gradient direction as follows: Where L is the spatial image closest to the feature point scale; Step S304: after obtaining the main direction of each feature point, first turn the main direction of the feature point to the horizontal direction. Axis is used to obtain rotation invariance, and then a domain window is taken with the feature point as the center, which is divided into sub-regions. The gradient amplitude and direction of the pixels in each sub-region are calculated, and the gradient direction histogram is constructed and combined to obtain the feature descriptor.

6. A photovoltaic module hyperspectral image registration and stitching method according to claim 1, characterized in that: In step S4, the preliminary matching algorithm adopts the nearest neighbor distance ratio algorithm, which includes the following steps: Step S401, storing the feature point set of the image to be registered obtained from step S3 and , feature descriptor set and ; Step S402: Calculate the Euclidean distance of the feature descriptor set of the image to be registered : In the formula, is the value of the nth dimension in the feature vector of the i-th feature point of the reference image, is the value of the nth dimension in the feature vector of the jth feature point of the image to be registered; i and j are the serial numbers of the feature points, used to identify different feature points, and n is the serial number of the dimension of the feature vector, with a total of 128 dimensions; Step S403: Calculate the Euclidean distance between the nearest neighbor points on the two images to be registered. The Euclidean distance between the point and the next nearest neighbor The distance ratio is compared with the preset threshold to judge whether the descriptor matches. The calculation method of the matching point pair is as follows: In the formula, r is the feature point of the reference image, and The nearest neighbor point and the next nearest neighbor point; Represents the distance ratio threshold.

7. A photovoltaic module hyperspectral image registration and stitching method according to claim 1, characterized in that: In step S5, the exact matching and transformation model calculation method adopts a random sampling consensus algorithm, including the following steps: Step S501, extract a minimum sample set from the set of matching point pairs after preliminary matching, then the coordinate point pairs of the image to be registered are and The projection transformation can be represented by the assumed matrix H, which is calculated as follows: In the formula, is the scale factor of the i-th matching point pair, ~ are all constants; Step S502, bringing the point pair into the homography transformation equation, expanding to obtain two equations, substituting multiple points into the equation system, and solving the assumption matrix H by the least squares method; Step S503, using the obtained hypothesis matrix H to change all matching points, and by calculating the error between the transformed point and the actual matching point and comparing it with the threshold, if it is less than the threshold, it is an inlier point; the error calculation method is as follows: Step S504, repeat the process of iterative random sampling, model estimation and calculation of inliers, and select the matching point with the largest number of inliers, and finally obtain the inlier set with a confidence level of p and the corresponding optimal transformation model ; The number of iterations is calculated as follows: Where p is the confidence level, w and m are the proportion of internal points and the minimum number of samples, respectively.

8. The photovoltaic module hyperspectral image registration and stitching method according to claim 1, characterized in that: In step S6, the image fusion stitching algorithm uses a dynamic programming algorithm to search for an optimal stitching line with the minimum error in the overlapping area, including the following steps: Step S601, calculate the weight of the search edge, the calculation method is as follows: In the formula, e is the weight of the search edge, and is the overlapping area of ​​the images to be registered; Step S602, dynamic programming is applied to find the minimum cost path, traverse the overlapping area from top to bottom and calculate the cumulative minimum square difference E, the calculation method is as follows: Where u is the row coordinate and v is the column coordinate; Step S603, the best path is traced back from the bottom to the top to find the path with the minimum cost; then the overlapping area is divided into two parts, with the best gap as the boundary, to obtain the image and The corresponding binary splicing template and ; Step S604: stitching the image sequence using the optimal transformation model To calculate the image to be registered and Fusion model based on optimal stitching and ,Bundle and The intersection of As an image A stitching template for a panorama, wherein the first image is an image obtained by registering and stitching the high-resolution image with the first hyperspectral image; Step S605, iteratively executing step S604 until all the hyperspectral images in the image sequence are stitched into the panoramic image, thereby obtaining a final panoramic image.

9. A photovoltaic module hyperspectral image registration and stitching system for implementing the method of claim 1, characterized in that: The system comprises: An image acquisition unit (1) is used to photograph the photovoltaic module to be inspected to acquire a hyperspectral image and a high-resolution image; A preprocessing unit (2) is used to perform reflectance correction and geometric correction preprocessing on the hyperspectral image; A band selection unit (3) is used to select a band with an optimal high-spectral signal-to-noise ratio of the photovoltaic module for subsequent alignment; A feature extraction unit (4), used for extracting features from the high-resolution image and the hyperspectral image to be registered; An image registration unit (5), used for matching feature points and calculating a spatial transformation model to achieve registration of high-resolution images and hyperspectral images; The image fusion and stitching unit (6) is used to sequentially stitch and fuse the registered images to obtain a panoramic image.

10. A photovoltaic module hyperspectral image registration and splicing device, characterized in that: include: Memory (7) for storing computer programs and data; A processor (8) is used to implement the steps of the photovoltaic assembly hyperspectral image registration and stitching method as described in any one of claims 1 to 8 when executing a computer program.

Images