An image-based monitoring method for photovoltaic construction conditions

Abstract

The present invention discloses an image-based monitoring method for photovoltaic construction conditions, belonging to the technical field of image processing. First, the construction image is divided into multiple photovoltaic module areas, and then the horizontal joint area and the vertical joint area are obtained based on adjacent photovoltaic modules. Subsequently, the construction images of the horizontal joint area and the vertical joint area are respectively subjected to binary processing to obtain a horizontal contour binary image and a vertical contour binary image. Further, the widths at each edge pixel point of the horizontal joint area and the vertical joint area are obtained, and a horizontal local weight image, a horizontal global weight image, a vertical local weight image, and a vertical global weight image are constructed. Finally, each binary image and weight image are input into the photovoltaic construction monitoring model to obtain the photovoltaic construction quality score. This method effectively improves the accuracy of monitoring the photovoltaic construction conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a method for monitoring the photovoltaic construction situation based on images. Background Art

[0002] Photovoltaic power stations are often located in remote areas with rich light resources. However, the construction of photovoltaic power stations in remote areas faces many challenges. On the one hand, photovoltaic power stations cover a vast area, often covering several square kilometers or even a larger area, making it difficult for traditional manual inspection and management methods to comprehensively cover, and the efficiency is extremely low. On the other hand, the transportation in remote areas is inconvenient and the geographical environment is complex, which brings great difficulties to the deployment of management personnel and materials, increasing the management cost and difficulty. At the same time, the number of photovoltaic modules is huge, and their installation quality has a decisive impact on the overall power generation efficiency and stability of the power station. Traditional manual quality inspection is highly subjective and has limited accuracy, making it difficult to accurately evaluate the photovoltaic construction situation. Summary of the Invention

[0003] In view of the above deficiencies in the prior art, a method for monitoring the photovoltaic construction situation based on images provided by the present invention solves the problem of low monitoring accuracy of the existing photovoltaic construction situation.

[0004] In order to achieve the above invention purpose, the technical solution adopted by the present invention is: a method for monitoring the photovoltaic construction situation based on images, including:

[0005] Dividing the construction image into multiple photovoltaic module areas;

[0006] Obtaining the horizontal joint area and the vertical joint area according to adjacent photovoltaic modules;

[0007] Performing binary processing on the construction image according to the horizontal joint area and the vertical joint area respectively to obtain a horizontal contour binary image and a vertical contour binary image;

[0008] Respectively obtaining the widths of the horizontal joint area and the vertical joint area at each edge pixel point, and constructing a horizontal local weight image, a horizontal global weight image, a vertical local weight image, and a vertical global weight image;

[0009] Inputting each binary image and weight image into a photovoltaic construction monitoring model to obtain a photovoltaic construction quality score.

[0010] Further, the process of dividing the construction image into multiple photovoltaic module areas includes:

[0011] Calculating the similarity between the pixel value of each pixel point in the construction image and the pixel value of the stored frame. When the similarity is greater than the similarity threshold, the pixel point is classified as a frame pixel point;

[0012] Calculate the similarity between the pixel value of each pixel point in the construction image and the pixel value of the stored solar panel. When the similarity is greater than the similarity threshold, classify the pixel point as a solar panel pixel point;

[0013] Regard the connected region formed by each solar panel pixel point as a solar panel region;

[0014] Regard the connected region formed by each border pixel point as a border region;

[0015] When the solar panel region is inside the border region, regard the solar panel region and the border region as a whole photovoltaic module region.

[0016] Further, the process of obtaining the horizontal seam region includes:

[0017] Taking each photovoltaic module as the center, obtain its upper and lower neighboring photovoltaic modules to get the upper-side photovoltaic module and the lower-side photovoltaic module;

[0018] Mark the pixel points between the central photovoltaic module and the upper-side photovoltaic module as upper seam points;

[0019] Regard the region formed by each upper seam point as a horizontal seam region;

[0020] Mark the pixel points between the central photovoltaic module and the lower-side photovoltaic module as lower seam points;

[0021] Regard the region formed by each lower seam point as a horizontal seam region.

[0022] Further, the process of obtaining the vertical seam region includes:

[0023] Taking each photovoltaic module as the center, obtain its left and right neighboring photovoltaic modules to get the left-side photovoltaic module and the right-side photovoltaic module;

[0024] Mark the pixel points between the central photovoltaic module and the left-side photovoltaic module as left seam points;

[0025] Regard the region formed by each left seam point as a vertical seam region;

[0026] Mark the pixel points between the central photovoltaic module and the right-side photovoltaic module as right seam points;

[0027] Regard the region formed by each right seam point as a vertical seam region.

[0028] Further, the process of obtaining the horizontal contour binary image and the vertical contour binary image includes:

[0029] Set the pixel value of the pixels belonging to the horizontal seam region to 1 on the construction image, and set the pixel value of other pixel points to 0 to obtain the horizontal initial binary image;

[0030] In the horizontal initial binary image, when there is a pixel with a pixel value of 0 within the neighborhood of a pixel with a pixel value of 1, mark the pixel with a pixel value of 1 as an edge pixel, retain the pixel value of the edge pixel, and set other pixel values to 0 to obtain a horizontal contour binary image;

[0031] On the construction image, set the pixel values belonging to the vertical joint area to 1 and set other pixel values to 0 to obtain a vertical initial binary image;

[0032] In the vertical initial binary image, when there is a pixel with a pixel value of 0 within the neighborhood of a pixel with a pixel value of 1, mark the pixel with a pixel value of 1 as an edge pixel, retain the pixel value of the edge pixel, and set other pixel values to 0 to obtain a vertical contour binary image.

[0033] Furthermore, the process of constructing the horizontal local weight image and the horizontal global weight image includes:

[0034] Extract edge pixels for each horizontal joint area, extract the abscissas of the edge pixels, count the number of pixels at the same abscissa in the horizontal joint area, and record the number as the horizontal width of the edge pixel;

[0035] Take the average of the horizontal widths of all edge pixels in a horizontal joint area to obtain the horizontal local width average;

[0036] Calculate the difference between the horizontal width of each edge pixel in the horizontal joint area and the corresponding horizontal local width average, and perform normalization processing on the difference to obtain the horizontal local deviation value of the corresponding edge pixel;

[0037] Take the average of all horizontal local width averages to obtain the horizontal global width average;

[0038] Calculate the difference between the horizontal width of each edge pixel in the horizontal joint area and the horizontal global width average, and perform normalization processing on the difference to obtain the horizontal global deviation value of the corresponding edge pixel;

[0039] Replace the pixel value of the original edge pixel with the horizontal local deviation value, and set other pixel values to 0 to obtain the horizontal local weight image;

[0040] Replace the pixel value of the original edge pixel with the horizontal global deviation value, and set other pixel values to 0 to obtain the horizontal global weight image.

[0041] Furthermore, the process of constructing the vertical local weight image and the vertical global weight image includes:

[0042] Extract the edge pixel points for each vertical seam area, extract the vertical coordinates of the edge pixel points, count the number of pixel points under the same vertical coordinate in the vertical seam area, and record the number as the vertical width of the edge pixel point;

[0043] Take the average value of the vertical widths of all edge pixel points in a vertical seam area to obtain the average value of the vertical local width;

[0044] Calculate the difference between the vertical width of each edge pixel point in the vertical seam area and the corresponding average value of the vertical local width, and perform normalization processing on the difference to obtain the vertical local deviation value of the corresponding edge pixel point;

[0045] Take the average value of all average values of the vertical local width to obtain the average value of the vertical global width;

[0046] Calculate the difference between the vertical width of each edge pixel point in the vertical seam area and the average value of the vertical global width, and perform normalization processing on the difference to obtain the vertical global deviation value of the corresponding edge pixel point;

[0047] Replace the pixel value of the original edge pixel point with the vertical local deviation value, and set other pixel values to 0 to obtain the vertical local weight image;

[0048] Replace the pixel value of the original edge pixel point with the vertical global deviation value, and set other pixel values to 0 to obtain the vertical global weight image.

[0049] Furthermore, the photovoltaic construction monitoring model includes: a first image feature fusion unit, a second image feature fusion unit, a first CNN network, a second CNN network, a Concat layer, and a fully connected layer;

[0050] The first image feature fusion unit is used to perform feature fusion on the horizontal contour binary image, the horizontal local weight image, and the horizontal global weight image to obtain a first fusion feature; the second image feature fusion unit is used to perform feature fusion on the vertical contour binary image, the vertical local weight image, and the vertical global weight image to obtain a second fusion feature;

[0051] The first CNN network is used to extract deep features from the first fusion feature; the second CNN network is used to extract deep features from the second fusion feature; the Concat layer is used to splice the deep features output by the first CNN network and the second CNN network to obtain a spliced feature; the fully connected layer is used to output the photovoltaic construction quality score according to the spliced feature.

[0052] Furthermore, both the first image feature fusion unit and the second image feature fusion unit include: a first convolutional layer, a second convolutional layer, a third convolutional layer, a multiplier M1, a multiplier M2, and an adder A1;

[0053] The first input terminal of multiplier M1 is connected to the output terminal of the first convolutional layer, and its second input terminal is connected to the output terminal of the second convolutional layer; the first input terminal of multiplier M2 is connected to the output terminal of the second convolutional layer, and its second input terminal is connected to the output terminal of the third convolutional layer; the first input terminal of adder A1 is connected to the output terminal of multiplier M1, its second input terminal is connected to the output terminal of multiplier M2, and its output terminal serves as the output terminal of the first image feature fusion unit or the second image feature fusion unit.

[0054] Further, in the first image feature fusion unit, the input terminal of the first convolutional layer is used to input a horizontal local weight image, the input terminal of the second convolutional layer is used to input a horizontal contour binary image, and the input terminal of the third convolutional layer is used to input a horizontal global weight image;

[0055] In the second image feature fusion unit, the input terminal of the first convolutional layer is used to input a vertical local weight image, the input terminal of the second convolutional layer is used to input a vertical contour binary image, and the input terminal of the third convolutional layer is used to input a vertical global weight image.

[0056] The beneficial effects of the present invention are as follows:

[0057] The present invention divides the construction image into multiple photovoltaic modules, obtains the horizontal joint area and the vertical joint area, and thereby evaluates the construction quality when each photovoltaic module is spliced. Subsequently, the contours of the horizontal joint area and the vertical joint area are respectively extracted, and then a horizontal contour binary image and a vertical contour binary image are obtained, comprehensively evaluating the construction situation from both horizontal and vertical angles. The present invention reflects the width of the joint area at each edge pixel point by obtaining the width at each edge pixel point, thereby constructing local weight images and global weight images in the horizontal and vertical directions. This facilitates increasing the attention to key features in the photovoltaic construction monitoring model, improving the monitoring accuracy of the photovoltaic construction situation, while reducing the manual inspection and management costs and reducing the reliance on manpower. Description of the Drawings

[0058] Figure 1 Is a flowchart of a method for monitoring photovoltaic construction conditions based on images;

[0059] Figure 2 Is a schematic diagram of the horizontal joint area and the vertical joint area;

[0060] Figure 3 Is a schematic diagram of the structure of the photovoltaic construction monitoring model;

[0061] Figure 4 Is a schematic diagram of the structure of the first image feature fusion unit and the second image feature fusion unit. Detailed Embodiments

[0062] The specific embodiments of the present invention will be described below to facilitate those skilled in the art of the present technology to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art of the present technology, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concept of the present invention are within the scope of protection.

[0063] As Figure 1 shown, an image-based monitoring method for photovoltaic construction conditions includes:

[0064] Dividing the construction image into multiple photovoltaic module areas;

[0065] Obtaining the horizontal joint area and the vertical joint area according to adjacent photovoltaic modules;

[0066] Performing binary processing on the construction image according to the horizontal joint area and the vertical joint area respectively to obtain a horizontal contour binary image and a vertical contour binary image;

[0067] Respectively obtaining the widths of the horizontal joint area and the vertical joint area at each edge pixel point, and constructing a horizontal local weight image, a horizontal global weight image, a vertical local weight image, and a vertical global weight image;

[0068] Inputting each binary image and weight image into the photovoltaic construction monitoring model to obtain a photovoltaic construction quality score.

[0069] In this embodiment, the construction image can be taken by a drone or by a worker for the spliced photovoltaic modules.

[0070] In this embodiment, the process of dividing the construction image into multiple photovoltaic module areas includes:

[0071] Calculating the similarity between the pixel value of each pixel point in the construction image and the pixel value of the stored frame pixel. When the similarity is greater than the similarity threshold, the pixel point is classified as a frame pixel point;

[0072] Calculating the similarity between the pixel value of each pixel point in the construction image and the pixel value of the stored battery panel. When the similarity is greater than the similarity threshold, the pixel point is classified as a battery panel pixel point;

[0073] Regarding the connected area formed by each battery panel pixel point as a battery panel area;

[0074] Regarding the connected area formed by each frame pixel point as a frame area;

[0075] When the battery panel area is inside the frame area, regarding the battery panel area and the frame area as a whole as a photovoltaic module area, as Figure 2 shown.

[0076] In this embodiment, the pixel values of each pixel point in the construction image, the pixel values of the storage border, and the pixel values of the storage solar panel all include: R-channel value, G-channel value, and B-channel value, and the cosine similarity is used to calculate the similarity.

[0077] In this embodiment, the similarity threshold can be set to 0.5, and the pixel values with a similarity greater than 0.5 are classified into one area.

[0078] In this embodiment, the process of obtaining the horizontal seam area includes:

[0079] Taking each photovoltaic module as the center, obtaining its upper and lower neighboring photovoltaic modules to get the upper-side photovoltaic module and the lower-side photovoltaic module;

[0080] Marking the pixel points between the central photovoltaic module and the upper-side photovoltaic module as upper seam points;

[0081] Taking the area formed by each upper seam point as a horizontal seam area;

[0082] Marking the pixel points between the central photovoltaic module and the lower-side photovoltaic module as lower seam points;

[0083] Taking the area formed by each lower seam point as a horizontal seam area, as Figure 2 shown.

[0084] When a photovoltaic module at the edge position has only an upper-side photovoltaic module or a lower-side photovoltaic module, only one side needs to be considered.

[0085] In this embodiment, the process of obtaining the vertical seam area includes:

[0086] Taking each photovoltaic module as the center, obtaining its left and right neighboring photovoltaic modules to get the left-side photovoltaic module and the right-side photovoltaic module;

[0087] Marking the pixel points between the central photovoltaic module and the left-side photovoltaic module as left seam points;

[0088] Taking the area formed by each left seam point as a vertical seam area;

[0089] Marking the pixel points between the central photovoltaic module and the right-side photovoltaic module as right seam points;

[0090] Taking the area formed by each right seam point as a vertical seam area, as Figure 2 shown.

[0091] When a photovoltaic module at the edge position has only a left-side photovoltaic module or a right-side photovoltaic module, only one side needs to be considered.

[0092] In this embodiment, the process of obtaining the horizontal contour binary image and the vertical contour binary image includes:

[0093] On the construction image, set the pixel values belonging to the horizontal joint area to 1, and set the pixel values of other pixel points to 0 to obtain the horizontal initial binary image;

[0094] In the horizontal initial binary image, when there are pixel points with pixel value 0 within the neighborhood range of a pixel point with pixel value 1, mark the pixel point with pixel value 1 as an edge pixel point, retain the pixel value of the edge pixel point, and set other pixel values to 0 to obtain the horizontal contour binary image;

[0095] On the construction image, set the pixel values belonging to the vertical joint area to 1, and set other pixel values to 0 to obtain the vertical initial binary image;

[0096] In the vertical initial binary image, when there are pixel points with pixel value 0 within the neighborhood range of a pixel point with pixel value 1, mark the pixel point with pixel value 1 as an edge pixel point, retain the pixel value of the edge pixel point, and set other pixel values to 0 to obtain the vertical contour binary image.

[0097] In this embodiment, the size of the neighborhood range of the pixel point is .

[0098] The present invention highlights the joint area from the complex construction image background quickly and intuitively by setting the pixel values belonging to the horizontal and vertical joint areas to 1 and other pixel values to 0, forming the horizontal initial binary image and the vertical initial binary image. This method greatly simplifies the subsequent process of extracting joint features, avoids the interference of a large amount of irrelevant information, and can accurately focus on the joint area.

[0099] In the initial binary image, by judging the neighborhood pixel values to mark the edge pixel points and only retaining their pixel values, the edge contour of the joint area is effectively highlighted, obtaining the horizontal contour binary image and the vertical contour binary image. This operation makes the boundary features of the joints clearer, which helps the model to more accurately analyze the shape parameters of the joints and judge construction quality problems such as whether the photovoltaic modules are spliced tightly and whether there is misalignment.

[0100] In this embodiment, the process of constructing the horizontal local weight image and the horizontal global weight image includes:

[0101] Extract the edge pixel points for each horizontal joint area, extract the abscissas of the edge pixel points, count the number of pixel points under the same abscissa in the horizontal joint area, and record the number as the horizontal width of the edge pixel point;

[0102] Take the average value of the horizontal widths of all edge pixel points in a horizontal joint area to obtain the horizontal local width average;

[0103] Calculate the difference between the horizontal width of each edge pixel in the horizontal seam region and the mean value of the corresponding horizontal local width, and normalize the difference to obtain the horizontal local deviation value of the corresponding edge pixel;

[0104] Take the mean of all the mean values of the horizontal local widths to obtain the mean value of the horizontal global width;

[0105] Calculate the difference between the horizontal width of each edge pixel in the horizontal seam region and the mean value of the horizontal global width, and normalize the difference to obtain the horizontal global deviation value of the corresponding edge pixel;

[0106] Replace the pixel value of the original edge pixel with the horizontal local deviation value, and set other pixel values to 0 to obtain the horizontal local weight image;

[0107] Replace the pixel value of the original edge pixel with the horizontal global deviation value, and set other pixel values to 0 to obtain the horizontal global weight image.

[0108] In this embodiment, preferably, after obtaining the horizontal width of the edge pixels in the horizontal seam region, the maximum value and the minimum value can be removed to prevent the influence of individual outliers.

[0109] In this embodiment, the formula for obtaining the horizontal local deviation value of the corresponding edge pixel is: , where γ h,p,i,j is the horizontal local deviation value of the j-th edge pixel in the i-th horizontal seam region, w h,i,j is the horizontal width of the j-th edge pixel in the i-th horizontal seam region, w h,i,avg is the mean value of the horizontal local width of the i-th horizontal seam region, w h,i,max is the maximum horizontal width in the i-th horizontal seam region, w h,i,min is the minimum horizontal width in the i-th horizontal seam region, i and j are positive integers, and | | is the absolute value operation.

[0110] In this embodiment, the formula for obtaining the horizontal global deviation value of the corresponding edge pixel is: , where γ h,a,i,j is the horizontal global deviation value of the j-th edge pixel in the i-th horizontal seam region, w h,avg is the mean value of the horizontal global width of the i-th horizontal seam region, w h,max is the maximum horizontal width in all horizontal seam regions, w h,min is the minimum horizontal width in all horizontal seam regions.

[0111] By extracting the abscissas of the edge pixels in the horizontal seam regions, counting the number of pixels at the same abscissa to obtain the horizontal width, and calculating the mean and deviation of the horizontal local widths, the present invention can accurately capture the local feature differences within each horizontal seam region. This method can effectively highlight the deviation degree between the widths at different positions in the seam region and the local average width, which helps to highlight the possible problem areas in the local seams.

[0112] The present invention calculates the mean of all horizontal local widths to obtain the horizontal global width mean, and based on this, obtains the horizontal global deviation value, which can reflect the differences of each horizontal seam region relative to the global average width as a whole. This helps to evaluate the consistency of the horizontal seam widths within the entire construction area, thereby judging the quality stability of the photovoltaic module splicing at the global level and avoiding ignoring the overall trend due to local deviations.

[0113] Using the horizontal local deviation value and the horizontal global deviation value to construct the horizontal local weight image and the horizontal global weight image respectively, when inputting into the subsequent photovoltaic construction monitoring model, it can guide the model to pay more attention to the areas with abnormal seam widths at different scales. The model can more accurately identify the seam regions with large differences from the local or global average conditions, thereby improving the accuracy of the photovoltaic construction quality assessment and effectively detecting potential construction defects.

[0114] In this embodiment, the process of constructing the vertical local weight image and the vertical global weight image includes:

[0115] Extract the edge pixels for each vertical seam region, extract the ordinates of the edge pixels, count the number of pixels at the same ordinate in the vertical seam region, and record the number as the vertical width of the edge pixel;

[0116] Take the mean of the vertical widths of all edge pixels in a vertical seam region to obtain the vertical local width mean;

[0117] Calculate the difference between the vertical width of each edge pixel in the vertical seam region and the corresponding vertical local width mean, and perform normalization processing on the difference to obtain the vertical local deviation value of the corresponding edge pixel;

[0118] Take the mean of all vertical local width means to obtain the vertical global width mean;

[0119] Calculate the difference between the vertical width of each edge pixel in the vertical seam region and the vertical global width mean, and perform normalization processing on the difference to obtain the vertical global deviation value of the corresponding edge pixel;

[0120] Replace the pixel value of the original edge pixel with the vertical local deviation value, and set other pixel values to 0 to obtain the vertical local weight image;

[0121] Replace the pixel values of the original edge pixels with the vertical global deviation values, and set other pixel values to 0 to obtain the vertical global weight image.

[0122] In this embodiment, preferably, after obtaining the vertical width of the edge pixels in the vertical joint area, the maximum value and the minimum value can be removed to prevent the influence of individual abnormal values.

[0123] In this embodiment, the formula for obtaining the vertical local deviation value of the corresponding edge pixel is: , where γ v,p,i,j is the vertical local deviation value of the j-th edge pixel in the i-th vertical joint area, w v,i,j is the vertical width of the j-th edge pixel in the i-th vertical joint area, w v,i,avg is the average value of the vertical local widths of the i-th vertical joint area, w v,i,max is the maximum vertical width in the i-th vertical joint area, w v,i,min is the minimum vertical width in the i-th vertical joint area.

[0124] In this embodiment, the formula for obtaining the vertical global deviation value of the corresponding edge pixel is: , where γ v,a,i,j is the vertical global deviation value of the j-th edge pixel in the i-th vertical joint area, w v,avg is the average value of the vertical global widths of the i-th vertical joint area, w v,max is the maximum vertical width in all vertical joint areas, w v,min is the minimum vertical width in all vertical joint areas.

[0125] The present invention extracts the ordinate of the edge pixels from the vertical joint area, and counts the number of pixels under the same ordinate to determine the vertical width. By calculating the average value of the vertical local widths and the deviation value, the local feature changes within each vertical joint area can be clearly shown. This helps to accurately locate the abnormal fluctuations in the width of the vertical joints at the local positions.

[0126] The present invention obtains the average value of the vertical global widths by taking the average value of all the average values of the vertical local widths, and then calculates the vertical global deviation value, so as to grasp the deviation of each vertical joint area from the global average width as a whole. This provides an important basis for evaluating the consistency of the joint widths in the vertical direction of the entire photovoltaic construction area, helps to judge the stability of the construction quality in the vertical global sense, and avoids ignoring the overall situation due to local problems.

[0127] The vertical local deviation value and the vertical global deviation value are respectively used to construct a vertical local weight image and a vertical global weight image. When inputting into the subsequent photovoltaic construction monitoring model, it can guide the model to focus on the areas where the vertical joint width has a large difference from the local or global average situation. The model can more sensitively capture the quality defects of the joints in the vertical direction, improve the accuracy of the assessment of the photovoltaic construction quality in the vertical dimension, and effectively identify potential construction defects.

[0128] As Figure 3 shown, the photovoltaic construction monitoring model includes: a first image feature fusion unit, a second image feature fusion unit, a first CNN network, a second CNN network, a Concat layer, and a fully connected layer;

[0129] The first image feature fusion unit is used to perform feature fusion on the horizontal contour binary image, the horizontal local weight image, and the horizontal global weight image to obtain a first fusion feature; the second image feature fusion unit is used to perform feature fusion on the vertical contour binary image, the vertical local weight image, and the vertical global weight image to obtain a second fusion feature;

[0130] The first CNN network is used to extract deep features from the first fusion feature; the second CNN network is used to extract deep features from the second fusion feature; the Concat layer is used to splice the deep features output by the first CNN network and the second CNN network to obtain a spliced feature; the fully connected layer is used to output a photovoltaic construction quality score according to the spliced feature.

[0131] In the present invention, the first image feature fusion unit processes the horizontal contour binary image, the horizontal local weight image, and the horizontal global weight image, the second image feature fusion unit processes the vertical contour binary image, the vertical local weight image, and the vertical global weight image, and then respectively extracts deep features through the CNN network to realize feature extraction of two channels, and combines the features of the two channels to evaluate the photovoltaic construction quality score.

[0132] As Figure 4 shown, both the first image feature fusion unit and the second image feature fusion unit include: a first convolutional layer, a second convolutional layer, a third convolutional layer, a multiplier M1, a multiplier M2, and an adder A1;

[0133] The first input end of the multiplier M1 is connected to the output end of the first convolutional layer, and its second input end is connected to the output end of the second convolutional layer; the first input end of the multiplier M2 is connected to the output end of the second convolutional layer, and its second input end is connected to the output end of the third convolutional layer; the first input end of the adder A1 is connected to the output end of the multiplier M1, its second input end is connected to the output end of the multiplier M2, and its output end serves as the output end of the first image feature fusion unit or the second image feature fusion unit.

[0134] In this embodiment, at the input end of the first convolutional layer in the first image feature fusion unit, a horizontal local weight image is input; at the input end of the second convolutional layer, a horizontal contour binary image is input; and at the input end of the third convolutional layer, a horizontal global weight image is input.

[0135] In the second image feature fusion unit, at the input end of the first convolutional layer, a vertical local weight image is input; at the input end of the second convolutional layer, a vertical contour binary image is input; and at the input end of the third convolutional layer, a vertical global weight image is input.

[0136] In the present invention, the convolutional kernels of the first convolutional layer, the second convolutional layer, and the third convolutional layer have the same size, so that the sizes of the extracted image features are the same.

[0137] In the present invention, the first convolutional layer, the second convolutional layer, and the third convolutional layer perform convolutional operations on different types of images (local weight images, contour binary images, global weight images) respectively to extract their respective features. Then, the multiplier M1 multiplies the corresponding features of the local weight image and the contour binary image element by element to increase the attention to the corresponding features according to the local weight. Next, the multiplier M2 multiplies the corresponding features of the global weight image and the contour binary image element by element to increase the attention to the corresponding features according to the global weight. Finally, the adder A1 adds them element by element to achieve feature fusion.

[0138] In this embodiment, for the photovoltaic construction quality score in the range of 90 - 100 points: The photovoltaic modules are installed extremely precisely during construction. The widths of multiple horizontal joint areas are almost the same, the widths of multiple vertical joint areas are almost the same, and the widths are uniform.

[0139] In the range of 70 - 89 points: The overall installation of the photovoltaic modules is qualified. There are slight width changes in some horizontal or vertical joint areas, and there is slight unevenness in splicing during assembly.

[0140] In the range of 60 - 69 points: It reflects that there are certain problems with the photovoltaic construction quality. There are relatively serious width changes in some horizontal or vertical joint areas, and there is relatively serious unevenness in splicing during assembly. The construction party needs to reinstall or adjust the problematic modules.

[0141] In the range of 0 - 59 points: It reflects that the photovoltaic construction quality is worrying. There are serious misalignments in the splicing of a large number of photovoltaic modules, and the widths of the horizontal and vertical joint areas are chaotic. Most of the modules must be reworked and reinstalled, and installed again strictly in accordance with the construction standards.

[0142] The present invention divides the construction image into multiple photovoltaic modules, obtains the horizontal joint area and the vertical joint area, and thereby evaluates the construction quality when each photovoltaic module is spliced. Subsequently, the contours of the horizontal joint area and the vertical joint area are respectively extracted, and then a horizontal contour binary image and a vertical contour binary image are obtained, and the construction situation is comprehensively evaluated from both the horizontal and vertical perspectives. The present invention reflects the width of the joint area at each edge pixel point by obtaining the width at each edge pixel point, thereby constructing local weight images and a global weight image in the horizontal and vertical directions. This facilitates increasing the attention to key features in the photovoltaic construction monitoring model, improving the monitoring accuracy of the photovoltaic construction situation, reducing the manual inspection and management costs at the same time, and reducing the reliance on manpower.

[0143] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image-based monitoring method for photovoltaic construction conditions, characterized in that Including: Dividing the construction image into multiple photovoltaic module areas; Obtaining the horizontal seam area and the vertical seam area according to adjacent photovoltaic modules; Performing binary processing on the construction image respectively according to the horizontal seam area and the vertical seam area to obtain a horizontal contour binary image and a vertical contour binary image; Respectively obtaining the widths of the horizontal seam area and the vertical seam area at each edge pixel point, and constructing a horizontal local weight image, a horizontal global weight image, a vertical local weight image, and a vertical global weight image; The process of constructing the horizontal local weight image and the horizontal global weight image includes: Extracting edge pixel points for each horizontal seam area, extracting the abscissas of the edge pixel points, counting the number of pixel points with the same abscissa in the horizontal seam area, and recording the number as the horizontal width of the edge pixel point; Taking the average of the horizontal widths of all edge pixel points in a horizontal seam area to obtain the horizontal local width average value; Calculating the difference between the horizontal width of each edge pixel point in the horizontal seam area and the corresponding horizontal local width average value, and performing normalization processing on the difference to obtain the horizontal local deviation value of the corresponding edge pixel point; Taking the average of all horizontal local width average values to obtain the horizontal global width average value; Calculating the difference between the horizontal width of each edge pixel point in the horizontal seam area and the horizontal global width average value, and performing normalization processing on the difference to obtain the horizontal global deviation value of the corresponding edge pixel point; Replacing the pixel value of the original edge pixel point with the horizontal local deviation value, and setting other pixel values to 0 to obtain the horizontal local weight image; Replacing the pixel value of the original edge pixel point with the horizontal global deviation value, and setting other pixel values to 0 to obtain the horizontal global weight image; The process of constructing the vertical local weight image and the vertical global weight image includes: Extracting edge pixel points for each vertical seam area, extracting the ordinates of the edge pixel points, counting the number of pixel points with the same ordinate in the vertical seam area, and recording the number as the vertical width of the edge pixel point; Taking the average of the vertical widths of all edge pixel points in a vertical seam area to obtain the vertical local width average value; Calculating the difference between the vertical width of each edge pixel point in the vertical seam area and the corresponding vertical local width average value, and performing normalization processing on the difference to obtain the vertical local deviation value of the corresponding edge pixel point; Taking the average of all vertical local width average values to obtain the vertical global width average value; Calculating the difference between the vertical width of each edge pixel point in the vertical seam area and the vertical global width average value, and performing normalization processing on the difference to obtain the vertical global deviation value of the corresponding edge pixel point; Replacing the pixel value of the original edge pixel point with the vertical local deviation value, and setting other pixel values to 0 to obtain the vertical local weight image; Replacing the pixel value of the original edge pixel point with the vertical global deviation value, and setting other pixel values to 0 to obtain the vertical global weight image; Inputting each binary image and weight image into the photovoltaic construction monitoring model to obtain the photovoltaic construction quality score; The photovoltaic construction monitoring model includes: a first image feature fusion unit, a second image feature fusion unit, a first CNN network, a second CNN network, a Concat layer, and a fully connected layer; the first image feature fusion unit is used to perform feature fusion on the horizontal contour binary image, the horizontal local weight image, and the horizontal global weight image to obtain a first fusion feature; the second image feature fusion unit is used to perform feature fusion on the vertical contour binary image, the vertical local weight image, and the vertical global weight image to obtain a second fusion feature; The first CNN network is used to extract deep features from the first fusion feature; the second CNN network is used to extract deep features from the second fusion feature; the Concat layer is used to splice the deep features output by the first CNN network and the second CNN network to obtain a spliced feature; the fully connected layer is used to output the photovoltaic construction quality score according to the spliced feature.

2. The method for monitoring the photovoltaic construction situation based on images according to claim 1, wherein The process of dividing the construction image into multiple photovoltaic module areas includes: Calculating the similarity between the pixel value of each pixel point in the construction image and the pixel value of the stored border pixel. When the similarity is greater than the similarity threshold, the pixel point is classified as a border pixel point; Calculating the similarity between the pixel value of each pixel point in the construction image and the pixel value of the stored solar panel pixel. When the similarity is greater than the similarity threshold, the pixel point is classified as a solar panel pixel point; Regarding the connected area formed by each solar panel pixel point as a solar panel area; Regarding the connected area formed by each border pixel point as a border area; When the solar panel area is inside the border area, regarding the solar panel area and the border area as a whole as a photovoltaic module area.

3. The method for monitoring the photovoltaic construction situation based on images according to claim 1, characterized in that The process of obtaining the horizontal seam area includes: Taking each photovoltaic module as the center, obtaining its upper and lower neighboring photovoltaic modules to get the upper-side photovoltaic module and the lower-side photovoltaic module; Marking the pixel points between the central photovoltaic module and the upper-side photovoltaic module as upper seam points; Regarding the area formed by each upper seam point as a horizontal seam area; Marking the pixel points between the central photovoltaic module and the lower-side photovoltaic module as lower seam points; Regarding the area formed by each lower seam point as a horizontal seam area.

4. The method for monitoring the photovoltaic construction situation based on images according to claim 1, wherein, The process of obtaining the vertical seam area includes: Taking each photovoltaic module as the center, obtaining its left and right neighboring photovoltaic modules to get the left-side photovoltaic module and the right-side photovoltaic module; Marking the pixel points between the central photovoltaic module and the left-side photovoltaic module as left seam points; Regarding the area formed by each left seam point as a vertical seam area; Marking the pixel points between the central photovoltaic module and the right-side photovoltaic module as right seam points; Regarding the area formed by each right seam point as a vertical seam area.

5. The method for monitoring the photovoltaic construction situation based on images according to claim 1, wherein The process of obtaining the horizontal contour binary image and the vertical contour binary image includes: Setting the pixel value of the pixel points belonging to the horizontal seam area to 1 on the construction image and setting the pixel values of other pixel points to 0 to obtain a horizontal initial binary image; In the horizontal initial binary image, when there are pixel points with pixel value 0 within the neighborhood range of a pixel point with pixel value 1, marking the pixel point with pixel value 1 as an edge pixel point, retaining the pixel value of the edge pixel point, and setting other pixel values to 0 to obtain the horizontal contour binary image; Set the pixel values belonging to the vertical joint area to 1 on the construction image, and set the other pixel values to 0 to obtain the vertical initial binary image; When there are pixel points with pixel value 0 within the neighborhood range of a pixel point with pixel value 1 in the vertical initial binary image, mark the pixel point with pixel value 1 as an edge pixel point, retain the pixel value of the edge pixel point, and set the other pixel values to 0 to obtain the vertical contour binary image.

6. The method for monitoring the photovoltaic construction situation based on images according to claim 1, wherein, Both the first image feature fusion unit and the second image feature fusion unit include: a first convolutional layer, a second convolutional layer, a third convolutional layer, a multiplier M1, a multiplier M2, and an adder A1; The first input end of the multiplier M1 is connected to the output end of the first convolutional layer, and its second input end is connected to the output end of the second convolutional layer; the first input end of the multiplier M2 is connected to the output end of the second convolutional layer, and its second input end is connected to the output end of the third convolutional layer; the first input end of the adder A1 is connected to the output end of the multiplier M1, its second input end is connected to the output end of the multiplier M2, and its output end serves as the output end of the first image feature fusion unit or the second image feature fusion unit.

7. The method for monitoring the photovoltaic construction situation based on images according to claim 6, wherein In the first image feature fusion unit, the input end of the first convolutional layer is used to input the horizontal local weight image, the input end of the second convolutional layer is used for the horizontal contour binary image, and the input end of the third convolutional layer is used to input the horizontal global weight image; In the second image feature fusion unit, the input end of the first convolutional layer is used to input the vertical local weight image, the input end of the second convolutional layer is used for the vertical contour binary image, and the input end of the third convolutional layer is used to input the vertical global weight image.

Images