A stain posture recognition method for intelligent cleaning of photovoltaic panels

Abstract

The present invention belongs to the field of photovoltaic panel cleaning, and provides a stain posture recognition method for intelligent cleaning of photovoltaic panels, comprising the following steps: step 1, deploying a mobile operating robot in a photovoltaic power station, the robot carrying a camera to scan the photovoltaic panel to obtain a stain image; step 2, accurately identifying the area where the stain is located in the stain image obtained in step 1 and classifying the stain; step 3, performing edge detection on the image of the area where the stain is located identified in step 2 by an adaptive edge detection algorithm to obtain the edge of the stain; step 4, using the image obtained by edge detection to draw the outer enclosing contour of the stain using a tiltable minimum circumscribed rectangular frame, and the maximum length, maximum width and deflection angle are the stain posture. Through the method of the present invention, the posture of the stain can be accurately identified, providing accurate data for subsequent automatic cleaning.

Description

Technical Field

[0001] The present invention relates to the field of photovoltaic panel cleaning, and in particular to a stain posture recognition method for intelligent photovoltaic panel cleaning. Background Art

[0002] With the advancement of human science and technology and the growth of the global economy, robotics technology has risen rapidly. Robots have been used in various fields to assist or replace human work to varying degrees. Against this background, the operation and maintenance management model of photovoltaic power stations is gradually changing from rough to refined. The unmanned, less-manned, standardized, technological, and intelligent operation and maintenance management model based on robotics technology has become the development direction of the future operation and maintenance industry. That is, based on system safety, preventive maintenance, periodic maintenance, and regular equipment performance testing methods are used to scientifically and rationally maintain and manage photovoltaic power stations, involving fault detection, equipment maintenance and upgrades, component cleaning, dust removal and other services to ensure the safe, stable, and efficient operation of the entire photovoltaic power generation system, thereby improving investors' investment returns.

[0003] At present, the maintenance of photovoltaic power stations, cleaning of fixed stains and other operation and maintenance work are mainly completed manually. There are many problems in related work efficiency and personnel safety. Moreover, with the increase in installed capacity of photovoltaic power stations, the operation and maintenance work content becomes more cumbersome and the task intensity becomes higher. Manual maintenance is more time-consuming and manpower-intensive, and the maintenance cost increases significantly. The above-mentioned problems will also become more prominent. Summary of the invention

[0004] The main purpose of the present invention is to provide a stain posture recognition method for intelligent cleaning of photovoltaic panels, which is used to identify stains on photovoltaic panels to obtain data on the stains on the photovoltaic panels for subsequent cleaning.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is: a stain posture recognition method for intelligent cleaning of photovoltaic panels, comprising the following steps:

[0006] Step 1: deploy a mobile operating robot in the photovoltaic power station, and the robot carries a camera to scan the photovoltaic panel to obtain the stain image;

[0007] Step 2: Use the improved YOLOv5 algorithm model based on the pytorch framework to accurately identify the area where the stains are located in the stain image obtained in step 1 and classify the stains. Each stain area is represented by a rectangular target bounding box. The length and width directions of the target bounding box are parallel to the predetermined direction. The position of each stain area is represented by the upper left point coordinates and the lower right point coordinates of the target bounding box. The improved YOLOv5 algorithm model includes an input layer, a backbone network, an intermediate layer and a prediction layer.

[0008] Step 3: Perform edge detection on the image of the area where the stain is located identified in step 2 by using an adaptive edge detection algorithm to obtain the edge of the stain;

[0009] Step 4: Use a tiltable minimum circumscribed rectangular frame to draw the outer contour of the stain on the image obtained through edge detection, and calculate the inclination angle of the minimum circumscribed rectangular frame. The length and width of the minimum circumscribed rectangular frame are the corresponding maximum length and maximum width of the stain, and the inclination angle is the deflection angle of the length direction or width direction of the corresponding stain relative to the reference direction. The maximum length, maximum width and deflection angle are the stain posture.

[0010] Preferably, step 2 specifically includes the following steps:

[0011] Step 21, input the stain image obtained in step 1 into the improved YOLOv5 algorithm recognition model;

[0012] Step 22: The backbone network extracts features from the stain image to obtain a feature map;

[0013] Step 23: The middle layer adopts the structure of FPN+PAN to fuse the feature maps to obtain feature representations corresponding to different targets;

[0014] Step 24: The prediction layer determines the position and type of the target based on the feature representations corresponding to different targets obtained in step 23. The position of the target is the position of the target bounding box.

[0015] Preferably, the step three specifically includes the following steps:

[0016] Step 31, converting the image of the area covered by the target bounding box into a grayscale image, and adding salt and pepper noise to the grayscale image;

[0017] Step 32, performing a top hat transformation on the image obtained in step 31: performing an opening operation on the image obtained in step 31, wherein the opening operation is to first perform an erosion operation on the image and then perform an expansion operation to remove noise in the image and smooth the boundary of the object, and subtract the result of the opening operation from the image obtained in step 31 to highlight the small-scale features or local brightness changes in the image obtained in step 31;

[0018] Step 33, performing black hat transformation on the image obtained in step 32: performing a closing operation on the image obtained in step 32, wherein the closing operation is to first perform an expansion operation on the image and then perform an erosion operation to fill small holes inside the object and smooth the boundary of the object, and subtract the result of the closing operation from the image in step 32 to highlight small-scale dark features or local darkness changes in the image;

[0019] Step 34, performing median filtering on the image after top hat transformation and black hat transformation;

[0020] Step 35: Calculate the gradient on the filtered image using the operator of the edge detection algorithm to find the gradient value and direction of each pixel in the image;

[0021] Step 36: using a double threshold algorithm according to the obtained gradient value, taking the pixel points whose gradient values ​​are greater than or equal to the high threshold as strong edges, taking the pixel points whose gradient values ​​are less than the high threshold and greater than or equal to the low threshold as intermediate edges, taking the pixel points whose gradient values ​​are less than the low threshold as weak edges, suppressing the weak edges, connecting the strong edges, connecting the intermediate edges connected to the strong edges to the strong edges, and suppressing the intermediate edges not connected to the strong edges, to obtain edges;

[0022] Step 37: superimpose the obtained edge result on the image obtained in step 2, so as to find the position where the intensity change exceeds the predetermined value in each stain area, and extract the edge information of the stain area.

[0023] Preferably, the image size output in step 34 is assumed to be M×N, and the set represents the L gray levels of the image, Indicates the gray level is The number of pixels in the image is:

[0024] ,

[0025] The pixel value is Probability for:

[0026] ;

[0027] Set a threshold , use this threshold to classify the image into foreground With background , The gray value range is , The gray value range is , the pixel belongs to the foreground Probability for:

[0028] ,

[0029] Pixels belong to the background Probability Then:

[0030] ,

[0031] The average grayscale value of pixels in for:

[0032] ,

[0033] The average grayscale value of pixels in for:

[0034] ,

[0035] The average gray value of the pixel values ​​with gray levels from 0 to k for:

[0036] ,

[0037] The average gray value of the image for:

[0038] ,

[0039] From this we can get:

[0040] ,

[0041] The inter-class variance formula can be deduced as:

[0042] ,in is the between-class variance;

[0043] According to the above formula, the prospect With background The threshold for maximum inter-class variance ,make is the high threshold for edge detection, and the low threshold is set to .

[0044] Preferably, step 4 specifically includes the following steps:

[0045] Step 41, converting the image obtained in step 3 into a grayscale image using the Canny edge detection method;

[0046] Step 42, performing contour detection on the grayscale image obtained in step 41 to find all contours;

[0047] Step 43, approximating each contour, setting a pixel threshold, retaining contours whose pixels covered by the contours are greater than the pixel threshold, and filtering out contours whose pixels covered by the contours are less than the pixel threshold;

[0048] Step 44: For the screened contour, a rotating rectangle fitting algorithm is used to fit the minimum circumscribed rectangle that can be tilted;

[0049] Step 45: Use a minimum bounding rectangle calculator to obtain rectangle parameters of the minimum bounding rectangle frame, where the rectangle parameters include tilt angle information relative to the reference direction, center coordinates of the rectangle, and width and height of the rectangle;

[0050] Step 46: Apply the calculated rectangular parameters to the original image, draw a tiltable minimum bounding rectangular frame, and determine the stain posture according to the parameters of the minimum bounding rectangular frame.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The present invention can accurately identify the boundary, type, etc. of the stain by processing the captured image. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0054] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0055] A stain posture recognition method for intelligent cleaning of photovoltaic panels comprises the following steps:

[0056] Step 1: deploy a mobile operating robot in the photovoltaic power station, the robot carries a sensor to scan the photovoltaic panel to obtain a stain image, and pre-processes the obtained stain image;

[0057] Step 2: Use the improved YOLOv5 algorithm model based on the pytorch framework to accurately identify the area where the stains are located in the stain image preprocessed in step 1 and classify the stains. Each stain area is represented by a rectangular target bounding box. The length and width directions of the target bounding box are parallel to the predetermined direction. The position of each stain area is represented by the upper left point coordinates and the lower right point coordinates of the target bounding box. The improved YOLOv5 algorithm model includes an input layer, a backbone network, an intermediate layer and a prediction layer; obtain different stain images of the photovoltaic panel, mark the type of stain and the target bounding box as labels, and use the different stain images with labels as training samples to train the YOLOv5 algorithm model to obtain the improved YOLOv5 algorithm model;

[0058] Step 3: Perform edge detection on the image of the area where the stain identified in step 2 is located by using an adaptive edge detection algorithm;

[0059] Step 4: Use a tiltable minimum circumscribed rectangular frame to draw the outer contour of the stain on the image obtained through edge detection, and calculate the inclination angle of the minimum circumscribed rectangular frame. The length and width of the minimum circumscribed rectangular frame are the corresponding maximum length and maximum width of the stain, and the inclination angle is the deflection angle of the length direction or width direction of the corresponding stain relative to the reference direction. The maximum length, maximum width and deflection angle are the stain posture.

[0060] Step 1 specifically includes the following steps:

[0061] Step 11: deploy a mobile operating robot in the photovoltaic power station. The robot carries a high-resolution camera and uses a path planning algorithm to determine the moving path of the mobile operating robot in the photovoltaic power station. The photovoltaic panels are scanned and the camera captures high-resolution images.

[0062] Step 2 specifically includes the following steps:

[0063] Step 21, input the stain image obtained in step 1 into the improved YOLOv5 algorithm recognition model;

[0064] Step 22: The backbone network extracts features from the stain image to obtain a feature map;

[0065] Step 23: The middle layer adopts the structure of FPN+PAN to fuse the feature maps to obtain feature representations corresponding to different targets;

[0066] Step 24: The prediction layer determines the position and type of the target according to the feature representation corresponding to different targets obtained in step 23, and the position of the target is the position of the target bounding box;

[0067] Step 25: Using the built-in parameters of the camera, the two-dimensional position of the target in the image is converted into three-dimensional coordinates, and further converted into a position in the world coordinate system, so as to calculate the three-dimensional translation information of the target relative to the camera, and then calculate the distance between the stain and the camera by triangulation, and finally obtain the actual position of the stain.

[0068] The step 2 may also include step 26, where the output end uses the MPDIoU_Loss loss function instead of the GIOU_Loss to measure the accuracy of the predicted center position information;

[0069]

[0070] In the formula, is the MPDIoU loss function, represents the predicted box coordinates, represents the real box coordinates, and represents the width and height of the input feature, is the distance between the upper left point coordinates of the predicted box and the upper left point coordinates of the real box, The distance between the coordinates of the lower right point of the predicted box and the coordinates of the lower right point of the real box. The position of the target is represented by a bounding box. The predicted box is the bounding box of the predicted target, and the real box is the bounding box of the real target. The larger the value, the greater the deviation and the less accurate it is.

[0071] The step 26 can be used during model training or in actual recognition. In actual recognition, step 26 can be used once after a certain period, and the user can make appropriate adjustments to the parameters in the model.

[0072] The step three specifically includes the following steps:

[0073] Step 31, converting the image of the area covered by the target bounding box into a grayscale image, and adding salt and pepper noise to the grayscale image;

[0074] Step 32, performing a top hat transformation on the image obtained in step 31: performing an opening operation on the image obtained in step 31, wherein the opening operation is to first perform an erosion operation on the image and then perform an expansion operation to remove noise in the image and smooth the boundary of the object, and subtract the result of the opening operation from the image obtained in step 31 to highlight the small-scale features or local brightness changes in the image obtained in step 31;

[0075] Step 33, performing black hat transformation on the image obtained in step 32: performing a closing operation on the image obtained in step 32, wherein the closing operation is to first perform an expansion operation on the image and then perform an erosion operation to fill small holes inside the object and smooth the boundary of the object, and subtract the result of the closing operation from the image in step 32 to highlight small-scale dark features or local darkness changes in the image;

[0076] Step 34, performing median filtering on the image after top hat transformation and black hat transformation to remove the remaining noise;

[0077] Step 35: Calculate the gradient on the filtered image using the operator of the edge detection algorithm to find the gradient value and direction of each pixel in the image;

[0078] Step 36: using a double threshold algorithm according to the obtained gradient value, taking the pixel points whose gradient values ​​are greater than or equal to the high threshold as strong edges, taking the pixel points whose gradient values ​​are less than the high threshold and greater than or equal to the low threshold as intermediate edges, taking the pixel points whose gradient values ​​are lower than the low threshold as weak edges, suppressing the weak edges, connecting the strong edges, connecting the intermediate edges connected to the strong edges to the strong edges, and suppressing the intermediate edges not connected to the strong edges, to obtain edges;

[0079] Step 37: superimpose the obtained edge result on the image obtained in step 2, so as to find the position where the intensity change exceeds the predetermined value in each stain area, and extract the edge information of the stain area.

[0080] The high threshold and the low threshold in step 36 are adaptively obtained using the Otsu algorithm. The specific steps are as follows:

[0081] Assume that the image size output in step 34 is M×N, and the set represents the L gray levels of the image, Indicates the gray level is The number of pixels in the image is:

[0082] ,

[0083] The pixel value is Probability for:

[0084] ;

[0085] Set a threshold , use this threshold to classify the image into foreground With background , The gray value range is , The gray value range is , the pixel belongs to the foreground Probability for:

[0086] ,

[0087] Pixels belong to the background Probability Then:

[0088] ,

[0089] The average grayscale value of pixels in for:

[0090] ,

[0091] The average grayscale value of pixels in for:

[0092] ,

[0093] The average gray value of the pixel values ​​with gray levels from 0 to k for:

[0094] ,

[0095] The average gray value of the image for:

[0096] ,

[0097] From this we can get:

[0098] ,

[0099] The inter-class variance formula can be deduced as:

[0100] ,

[0101] According to the above formula, the prospect With background The threshold for maximum inter-class variance ,make is the high threshold for edge detection, and the low threshold is set to .

[0102] Step 4 specifically includes the following steps:

[0103] Step 41: Use the Canny edge detection method to convert the stain image obtained in step 3 into a grayscale image, and highlight the edge of the area where the stain is located;

[0104] Step 42, use the functions provided by image processing databases such as OpenCV to perform contour detection on the grayscale image and find all contours;

[0105] Step 43, approximating each contour, setting a pixel threshold, retaining contours whose pixels covered by the contours are greater than the pixel threshold, filtering out contours whose pixels covered by the contours are less than the pixel threshold, and reducing redundant points in the contours into a more compact representation;

[0106] Step 44: For the selected contours, since the stains are not necessarily horizontal or vertical in the image, in some cases, the stains on the photovoltaic panel are tilted or rotated due to the shooting angle or other factors during the image acquisition process, a rotating rectangle (the minimum rectangle surrounding the contour, with the boundary consistent with the contour direction) fitting algorithm is used to fit the tiltable minimum circumscribed rectangle;

[0107] Step 45: Use a minimum bounding rectangle calculator to obtain rectangle parameters of the minimum bounding rectangle frame, where the rectangle parameters include tilt angle information relative to the reference direction, center coordinates of the rectangle, and width and height of the rectangle;

[0108] Step 46: Apply the calculated rectangular parameters to the original image, draw a tiltable minimum bounding rectangular frame, and determine the stain posture according to the parameters of the minimum bounding rectangular frame.

[0109] In practice, the type of target obtained in step 24, the actual position obtained in step 25, and the stain posture obtained in step 4 can be used as stain data to facilitate subsequent automatic cleaning using a cleaning robot.

[0110] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A stain posture recognition method for intelligent cleaning of photovoltaic panels, characterized in that: The steps include: Step 1: deploy a mobile operating robot in the photovoltaic power station, and the robot carries a camera to scan the photovoltaic panel to obtain the stain image; Step 2: Use the improved YOLOv5 algorithm model based on the pytorch framework to accurately identify the area where the stains are located in the stain image obtained in step 1 and classify the stains. Each stain area is represented by a rectangular target bounding box. The length and width directions of the target bounding box are parallel to the predetermined direction. The position of each stain area is represented by the upper left point coordinates and the lower right point coordinates of the target bounding box. The improved YOLOv5 algorithm model includes an input layer, a backbone network, an intermediate layer and a prediction layer. Step 3: Perform edge detection on the image of the area where the stain is located identified in step 2 by using an adaptive edge detection algorithm to obtain the edge of the stain; Step 4: Use a tiltable minimum circumscribed rectangular frame to draw the outer contour of the stain on the image obtained through edge detection, and calculate the tilt angle of the minimum circumscribed rectangular frame. The length and width of the minimum circumscribed rectangular frame are the maximum length and maximum width of the corresponding stain, and the tilt angle is the deflection angle of the length direction or width direction of the corresponding stain relative to the reference direction. The maximum length, maximum width and deflection angle are the stain posture; The step three specifically includes the following steps: Step 31, converting the image of the area covered by the target bounding box into a grayscale image, and adding salt and pepper noise to the grayscale image; Step 32, performing a top hat transformation on the image obtained in step 31: performing an opening operation on the image obtained in step 31, wherein the opening operation is to first perform an erosion operation on the image and then perform an expansion operation to remove noise in the image and smooth the boundary of the object, and subtract the result of the opening operation from the image obtained in step 31 to highlight the small-scale features or local brightness changes in the image obtained in step 31; Step 33, performing black hat transformation on the image obtained in step 32: performing a closing operation on the image obtained in step 32, wherein the closing operation is to first perform an expansion operation on the image and then perform an erosion operation to fill small holes inside the object and smooth the boundary of the object, and subtract the result of the closing operation from the image obtained in step 32 to highlight small-scale dark features or local darkness changes in the image; Step 34, performing median filtering on the image after top hat transformation and black hat transformation; Step 35: Calculate the gradient on the filtered image using the operator of the edge detection algorithm to find the gradient value and direction of each pixel in the image; Step 36: using a double threshold algorithm according to the obtained gradient value, taking the pixel points whose gradient values ​​are greater than or equal to the high threshold as strong edges, taking the pixel points whose gradient values ​​are less than the high threshold and greater than or equal to the low threshold as intermediate edges, taking the pixel points whose gradient values ​​are less than the low threshold as weak edges, suppressing the weak edges, connecting the strong edges, connecting the intermediate edges connected to the strong edges to the strong edges, and suppressing the intermediate edges not connected to the strong edges, to obtain edges; Step 37, superimposing the obtained edge result on the image obtained in step 2, thereby finding the position where the intensity change exceeds a predetermined value in each stain area, and extracting the edge information of the stain area; Step 4 specifically includes the following steps: Step 41, converting the image obtained in step 3 into a grayscale image using the Canny edge detection method; Step 42, performing contour detection on the grayscale image obtained in step 41 to find all contours; Step 43, approximating each contour, setting a pixel threshold, retaining contours whose pixels covered by the contours are greater than the pixel threshold, and filtering out contours whose pixels covered by the contours are less than the pixel threshold; Step 44: For the selected contour, a rotating rectangle fitting algorithm is used to fit the minimum circumscribed rectangular frame that can be tilted; Step 45: Use a minimum bounding rectangle calculator to obtain rectangle parameters of the minimum bounding rectangle frame, where the rectangle parameters include tilt angle information relative to the reference direction, center coordinates of the rectangle, and width and height of the rectangle; Step 46: Apply the calculated rectangular parameters to the original image, draw a tiltable minimum bounding rectangular frame, and determine the stain posture according to the parameters of the minimum bounding rectangular frame.

2. A stain posture recognition method for intelligent cleaning of photovoltaic panels according to claim 1, characterized in that: Step 2 specifically includes the following steps: Step 21, input the stain image obtained in step 1 into the improved YOLOv5 algorithm recognition model; Step 22: The backbone network extracts features from the stain image to obtain a feature map; Step 23: The middle layer adopts the structure of FPN+PAN to fuse the feature maps to obtain feature representations corresponding to different targets; Step 24: The prediction layer determines the position and type of the target based on the feature representation corresponding to different targets obtained in step 23. The position of the target is the position of the target bounding box.

3. The stain posture recognition method for intelligent cleaning of photovoltaic panels according to claim 1 is characterized in that: The high threshold and the low threshold in step 36 are adaptively obtained using the OTSU algorithm. The specific steps are as follows: Assume that the image size output in step 34 is M×N, and the set represents the L gray levels of the image, Indicates the gray level is The number of pixels in the image is: ， The pixel value is Probability for: ； Set a threshold , use this threshold to classify the image into foreground With background , The gray value range is , The gray value range is , the pixel belongs to the foreground Probability for: ， Pixels belong to the background Probability Then: ， The average grayscale value of pixels in for: ， The average grayscale value of pixels in for: ， The average gray value of the pixel values ​​with gray levels from 0 to k for: ， The average gray value of the image for: ， From this we can get: ， The inter-class variance formula can be deduced as: ,in is the between-class variance; According to the above formula, the prospect With background The threshold for maximum inter-class variance ,make is the high threshold for edge detection, and the low threshold is set to .

Images