Vision-based photovoltaic panel cleaning path planning method
By improving the YOLOv5 detection model and two-dimensional linked list node strategy, the problem of insufficient adaptability of traditional path planning in dynamic environments is solved, and efficient path planning and coverage integrity of photovoltaic cleaning robots is achieved.
Patent Information
- Application Number
- CN202411201987.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-06
AI Technical Summary
Traditional path planning algorithms rely on establishing global maps, resulting in the separation of perception and decision-making, it is difficult to adapt to dynamically changing environments, and lack sufficient adaptability and robustness in complex environments.
By improving the YOLOv5 detection model, combining the MobileNetV3-Small network and CBAM attention mechanism, optimizing photovoltaic panel boundary detection, using the camera to obtain images, identify photovoltaic panel boundary type information, and real-time path planning is carried out through two-dimensional linked list nodes and boundary priority strategies.
The path planning of photovoltaic cleaning robots is achieved in dynamically changing environments, ensuring cleaning efficiency and coverage integrity.
Smart Images

Figure CN119085676A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of photovoltaic panel cleaning, and in particular to a vision-based photovoltaic panel cleaning path planning method. Background Art
[0002] The photovoltaic cleaning robot must completely replace the power plant staff to complete the cleaning of solar photovoltaic panels. The first problem to be solved is the path planning problem of the photovoltaic cleaning robot. The path planning of the photovoltaic cleaning robot is to find an optimal path from the starting node to the working target point based on the known geographical environment information of the solar power plant and the constraints of the photovoltaic cleaning robot itself. Whether the final route meets the requirements of the photovoltaic cleaning robot and environmental conditions, whether the planning time and space occupancy are minimized, and whether the route is optimal, all directly affect the on-site operation status and work efficiency of the photovoltaic cleaning robot. The main problems facing path planning at present include difficulty in environmental modeling, slow algorithm convergence, and easy to fall into local optimal solutions. In addition, traditional path planning algorithms usually rely on the establishment of a global map and perform path planning based on a known map, which leads to the separation of perception and decision-making and is difficult to adapt to dynamically changing environments.
[0003] There are currently two main types of algorithms used in the field of boundary detection: two-stage detectors and single-stage detectors. Two-stage detectors, such as MaskR-CNN and FasterR-CNN, first generate candidate regions (RegionProposals) and then accurately locate and classify these regions. In contrast, single-stage detectors, such as SSD and YOLO, take a more direct approach. They use fully convolutional neural networks (FCN) to directly predict the location, confidence score, and type probability of objects from the input image in parallel. This process does not require the generation of candidate regions, realizing an end-to-end detection process. Although single-stage detectors stand out for their excellent speed advantage, their detection accuracy may sometimes be slightly inferior to that of two-stage detectors.
[0004] Path planning algorithms are computational methods and technologies to solve this problem. They are widely used in automation, photovoltaic cleaning robots, aerospace, transportation and other fields, and are of great significance in practical applications such as unmanned driving and intelligent navigation. At the same time, path planning algorithms are also the product of the intersection of different fields such as computational geometry, optimization algorithms and artificial intelligence. Although these algorithms have achieved remarkable success in practical applications, the currently widely used algorithms still face many challenges due to complex factors such as vehicle interactions and environmental changes. Therefore, researchers need to optimize and improve existing algorithms to make them more adaptable and robust in more complex environments. To this end, a vision-based photovoltaic panel cleaning path planning method is proposed. Summary of the invention
[0005] The technical problem to be solved by the present invention is: how to solve the problem that traditional path planning algorithms usually rely on establishing a global map, resulting in the separation of perception and decision-making, and are difficult to adapt to dynamically changing environments, and to optimize and improve existing path planning algorithms so that they have stronger adaptability and robustness in more complex environments. A vision-based photovoltaic panel cleaning path planning method is provided, which acquires images through a camera in a visual component, and judges various types of photovoltaic panel boundaries through an improved YOLOv5 detection model, and then performs path planning based on the photovoltaic panel boundary type information.
[0006] The present invention solves the above technical problems through the following technical solutions, and the present invention comprises the following steps:
[0007] S1: Detection model improvement
[0008] The image data of the photovoltaic panel boundary is collected and processed to obtain a data set; on the basis of the original YOLOv5 detection model, the MobileNetV3-Small network is introduced and combined with the CBAM attention mechanism module to optimize the model structure, and the CSP module in the original YOLOv5 detection model is adjusted to obtain an improved YOLOv5 detection model; then the improved YOLOv5 detection model is trained to obtain a trained model, namely the photovoltaic panel boundary detection model; finally, the photovoltaic panel boundary detection model that meets the performance requirements is deployed in the photovoltaic cleaning robot;
[0009] S2: Path planning and cleaning
[0010] Determine the starting position of the photovoltaic cleaning robot and set the origin; during the cleaning process, the photovoltaic cleaning robot identifies the direction of the photovoltaic cleaning robot through the gyroscope, and obtains the displacement information of the photovoltaic cleaning robot through the accelerometer calculation, and at the same time obtains images through the camera, and transmits the images to the photovoltaic panel boundary detection model, identifies the boundary type information of the photovoltaic panel in front of the photovoltaic cleaning robot, converts the identified boundary type information and displacement information into two-dimensional linked list nodes, performs cleaning work according to the two-dimensional linked list nodes and the boundary priority strategy, and realizes real-time path planning during the cleaning process.
[0011] Furthermore, in step S1, the specific processing process is as follows:
[0012] S11: Data collection, preprocessing and enhancement
[0013] Collect image data of the photovoltaic panel boundary, preprocess the image to obtain the preprocessed image, constitute the initial photovoltaic panel boundary data set, and then perform data enhancement processing on the photovoltaic panel boundary data set to obtain the final photovoltaic panel boundary data set; wherein the preprocessing includes size standardization, random flipping and cropping;
[0014] S12: Dataset annotation and division
[0015] Use annotation tools to annotate each image in the final photovoltaic panel boundary dataset, marking the location and type of the photovoltaic panel boundary; after the annotation is completed, the dataset is divided into a training set, a test set, and a validation set according to the set ratio;
[0016] S13: Model construction
[0017] On the basis of the original YOLOv5 detection model, the MobileNetV3-Small network is introduced and combined with the CBAM attention mechanism module to optimize the model structure. At the same time, the CSP module in the original YOLOv5 detection model is adjusted to obtain the improved YOLOv5 detection model.
[0018] S14: Model training and evaluation
[0019] The improved YOLOv5 detection model is trained using the divided training set. After the training is completed, the model is preliminarily evaluated using the test set, and the photovoltaic panel boundary detection model that has passed the preliminary evaluation is saved.
[0020] S15: Model application and optimization
[0021] Based on the test results obtained through the validation set, the performance of the photovoltaic panel boundary detection model that has passed the preliminary evaluation is comprehensively evaluated, and a photovoltaic panel boundary detection model that meets the performance requirements is obtained. The model is then deployed in the photovoltaic cleaning robot controller, which is communicated with the camera in the visual component.
[0022] Furthermore, in step S11, the photovoltaic panel boundary types include left boundary, right boundary, left corner, right corner, top boundary and normal area.
[0023] Furthermore, in step S13, the MobileNetV3-Small network is added to the Backbone network of the original YOLOv5 detection model; the MobileNetV3-Small network includes a common convolution layer, eleven bottleneck layers, an average pooling layer, and a point-by-point convolution layer connected in sequence; wherein the common convolution layer is used to extract initial features, the average pooling layer is used to extract global features, the point-by-point convolution layer is used to perform final channel adjustment, and the bottleneck layer is used to process the feature map using the common convolution layer, the depth-separable convolution layer, and the CBAM module. In the CBAM module, the channel attention module is used to enhance the attention to important channels and suppress secondary channels, and the spatial attention module is used to enhance the attention to important spatial positions and suppress secondary positions;
[0024] When adjusting the CSP module in the original YOLOv5 detection model, the CSP module in the P5 layer is modified by replacing the traditional standard convolution in the CSP module in the P5 layer with a deformable convolution.
[0025] Furthermore, in step S2, the specific processing process is as follows:
[0026] S21: Starting position determination
[0027] Determine the starting position of the photovoltaic cleaning robot, the starting position is set at the upper left corner, upper left corner, upper right corner or lower right corner of the area to be cleaned, determine the corresponding traversal mode by the starting position, and set the starting position as the origin;
[0028] S22: Get traversal rules
[0029] Obtain traversal rules according to the determined traversal mode;
[0030] S23: Obtaining displacement information
[0031] The photovoltaic cleaning robot determines the path length through the secondary integration of the accelerometer, that is, obtains the displacement information, and obtains the current coordinates of the photovoltaic cleaning robot according to the displacement information;
[0032] S24: Create a two-dimensional linked list node
[0033] A two-dimensional linked list is used to represent the layout of all photovoltaic panels in the area to be cleaned, and each node in the two-dimensional linked list represents a photovoltaic panel;
[0034] S25: Cleaning work
[0035] During the cleaning process, the cleaning is performed according to all the attributes of the next node in the two-dimensional linked list node and the boundary priority strategy.
[0036] Furthermore, in step S22, the types of traversal modes and corresponding traversal rules are as follows:
[0037] Mode 1
[0038] The traversal rules are:
[0039] The path starts from the upper right corner and follows the top-to-bottom order. At the end of each row, the path moves left to the start of the next row, and then moves from left to right at the beginning of the new row.
[0040] Mode 2
[0041] The traversal rules are:
[0042] The path starts from the lower right corner and follows the order from bottom to top. At the end of each row, the path moves left to the start of the previous row, and then moves from left to right at the beginning of a new row.
[0043] Mode 3
[0044] The path starts from the lower left corner and follows the order from bottom to top. At the end of each row, the path moves right to the start of the previous row, and then moves from right to left at the beginning of a new row.
[0045] Mode 4
[0046] The traversal rules are:
[0047] The path starts from the upper left corner and follows a top-to-bottom order. At the end of each row, the path moves right to the start of the next row, and then starts moving right to left at the beginning of the new row.
[0048] Furthermore, in step S23, the specific processing process is as follows:
[0049] S231: The acceleration of the photovoltaic cleaning robot on the x-axis and y-axis is measured by an accelerometer, denoted as a x (t) and a h (t);
[0050] S232: Integrate the acceleration on the x-axis and y-axis respectively to obtain the velocity v in the corresponding direction x (t) and v y (t):
[0051]
[0052] Among them, v x0 and v y0 is the initial velocity;
[0053] S233: Integrate the velocities on the x-axis and y-axis respectively to obtain the displacement s in the corresponding direction x (t) and sy (t):
[0054]
[0055] Among them, s x0 and y0 is the initial displacement, the default is 0;
[0056] S324: obtained by x With s y The length of a single photovoltaic panel is L, and the width is W. By dividing the displacement information by the length and width of the photovoltaic panel and rounding the result, the current coordinates of the photovoltaic cleaning robot are obtained:
[0057]
[0058] Among them, round represents the rounding function.
[0059] Furthermore, in step S24, each node Node in the two-dimensional linked list includes the following attributes:
[0060] float x,y: the coordinates of the current photovoltaic panel, relative to the origin;
[0061] bool isAccessible: indicates whether the next node is accessible. Whether the next node is accessible is determined based on the boundary type information.
[0062] int dir: indicates direction, where a value of 1 indicates east, a value of 1 indicates south, a value of 1 indicates west, and a value of 1 indicates north.
[0063] int s: the distance from the current node to the previous node, i.e. displacement information;
[0064] next, prev, down, up pointers: point to the nodes on the right, left, bottom, and top respectively.
[0065] Furthermore, in step S2, when the boundary type information is detected, the following processing is performed:
[0066] When only the left boundary is detected, set isAccessible = 0 of the left node of the current node in the two-dimensional linked list, and set the prev pointer of the current node to null;
[0067] When only the right boundary is detected, set isAccessible = 0 of the right node of the current node in the two-dimensional linked list, and set the next pointer of the current node to null;
[0068] When only the top boundary is detected, the isAccessible of the node below the current node in the two-dimensional linked list is set to 0, and the down pointer of the current node is set to null, and the photovoltaic cleaning robot rotates 90 degrees to the right;
[0069] When the left boundary and the top boundary are detected at the same time: set isAccessible=0 for the nodes on the left and above the current node, and set the prev and up pointers to null, the photovoltaic cleaning robot rotates 90 degrees to the right, updates the direction dir, and records the displacement information;
[0070] When the right boundary and the top boundary are detected at the same time: set isAccessible = 0 for the nodes to the right and above the current node, and set the next and up pointers to null, the photovoltaic cleaning robot rotates 90 degrees to the left, updates the direction dir, and records the displacement information;
[0071] When the left and right boundaries are detected at the same time: set isAccessible = 0 for the nodes on the left and right of the current node, and set the prev and next pointers to null, keep moving in the current direction, and record the displacement information;
[0072] When the left boundary, right boundary and top boundary are detected at the same time: isAccessible of the nodes on the left, right and above the current node is set to 0, and the prev, next and up pointers are set to null, the photovoltaic cleaning robot reverses direction and moves back, updates the direction dir, and records the displacement information;
[0073] When only the normal area is detected, isAccessible=1 is set for the nodes on the left, right and above the current node, and the prev pointer of the current node is pointed to the next pointer of the left node, the next pointer is pointed to the prev pointer of the right node, and the down pointer is pointed to the up pointer of the node below; the photovoltaic cleaning robot moves forward normally, and the direction dir remains unchanged, which is equal to the dir of the previous node.
[0074] Furthermore, in step S2, the edge priority strategy is that when the photovoltaic cleaning robot is at any position in a row or a column, it determines the next turning position, turns along the nearest edge first, and follows the current traversal mode path for cleaning.
[0075] Compared with the prior art, the present invention has the following advantages: the vision-based photovoltaic panel cleaning path planning method, during the cleaning process, the photovoltaic cleaning robot identifies the direction of the photovoltaic cleaning robot through a gyroscope, and obtains the displacement information of the photovoltaic cleaning robot through accelerometer calculation, and at the same time obtains images through a camera, transmits the images to a photovoltaic panel boundary detection model, identifies the photovoltaic panel boundary type information, converts the identified boundary type information and displacement information into two-dimensional linked list nodes, and performs cleaning work according to the two-dimensional linked list nodes and the boundary priority strategy, which solves the problem that traditional path planning algorithms usually rely on establishing a global map, resulting in the separation of perception and decision-making, and are difficult to adapt to dynamically changing environments, and optimizes and improves existing path planning algorithms, making them more adaptable and robust in more complex environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0076] Figure 1 Schematic diagram of the structure of the improved YOLOv5 detection model in the first embodiment of the present invention;
[0077] Figure 2 It is a flowchart of a method for enhancing data of a training set, a test set, and a validation set in the first embodiment of the present invention;
[0078] Figure 3 Schematic diagram of the training and deployment process of the improved YOLOv5 detection model in the second embodiment of the present invention;
[0079] Figure 4 This is an example diagram of the running paths of four traversal modes in the third embodiment of the present invention;
[0080] Figure 5 is an example diagram of the operation path of the photovoltaic panel cleaning path planning method based on vision in the fourth embodiment of the present invention, and the coordinates in the diagram are the coordinates corresponding to each position or each photovoltaic panel;
[0081] Figure 6 This is an example diagram of the running path using the simple boundary line breaking strategy in the fifth embodiment of the present invention;
[0082] Figure 7 1 is a schematic diagram of the structure of the photovoltaic cleaning robot in the present invention, wherein 1 is a camera in the visual component, which is installed on both sides of the front end of the cleaning brush component, and 2 is a gyroscope and an accelerometer, which are built into the robot;
[0083] Figure 8 It is a flow chart of the vision-based photovoltaic panel cleaning path planning method of the present invention. DETAILED DESCRIPTION
[0084] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.
[0085] Embodiment 1
[0086] like Figure 8 As shown, this embodiment provides a technical solution: a vision-based photovoltaic panel cleaning path planning method, comprising the following steps:
[0087] S1: Detection model improvement
[0088] The image data of the photovoltaic panel boundary is collected and processed to obtain a data set; on the basis of the original YOLOv5 detection model, the MobileNetV3-Small network is introduced and combined with the CBAM attention mechanism module to optimize the model structure, and the CSP module in the original YOLOv5 detection model is adjusted to obtain an improved YOLOv5 detection model; then the improved YOLOv5 detection model is trained to obtain a trained model, namely the photovoltaic panel boundary detection model; finally, the photovoltaic panel boundary detection model that meets the performance requirements is deployed in the photovoltaic cleaning robot;
[0089] In this embodiment, the specific process in step S1 is as follows:
[0090] 1. Data collection, preprocessing and enhancement
[0091] First, we collected at least 2,000 images of the photovoltaic panel boundary, and then preprocessed these images, including size standardization, random flipping, and cropping. The preprocessed images were used to construct a photovoltaic panel boundary dataset, where the photovoltaic panel boundary types include left boundary, right boundary, left corner, right corner, top boundary, and normal area. These types of boundary data were compared with standard samples in the database to complete type matching and form a classified dataset. Figure 2 The data enhancement method increases data and enhances data generalization ability.
[0092] The following is a further explanation of the left corner, right corner, and normal area:
[0093] Left corner: When both the left and top boundaries are detected, it is considered a left corner;
[0094] Right corner: When the right border and the top border are detected, it is judged as a right corner;
[0095] Normal area: The set of left border, right border, left corner, right corner, and top border that are not detected at the same time.
[0096] 2. Dataset annotation and division
[0097] Use annotation tools (such as Labeling) to carefully annotate each image in the dataset, marking the location and type of the photovoltaic panel boundary. After the annotation is completed, the dataset is divided into training set, test set and validation set according to the ratio (5:3:2 or 7:2:1) to prepare for subsequent model training and evaluation.
[0098] 3. Model construction
[0099] Based on the original YOLOv5 detection model, the MobileNetV3-Small network is introduced and combined with the CBAM attention mechanism module to optimize the model structure. At the same time, the CSP module in the original YOLOv5 detection model is adjusted in a targeted manner to improve the model's performance in photovoltaic panel boundary detection. These improvements help the model improve detection accuracy and robustness while maintaining high efficiency.
[0100] In this embodiment, the MobileNetV3-Small network (i.e. Figure 1 The M3 in the YOLOv5 detection model is added to the Backbone network of the original YOLOv5 detection model. Backbone is the backbone network that is mainly responsible for extracting features from the input image. It gradually extracts multi-level features of the image through multiple convolutional layers and modules (such as CSP, Bottleneck, and SPPF). The role of the Backbone network is to convert the original image into feature maps of different scales, which are then passed to other parts of the network for object detection and classification.
[0101] More specifically, the MobileNetV3-Small network structure includes: a normal convolution layer (Conv layer) for initial feature extraction; eleven sequentially connected bottleneck layers (Bottleneck layers), each bottleneck layer contains a CBAM module; an average pooling layer (Avg Pool layer) for global feature extraction. A point-by-point convolution layer (Pw Conv) for final channel adjustment.
[0102] As a more specific example, the following describes the processing of each component in the MobileNetV3-Small network:
[0103] 1. Ordinary convolution layer (Conv layer)
[0104] Input: original image, size H×W×3, where H and W are the image height and width, and 3 is the number of RGB channels.
[0105] Processing: A standard 2D convolution operation is performed, with a kernel size of typically 3x3 and a stride of 2. Batch normalization (BN) and a non-linear activation function (H-Swish) are then applied.
[0106] Output: A feature map of size H / 2×W / 2×C, where C is the number of convolution kernels.
[0107] 2. Bottleneck layer
[0108] Input: Feature map from the previous layer.
[0109] Processing process:
[0110] Ordinary convolution (1x1 Conv): expand or reduce the number of channels, perform BN and activation.
[0111] Depthwise Separable Convolution (Dw Conv): Perform 3x3 convolution on each channel without changing the number of channels, and then use BN.
[0112] CBAM module: The CBAM module further processes the output feature map by first applying a channel attention module and then a spatial attention module to enhance the model's focus on important features. If the input and output are the same size, a skip connection is added to add the input directly to the output to enhance the gradient flow. After that, ordinary convolution and BN are used.
[0113] Output: processed feature map, the size depends on the input feature map and the configuration of the convolutional layer.
[0114] 3. Average Pooling Layer (Avg Pool Layer)
[0115] Input: Feature maps from the last bottleneck layer.
[0116] Processing process: Perform global average pooling to reduce the feature map of each channel to a scalar and obtain the global feature vector.
[0117] Output: A feature vector of size 1×1×C.
[0118] 4. Point-wise convolution layer (Pw Conv layer)
[0119] Input: Global feature vector from average pooling layer.
[0120] Processing: The number of channels is adjusted using a 1x1 convolution, which is usually used to reduce the number of channels to match the number of output categories. This is followed by the application of an activation function (H-Swish).
[0121] Output: A vector of size 1×1×N, where N is the number of output categories.
[0122] To be more specific, the CBAM attention mechanism module includes a channel attention module and a spatial attention module, which are respectively used to enhance the model's perception of channel and spatial information. The number of input and output channels of the CBAM attention mechanism module matches the number of output channels of the bottleneck layer.
[0123] The CBAM module mainly improves the feature representation capability through the following two attention modules: Channel attention module: enhances the attention to important channels and suppresses minor channels. Spatial attention module: enhances the attention to important spatial positions and suppresses minor positions.
[0124] As a more specific example, the following is the processing of each part in the CBAM module, from input to output.
[0125] 1. Channel Attention Module
[0126] Input: Feature map: size is H×W×C, where H is height, W is width, and C is the number of channels.
[0127] Processing process:
[0128] 1. Global pooling: maximum pooling and average pooling: The feature map is globally pooled in the spatial dimensions (H and W) to obtain two 1-dimensional vectors, each with a size of 1x1xC. These two vectors represent the global maximum and global average of each channel, respectively.
[0129] 2. Shared fully connected layer: The two pooled vectors are passed through a shared fully connected network (usually containing two fully connected layers) to generate two 1D vectors. The size of these two vectors is still 1x1xC.
[0130] 3. Weighted sum: Add the two vectors processed by the fully connected layer element by element to obtain the final channel weight.
[0131] 4. Activation function: compresses channel weights to between 0 and 1.
[0132] Output: Channel attention map: channel weight vector of size 1x1xC.
[0133] 5. Channel weighting: Multiply the original input feature map with the channel attention map channel by channel to obtain a weighted feature map with the same size of HxWxC.
[0134] 2. Spatial Attention Module
[0135] Input: Weighted feature map from channel attention module, size HxWxC.
[0136] Processing process:
[0137] 1. Channel pooling: maximum pooling and average pooling: pool the feature map in the channel dimension (C) to obtain two feature maps, each with a size of H×W×1
[0138] 2. Channel concatenation: Concatenate the two pooled feature maps in the channel dimension to form a feature map of size H×W×2.
[0139] 3. Convolution operation: The concatenated feature map is passed through a 3x3 convolution kernel to generate a spatial attention map of size H×W×1.
[0140] 4. Activation function: compresses the value of the spatial attention map to between 0 and 1.
[0141] Output: Spatial attention map: a spatial weight map of size H×W×1.
[0142] 5. Spatial weighting: Multiply the feature map processed by the channel attention module with the spatial attention map pixel by pixel to obtain the final weighted feature map with size HxWxC.
[0143] Final output: feature map after channel and spatial weighting, size is HxWxC.
[0144] In this embodiment, when the CSP module in the original YOLOv5 detection model is adjusted in a targeted manner, the CSP module in the P5 layer is modified.
[0145] As a further step, the modifications to the CSP module in the P5 layer are as follows:
[0146] Introducing Deformable Convolution (De Conv): In the CSP module of the P5 layer, the traditional standard convolution is replaced by deformable convolution. Deformable convolution can dynamically adjust the sampling position of the convolution kernel, so as to more flexibly adapt to the geometric changes of the input feature map.
[0147] Dilation rate setting: In deformable convolution, the dilation rate is set to 1, 2, 3 or 1, 2, 3, 4. At the same time, the size of the convolution kernel can be selected as 5×5 or 8×8.
[0148] The deformable convolution uses bicubic interpolation: a weighted average is performed between 16 pixels (4×4 area) near the target location, and a cubic polynomial is used for interpolation.
[0149] 4. Model training and evaluation
[0150] The improved YOLOv5 detection model is trained using the divided training set. During the training process, the model continuously learns the boundary features of the photovoltaic panel and optimizes its own weights until the best detection effect is achieved. After the training is completed, the model is preliminarily evaluated using the test set, focusing on key indicators such as accuracy, recall rate, and mean average precision (mAP).
[0151] 5. Model application and optimization
[0152] Based on the test results obtained through the validation set, the model performance is comprehensively evaluated. If the model performance meets the requirements, it can be applied to the actual photovoltaic panel boundary type recognition task. If the expected effect is not achieved, it is necessary to further analyze the test results and adjust the model parameters or optimization strategies in a targeted manner until the model performance meets the actual application requirements.
[0153] S2: Path planning and cleaning
[0154] Determine the starting position of the photovoltaic cleaning robot and set the origin; during the cleaning process, the photovoltaic cleaning robot uses the gyroscope to identify the orientation (direction) of the photovoltaic cleaning robot, and uses the accelerometer to calculate the displacement information of the photovoltaic cleaning robot. At the same time, the camera obtains the image, and transmits the image to the photovoltaic panel boundary detection model (improved YOLOv5 detection model), identifies the photovoltaic panel boundary type information in front of the photovoltaic cleaning robot, converts the identified boundary type information and displacement information into two-dimensional linked list nodes, and performs cleaning work according to the two-dimensional linked list nodes and the boundary priority strategy, so as to realize real-time path planning during the cleaning process.
[0155] In this embodiment, the specific process in step S2 is as follows:
[0156] 1. Determine the starting position
[0157] The starting position of the photovoltaic cleaning robot is set at the upper left corner, upper left corner, upper right corner or lower right corner of the area to be cleaned. The corresponding traversal mode (1, 2, 3, 4) is determined by the starting position, and the origin is set; it is helpful to start the cleaning operation from a fixed starting point.
[0158] 2. Get traversal rules
[0159] The traversal rules are obtained according to the determined traversal mode. The setting of the traversal rules helps to systematically cover each area.
[0160] 3. Obtain displacement information
[0161] The photovoltaic cleaning robot determines the path length through the quadratic integration of the accelerometer, that is, obtains the displacement information;
[0162] 4. Create a two-dimensional linked list node
[0163] A two-dimensional linked list is used to represent the layout of photovoltaic panels. Each node (each node represents a photovoltaic panel) contains the following attributes:
[0164] float x,y: the coordinates of the current photovoltaic panel.
[0165] bool isAccessible: indicates whether the next node is accessible. Whether the next node is accessible is determined based on the boundary type information.
[0166] int dir: indicates direction (1: east, 2: south, 3: west, 4: north).
[0167] int s: the distance from the current node to the previous node, that is, the displacement information.
[0168] Pointers *next, *prev, *down, *up: point to the right, left, down, and up nodes respectively.
[0169] 5. Carry out cleaning work
[0170] During the cleaning process, the cleaning is performed according to all the attributes of the next node in the two-dimensional linked list node and the boundary priority strategy.
[0171] Embodiment 2
[0172] like Figure 3 As shown in the figure, the improved YOLOv5 detection model is trained based on the training data, with the goal of obtaining a model that can accurately detect the boundaries of photovoltaic panels, namely, the photovoltaic panel boundary detection model. First, create a new yaml configuration file named "YOLOv5-M3.yaml" to configure the parameters for model training. At the same time, create a new Python script file named "M3former.py" to define the improved module. Next, add the improved module to the "common.py" file, which is a library file containing various functions and modules. Then, modify the "train.py" file and change the configuration file path to the path of the "YOLOv5-M3.yaml" file. The "train.py" file is the main script for training the model. In the "train.py" file, set the number of training epochs between 300 and 500 and the batch size to 32 or 64.
[0173] After completing these settings, run the "train.py" file to start training the model until the optimal model is obtained, that is, the photovoltaic panel boundary detection model. After the training is completed, the photovoltaic panel boundary detection model is tested and the test results are recorded, including training accuracy (precision), recall rate (recall) and mean average precision (mAP). mAP is an important indicator for evaluating the performance of the target detection model. Through the above steps, the original YOLOv5 detection model can be improved to perform better in the photovoltaic panel boundary detection task, and the performance of the model can be quantified through the test results. The performance indicator formula is:
[0174]
[0175] Among them, P = TP / (TP+FP), R = TP / (TP+FN), classes is the number of photovoltaic panel boundary types identified in the training results, P represents precision, R represents recall rate, TP represents the number of correctly identified positive samples, FP represents the number of incorrectly identified negative samples, and FN represents the number of incorrectly identified positive samples.
[0176] 1. Suppose there is a task of photovoltaic panel boundary detection. Use the YOLO v5 original network as the baseline model. Now we want to improve the detection performance of YOLO v5. The specific steps are as follows:
[0177] 1): Add the MobileNetV3-Small network to the Backbone network of the baseline model. The number of input channels through the CBAM module should be equal to the number of output channels of the Bottleneck layer. The CBAM attention mechanism module includes a channel attention module and a spatial attention module, which are used to enhance the model's perception of channel and spatial information, respectively. When integrating the CBAM attention mechanism module into the MobileNetV3-Small network, the key steps include designing the CBAM module and adapting it to the Bottleneck layer of the MobileNetV3-Small network. Ensure that the number of input and output channels of the CBAM module matches the number of output channels of the Bottleneck layer, which can effectively improve the performance of the model and enhance its performance in visual tasks.
[0178] 2): The CSP module in the P5 layer of the baseline model is modified to enhance the receptive field. 1 and 3 are taken as the expansion rates of different deformable convolutions, and a fixed convolution kernel size of 5x5 is used. Bicubic interpolation is used to reduce the interpolation error when processing images with subtle changes. In addition, the features of the P2 layer are fused to improve the ability to more accurately locate the target.
[0179] The improved YOLOv5 detection model is trained based on the training data to obtain the photovoltaic panel boundary detection model; the photovoltaic panel boundary detection model is tested, and the test results are recorded after the test is completed.
[0180] Load the divided data set into the improved YOLOv5 detection model, configure the environment variables and related hardware and software configurations. The relevant configurations are: windows10, 64-bit; CPU: Inter i5-12400F; GPU: NVIDIA GeForce4060Ti; torch: 1.10.8; python: 3.9; CUDA: 1.10.8; Ubuntu: 20.04.
[0181] Embedded related configuration: RK3568: 4+64G; buildroot: 2023.02-rc3-01869; rknn-toolkit2: 1.5.0;
[0182] Create a new yam1 configuration file and py file, named YOLOv5-M3.yam1 and MobileNetV3-Small.py files respectively; add the improved module to the common.py file, and change the cfg file path in the train.py file to the path of the YOLOv5-M3.yam1 file, configure the YOLOv5 related parameters, set the batch size to 32, the number of training rounds to 500, the initial learning rate to 0.01, and the weight decay coefficient to 0.0003; run the train.py file to train the model and get the optimal model; the yam1 file is the parameter for configuring model training, the common.py file is the function library, and the train.py file is the Python script for training the model.
[0183] The photovoltaic panel boundary detection model is evaluated according to the test results, and the photovoltaic panel boundary recognition is performed through the photovoltaic panel boundary detection model that has been evaluated as qualified;
[0184] When IoU is set to 0.5, the AP of all images of each category is calculated, and then the average of all types is calculated, that is, mAP. The total mAP is 81.5.
[0185] This paper combines model optimization, innovative strategy application and rigorous parameter tuning to comprehensively promote the performance upgrade of YOLOv5 in target recognition technology.
[0186] After the training target model best.pt is trained, in the YOLOv5 directory, use the script provided by YOLOv5 to convert the PyTorch model to an ONNX model. Run the command python export.py --weights best.pt --img640 --batch1 --device 0 --simplify, which will generate a best.onnx file.
[0187] Next, you need to use RKNN-Toolkit to convert ONNX to RKNN. Make sure you are using Python 3.7 or 3.8 and install RKNN-Toolkit2. Then create an RKNN object and configure the model's preprocessing parameters and channel order. Use the rknn.load_onnx(model = 'best.onnx') method to load the ONNX model. Then use the rknn.build(do_quantization = False) method to compile the model and configure quantization as needed. Finally, use the rknn.export_rknn('best.rknn') method to export the generated RKNN model file.
[0188] Deploy the generated best.rknn model file to the RK3568 device. Use the command pipinstall rknn-toolkit2 to install it on the RK3568 device. Create an RKNN object on the device and load the RKNN model. Use the rknn.load_rknn('best.rknn') method to load the model. Then use the rknn.init_runtime() method to initialize the model runtime environment on RK3568. Then use the rknn.inference method for inference, where input_data is the input data, such as an image. Process the inference results according to the output format of YOLOv5 to obtain the boundary detection box, type, and confidence of the photovoltaic panel.
[0189] The improved YOLOv5 detection model of the present invention has relatively low size and computational complexity, so that it can run faster and consume less power on platforms with limited resources. The MobileNetV3-Small network uses a deep separable convolutional layer, which requires fewer parameters and computation than traditional convolutional layers, making YOLOv5-MobileNetV3 (the improved YOLOv5 detection model) a lightweight target detection model that is easy to deploy on the edge device rk3568.
[0190] The present invention introduces the MobileNetV3-Small network and an adaptive activation function (such as H-Swish), which enables the model to dynamically adjust the activation strength according to the input, thereby improving the detection effect and the generalization ability of the model, better extracting features, and reducing information loss, thereby improving the recognition accuracy of the improved YOLOv5 detection model.
[0191] Embodiment 3
[0192] During the feature extraction and prediction phase, the YOLOv5 network generates a feature map, which contains information about multiple bounding boxes predicted at each location.
[0193] Bounding box parameters: The bounding box at each location contains four parameters: the x and y coordinates of the center point, as well as the width and height. In addition, it also contains a confidence score indicating the probability of whether there is an object in the bounding box.
[0194] Each bounding box also outputs a type score vector. The length of this vector is equal to the number of object types (for example, the dataset has 10 types, so the type score vector length is 10). Each value represents the probability that the object in the bounding box belongs to a certain type.
[0195] YOLOv5 applies the softmax function. The softmax function is defined as:
[0196]
[0197] Among them, x i is the i-th score in the type score vector. The softmax function normalizes these scores into probability values so that their sum is 1. After applying the softmax function, the scores of each type are converted into probability values. These probability values represent the likelihood that the object in the bounding box belongs to each type. Ultimately, YOLOv5 selects the type with the highest probability value as the predicted type of the bounding box. At the same time, it also outputs this probability value as the confidence of the prediction. The confidence output by the algorithm will be used as the condition for judging the left boundary, right boundary, and top boundary.
[0198] Through four different traversal modes, it aims to fully cover every cell in the two-dimensional space. Each mode ensures that all cells in the entire two-dimensional space are visited to avoid omissions or repeated coverage. The four modes start from different origins (upper left corner, upper right corner, lower left corner, lower right corner) and follow different directions (from top to bottom, from bottom to top, from left to right, from right to left). This design allows the algorithm to flexibly adapt to different path requirements in various situations. Setting a fixed path mode can avoid path conflicts and repeated access problems that may occur in complex environments.
[0199] The specific contents are as follows:
[0200] Select the traversal mode based on the starting position:
[0201] 1. Mode 1
[0202] The path starts from the upper right corner and follows the order from top to bottom.
[0203] The path moves left at the end of each row to the start of the next row, then starts moving left to right at the beginning of the new row.
[0204] 2. Mode 2
[0205] The path starts from the lower right corner and follows the order from bottom to top.
[0206] The path moves left at the end of each row to the start of the previous row, then starts moving left to right at the beginning of a new row.
[0207] Similar to mode 1, but the order is from bottom to top, which may be used for traversal needs in different directions.
[0208] 3. Mode 3
[0209] The path starts from the lower left corner and follows the order from bottom to top.
[0210] The path moves right at the end of each row to the start of the previous row, then starts moving right to left at the beginning of a new row.
[0211] This pattern is similar to pattern 2, but in the opposite direction, moving from left to right.
[0212] 4. Mode 4
[0213] The path starts from the upper left corner and follows the order from top to bottom.
[0214] The path moves right at the end of each row to the start of the next row, then starts moving right to left at the beginning of the new row.
[0215] Similar to mode 3, but the order is from top to bottom, suitable for situations where coverage from left to right is required.
[0216] The four modes cover traversal paths in different directions respectively to ensure a complete scan or visit of the two-dimensional area in a variety of situations.
[0217] Now create a two-dimensional linked list Node to express the layout of the photovoltaic panel. The two-dimensional linked list attributes include the distance from the current node to the previous node, the coordinates of the photovoltaic panel, the pointer, and whether it is accessible through the bool Boolean value (boolisAccessible). This is to compare this two-dimensional linked list to the photovoltaic panel, so that the photovoltaic cleaning robot can use the gyroscope and accelerometer to determine the current location. The structure of the node is as follows:
[0218]
[0219] The two-dimensional linked list structure is as follows:
[0220] A two-dimensional linked list is used to represent the photovoltaic panel layout. Each node contains the following attributes:
[0221] float x, y: The coordinates of the current photovoltaic panel, relative to the origin. It is changed according to the traversal mode selection and dir and displacement information. When reaching mode 1: the value of x is positive with dir=3 and gradually increases, and the value of y is positive with dir=2 and gradually increases. The accelerometer measures the acceleration of the photovoltaic cleaning robot on the x-axis and y-axis, recorded as a x (t) and a y (t).
[0222] Integrate the acceleration on the x-axis and y-axis respectively to get the velocity v in the corresponding direction x (t) and v y (t):
[0223]
[0224] Among them, v x0 and v y0 is the initial velocity;
[0225] Integrate the velocities on the x-axis and y-axis respectively to obtain the displacement s in the corresponding direction x (t) and s y (t):
[0226]
[0227] Among them, s x0 and y0 is the initial displacement, usually zero.
[0228] By getting x With s yThe length of a single photovoltaic panel is L (along the x direction) and the width is W (along the y direction). By dividing the displacement information by the length and width of the photovoltaic panel and rounding the result, the current coordinates of the photovoltaic cleaning robot can be obtained. Round represents the rounding function.
[0229]
[0230] bool isAccessible: Indicates whether the photovoltaic panel of the next node is accessible.
[0231] int dir: indicates direction (1: east, 2: south, 3: west, 4: north).
[0232] int s: the distance from the current node to the previous node, that is, the displacement information.
[0233] Pointers *next, *prev, *down, *up: point to the right, left, down, and up nodes respectively.
[0234] When a boundary is detected (such as the left boundary, right boundary, and top boundary), these pointers can help set the isAccessible flag of the adjacent node, and mark the unreachable area by setting the relevant pointer to null, thereby avoiding path conflicts and repeated visits. When the photovoltaic cleaning robot moves to a new node, it can use these pointers to update and record the current path information (such as distance, direction, etc.). During the second cleaning, the photovoltaic cleaning robot can avoid repeated visits to areas that have been cleaned. By using isAccessible and pointer settings, the system can directly skip unreachable or cleaned nodes. During the initial cleaning, the nodes that have been visited will be marked as "cleaned" or set to null through the pointer. When the second cleaning begins, the photovoltaic cleaning robot can quickly avoid these nodes and focus on areas that have not been fully cleaned.
[0235] Determine the size of the two-dimensional linked list: set the size of the two-dimensional linked list according to the maximum length and width of a single photovoltaic panel.
[0236] It should be noted that the purpose of determining the size of a two-dimensional linked list is to pre-plan the space for the data structure in program design. By pre-determining the size, unnecessary memory allocation and release operations can be reduced, improving the overall performance of the program. This helps prevent cross-bounds errors and memory overflows. In a multi-dimensional linked list, if the size is not determined in advance, the program may access illegal memory areas, causing runtime errors.
[0237] Origin setting: Place the photovoltaic cleaning robot at a corner of the photovoltaic panel group (upper left corner, upper left corner, upper right corner, lower right corner), obtain the traversal mode, the initial node (x=0, y=0, isAccessible=1, dir=initial direction), and the initial node is the origin.
[0238] The contents of path planning and navigation are as follows:
[0239] Going Forward and Recording:
[0240] The photovoltaic cleaning robot uses a camera to acquire images, and then uses the improved YOLOv5 detection model to detect boundary category information, and uses the accelerometer to perform secondary integration to determine the displacement information, while using the gyroscope to identify the direction of the photovoltaic cleaning robot.
[0241] When a boundary is detected, the photovoltaic cleaning robot moves along the column or row of the photovoltaic panel and records the current displacement information by updating the y or x coordinate and the linked list pointer.
[0242] Boundary processing:
[0243] Left boundary: If only the left boundary is detected, set isAccessible=0 of the left node of the current node in the two-dimensional linked list, and set the prev pointer of the current node to null.
[0244] Right boundary: If only the right boundary is detected, set isAccessible = 0 of the right node of the current node in the two-dimensional linked list, and set the next pointer of the current node to null.
[0245] Top boundary: If only the top boundary is detected, set isAccessible=0 of the node below the current node in the two-dimensional linked list, and set the down pointer of the current node to null, and the photovoltaic cleaning robot rotates 90 degrees to the right.
[0246] Special situation handling:
[0247] The left and top borders are detected at the same time:
[0248] Set isAccessible = 0 for the linked list nodes to the left and above the current node, and set the prev and up pointers to null. The photovoltaic cleaning robot rotates 90 degrees to the right. Update the direction dir (dir = (dir% 4) + 1, no problem, % in the assignment statement means the remainder), and record the displacement information.
[0249] Right and top borders detected at the same time:
[0250] Set isAccessible=0 for the linked list nodes to the right and above the current node, and set the next and up pointers to null. The photovoltaic cleaning robot rotates 90 degrees to the left. Update the direction dir(dir=(dir==1)?4:dir-1) ("==" in the assignment statement means to determine whether they are equal, and "?" in the assignment statement means that if dir=1, then dir=4, otherwise dir=dir-1), and record the displacement information.
[0251] Both left and right borders are detected:
[0252] Set isAccessible=0 for the linked list nodes on the left and right of the current node, and set the prev and next pointers to null, keep moving in the current direction, and record the displacement information.
[0253] The left, right, and top borders are detected simultaneously:
[0254] Set isAccessible=0 for the linked list nodes on the left, right and above the current node, and set the prev, next and up pointers to null. The photovoltaic cleaning robot reverses direction and moves backward, updating the direction dir (dir=((dir-1+2)%4)+1).
[0255] Only normal areas are detected: When only normal areas are detected, isAccessible=1 is set for the linked list nodes on the left, right, and above the current node, and the prev pointer of the current node points to the next pointer of the left node, the next pointer points to the prev pointer of the right node, and the down pointer points to the up pointer of the node below. The photovoltaic cleaning robot moves forward normally, and the direction dir remains unchanged, which is equal to the dir of the previous node.
[0256] Embodiment 4
[0257] Assume that a photovoltaic cleaning robot moves in a complex photovoltaic panel grid:
[0258] A single photovoltaic panel consists of 60 photovoltaic panel components (1.65 meters by 0.991 meters)
[0259] Initial position: The PV cleaning robot starts from a predetermined origin of the grid, e.g. Figure 5 The location of the photovoltaic cleaning robot is set to traversal mode 1.
[0260] Camera photography: The photovoltaic cleaning robot reaches the first photovoltaic panel, takes an image, and identifies the photovoltaic panel type information (whether it is at the left boundary, right boundary, left corner, right corner, top boundary, and normal area).
[0261] Gyroscope assistance: Use gyroscope data and accelerometer data to determine whether the photovoltaic cleaning robot has reached the expected position correctly.
[0262] Declare a struct Node called visited_node. visited_nodes is used to store a linked list of all visited nodes.
[0263] Record location: Record the coordinates (0, 0) of the photovoltaic panel in visited_nodes.
[0264] Move to the photovoltaic panel of the next node: The photovoltaic cleaning robot moves to the photovoltaic panel of the next node (such as (0,1)) and repeats the above steps.
[0265] In this way, the PV cleaning robot can gradually traverse all PV panels and record the coordinates of each PV panel. Gyroscope data fusion refers to combining the data from the gyroscope with the information provided by other sensors (accelerometers) to more accurately determine the actual direction and movement distance of the PV cleaning robot. The following is a detailed explanation:
[0266] 1. The role of gyroscope
[0267] The gyroscope is mainly used to measure the angular velocity of the photovoltaic cleaning robot, that is, the rotation speed of the photovoltaic cleaning robot on each axis. By integrating (accumulating) the angular velocity, the rotation angle of the photovoltaic cleaning robot can be obtained. This helps to determine the orientation, that is, the direction, of the photovoltaic cleaning robot.
[0268] 2. The role of accelerometer
[0269] The accelerometer measures linear acceleration, that is, the acceleration of the photovoltaic cleaning robot in all directions. By integrating the acceleration twice, the displacement information of the photovoltaic cleaning robot can be obtained.
[0270] 3. The role of the visual system
[0271] The visual system acquires images through the camera and identifies the type information of the photovoltaic panel through the improved YOLOv5 detection model. During the movement of the photovoltaic cleaning robot, the boundary type information of the photovoltaic panel is updated in real time by continuously acquiring images and performing boundary type detection. This information is used to determine whether the photovoltaic cleaning robot has crossed the boundary. For example, if the distance between the photovoltaic cleaning robot and the photovoltaic panel target detection frame is less than the preset safety distance, it is determined that the photovoltaic cleaning robot is not within the safety range of the photovoltaic panel. At this time, the photovoltaic cleaning robot will adjust the direction of travel in time according to the latest target detection frame, and perform turning control at the corner point of the target detection frame to ensure that the photovoltaic cleaning robot always walks along the path within the target detection frame.
[0272] 4. The specific steps of data fusion are as follows:
[0273] Step 1: Get the raw data
[0274] Gyroscope data: Get the angular velocity data of the photovoltaic cleaning robot (usually in three axes: x, y, z).
[0275] Accelerometer data: Get the linear acceleration data of the photovoltaic cleaning robot (usually in three axes: x, y, z).
[0276] Visual system data: Obtain images taken by the camera and use the improved YOLOv5 detection model to identify the boundary type information of the photovoltaic panel.
[0277] Step 2: Process and correct the data
[0278] Gyroscope data integration: Integrate the angular velocity data of the gyroscope to obtain the rotation angle of the photovoltaic cleaning robot, and then obtain the direction of the photovoltaic cleaning robot.
[0279] Accelerometer data integration: The acceleration data is integrated twice to obtain the displacement information of the photovoltaic cleaning robot. Since the accelerometer data may drift, it is usually necessary to correct it through a filter (such as a Kalman filter).
[0280] Step 3: Data Fusion
[0281] Initial position and orientation: Set the initial position and orientation of the photovoltaic cleaning robot.
[0282] Direction update: Update the current direction of the photovoltaic cleaning robot based on the gyroscope data.
[0283] Position update: Combined with the displacement information of the accelerometer, the current position of the photovoltaic cleaning robot is updated.
[0284] Error correction: Correct the data of gyroscope and accelerometer to reduce the accumulated error.
[0285] like Figure 5 As shown, the photovoltaic cleaning robot moves along a bow-shaped path in the photovoltaic panel grid (area to be cleaned):
[0286] Initial position: The photovoltaic cleaning robot is at position 0, direction is south, dir=2.
[0287] Set position 0 to coordinates (0, 0), i.e., the origin, and bool isAccessible = 1.
[0288] Obtaining data: The photovoltaic panel boundary detection model identifies the left boundary. According to the edge priority strategy, it should turn left after passing the photovoltaic panel of the next node.
[0289] Direction update: heading east, dir=1.
[0290] Position update: turn left, move forward 1.6 meters, and update to position 1.
[0291] Data fusion: Set position 1 to (-1, 1), set prev and up pointers to null, and boolisAccessible = 1.
[0292] Record position: record the node coordinates and related attributes corresponding to position 1 in visited_nodes. Move to position 2.
[0293] Direction update: turn south, dir=2.
[0294] Position update: turn right and move forward 2 meters, update to position 2, the photovoltaic panel boundary detection model recognizes the left corner, sets the prev and up pointers to null, and bool isAccessible = 1.
[0295] Record position: record the node coordinates (-1, 3) and related attributes corresponding to position 2 in visited_nodes.
[0296] The photovoltaic panel boundary detection model recognizes the left corner and should turn right and move to position 3.
[0297] Direction update: turn west, dir=3.
[0298] Position update: turn right and move forward 1.6 meters, update to position 3, the photovoltaic panel boundary detection model recognizes the left corner, sets the prev and up pointers to null, and boolisAccessible = 1.
[0299] Record position: record the node coordinates (0, 3) and related attributes corresponding to position 3 in visited_nodes.
[0300] The photovoltaic panel boundary detection model recognizes the left corner and should turn right and move to position 4.
[0301] Direction update: turn north, dir=4.
[0302] Position update: Go forward and turn right 1.6 meters, update to position 4, the photovoltaic panel boundary detection model recognizes the right boundary ahead. bool isAccessible=1.
[0303] Record position: record the node coordinates (0, 1) and related attributes corresponding to position 4 in visited_nodes.
[0304] The photovoltaic panel boundary detection model identifies the normal area and, based on the edge priority strategy, turns left and moves to position 5.
[0305] Direction update: turn west, dir=3.
[0306] Position update: Go forward, turn right and go forward 1.6 meters, update to position 5, the photovoltaic panel boundary detection model recognizes the normal area. bool isAccessible=1.
[0307] Record position: Record the node coordinates (1, 1) and related attributes corresponding to position 5 in visited_nodes.
[0308] The photovoltaic panel boundary detection model recognizes the normal area and, based on the edge priority strategy, determines that the photovoltaic panel boundary corresponding to (1, 2) is the top boundary. It should turn right and move forward to the position (2, 2), with direction dir=3. At this time, based on the edge priority strategy, it turns left 90 degrees, dir=2, and moves to position 6.
[0309] Direction update: Turn south, dir=2.
[0310] Position update: Go forward, turn right and go forward 1.6 meters, update to position 6, the photovoltaic panel boundary detection model recognizes the left, right and top boundaries. Set the prev, next and up pointers to null, and bool isAccessible = 1.
[0311] Record position: Record the node coordinates (2, 3) and related attributes corresponding to position 6 in visited_nodes.
[0312] The photovoltaic panel boundary detection model identifies the left, right and top boundaries and turns back.
[0313] Continue to move in a bow shape to traverse the entire photovoltaic panel grid:
[0314] When the photovoltaic cleaning robot moves to position 7 (the corresponding node coordinates are (8, -1)), it will determine that the cleaning direction has changed based on the gyroscope data and the node's dir data, and automatically switch to traversal mode 3 for cleaning.
[0315] Repeat the above steps until the entire photovoltaic panel grid is traversed.
[0316] It should be noted that in the present invention, the edge priority strategy is that when the photovoltaic cleaning robot is at any position in a row or a column, it determines the next turning position, preferentially turns along the nearest edge, and follows the current traversal mode path for cleaning.
[0317] Embodiment 5
[0318] according to Figure 6 In the case shown in the figure, when the photovoltaic cleaning robot operates with a simple boundary detection strategy, three uncleaned areas appear. At marker 1, the photovoltaic cleaning robot cannot judge the distance between areas 1 and 3, and fails to explore the new area below the left of marker 1. At marker 2, the area below the left of marker 2 is not cleaned, but the next area is cleaned in advance. At marker 3, the area above the right of marker 3 is not cleaned, but it returns to the initial area in advance. This shows that the photovoltaic cleaning robot needs to improve its path planning strategy to ensure more comprehensive coverage and cleaning of all areas.
[0319] To summarize, in the vision-based photovoltaic panel cleaning path planning method of the above-mentioned embodiment, during the cleaning process, the photovoltaic cleaning robot identifies the orientation of the photovoltaic cleaning robot through a gyroscope, and obtains the displacement information of the photovoltaic cleaning robot through accelerometer calculation, and at the same time obtains images through a camera, and transmits the images to the photovoltaic panel boundary detection model, identifies the photovoltaic panel boundary type information, and converts the identified boundary type information and displacement information into two-dimensional linked list nodes, and performs cleaning work according to the two-dimensional linked list nodes and the boundary priority strategy, which solves the problem that traditional path planning algorithms usually rely on establishing a global map, resulting in the separation of perception and decision-making, and are difficult to adapt to dynamically changing environments, and optimizes and improves the existing path planning algorithms, so that they have stronger adaptability and robustness in more complex environments.
[0320] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A vision-based photovoltaic panel cleaning path planning method, It is characterized in that The following steps are involved: S1: Detection model improvement The image data of the photovoltaic panel boundary is collected and processed to obtain a data set; on the basis of the original YOLOv5 detection model, the MobileNetV3-Small network is introduced and combined with the CBAM attention mechanism module to optimize the model structure, and the CSP module in the original YOLOv5 detection model is adjusted to obtain an improved YOLOv5 detection model; then the improved YOLOv5 detection model is trained to obtain a trained model, namely the photovoltaic panel boundary detection model; finally, the photovoltaic panel boundary detection model that meets the performance requirements is deployed in the photovoltaic cleaning robot; S2: Path planning and cleaning Determine the starting position of the photovoltaic cleaning robot and set the origin; during the cleaning process, the photovoltaic cleaning robot identifies the direction of the photovoltaic cleaning robot through the gyroscope, and obtains the displacement information of the photovoltaic cleaning robot through the accelerometer calculation, and at the same time obtains images through the camera, and transmits the images to the photovoltaic panel boundary detection model, identifies the boundary type information of the photovoltaic panel in front of the photovoltaic cleaning robot, converts the identified boundary type information and displacement information into two-dimensional linked list nodes, performs cleaning work according to the two-dimensional linked list nodes and the boundary priority strategy, and realizes real-time path planning during the cleaning process.
2. A photovoltaic panel cleaning path planning method based on vision according to claim 1, It is characterized in that In step S1, the specific processing process is as follows: S11: Data collection, preprocessing and enhancement Collect image data of the photovoltaic panel boundary, preprocess the image to obtain the preprocessed image, constitute the initial photovoltaic panel boundary data set, and then perform data enhancement processing on the photovoltaic panel boundary data set to obtain the final photovoltaic panel boundary data set; wherein the preprocessing includes size standardization, random flipping and cropping; S12: Dataset annotation and division Use annotation tools to annotate each image in the final photovoltaic panel boundary dataset, marking the location and type of the photovoltaic panel boundary; after the annotation is completed, the dataset is divided into a training set, a test set, and a validation set according to the set ratio; S13: Model construction On the basis of the original YOLOv5 detection model, the MobileNetV3-Small network is introduced and combined with the CBAM attention mechanism module to optimize the model structure. At the same time, the CSP module in the original YOLOv5 detection model is adjusted to obtain the improved YOLOv5 detection model. S14: Model training and evaluation The improved YOLOv5 detection model is trained using the divided training set. After the training is completed, the model is preliminarily evaluated using the test set, and the photovoltaic panel boundary detection model that has passed the preliminary evaluation is saved. S15: Model application and optimization Based on the test results obtained through the validation set, the performance of the photovoltaic panel boundary detection model that has passed the preliminary evaluation is comprehensively evaluated, and a photovoltaic panel boundary detection model that meets the performance requirements is obtained. The model is then deployed in the controller of the photovoltaic cleaning robot, and the controller is communicated with the camera in the visual component.
3. A photovoltaic panel cleaning path planning method based on vision according to claim 2, It is characterized in that In the step S11 , the photovoltaic panel boundary types include left boundary, right boundary, left corner, right corner, top boundary and normal area.
4. A photovoltaic panel cleaning path planning method based on vision according to claim 2, It is characterized in that In step S13, the MobileNetV3-Small network is added to the Backbone network of the original YOLOv5 detection model; the MobileNetV3-Small network includes a common convolution layer, eleven bottleneck layers, an average pooling layer, and a point-by-point convolution layer connected in sequence; wherein the common convolution layer is used to extract initial features, the average pooling layer is used to extract global features, the point-by-point convolution layer is used to perform final channel adjustment, and the bottleneck layer is used to process the feature map using the common convolution layer, the depth-separable convolution layer, and the CBAM module. In the CBAM module, the channel attention module is used to enhance the attention to important channels and suppress secondary channels, and the spatial attention module is used to enhance the attention to important spatial positions and suppress secondary positions; When adjusting the CSP module in the original YOLOv5 detection model, the CSP module in the P5 layer is modified by replacing the traditional standard convolution in the CSP module in the P5 layer with a deformable convolution.
5. A photovoltaic panel cleaning path planning method based on vision according to claim 1, It is characterized in that In step S2, the specific processing process is as follows: S21: Starting position determination Determine the starting position of the photovoltaic cleaning robot, the starting position is set at the upper left corner, upper left corner, upper right corner or lower right corner of the area to be cleaned, determine the corresponding traversal mode by the starting position, and set the starting position as the origin; S22: Get traversal rules Obtain traversal rules according to the determined traversal mode; S23: Obtaining displacement information The photovoltaic cleaning robot determines the path length through the secondary integration of the accelerometer, that is, obtains the displacement information, and obtains the current coordinates of the photovoltaic cleaning robot according to the displacement information; S24: Create a two-dimensional linked list node A two-dimensional linked list is used to represent the layout of all photovoltaic panels in the area to be cleaned, and each node in the two-dimensional linked list represents a photovoltaic panel; S25: Cleaning work During the cleaning process, the cleaning is performed according to all the attributes of the next node in the two-dimensional linked list node and the boundary priority strategy.
6. A photovoltaic panel cleaning path planning method based on vision according to claim 5, It is characterized in that In step S22, the types of traversal modes and corresponding traversal rules are as follows: Mode 1 The traversal rules are: The path starts from the upper right corner and follows the top-to-bottom order. At the end of each row, the path moves left to the start of the next row, and then moves from left to right at the beginning of the new row. Mode 2 The traversal rules are: The path starts from the lower right corner and follows the order from bottom to top. At the end of each row, the path moves left to the start of the previous row, and then moves from left to right at the beginning of a new row. Mode 3 The path starts from the lower left corner and follows the order from bottom to top. At the end of each row, the path moves right to the start of the previous row, and then moves from right to left at the beginning of a new row. Mode 4 The traversal rules are: The path starts from the upper left corner and follows a top-to-bottom order. At the end of each row, the path moves right to the start of the next row, and then starts moving right to left at the beginning of the new row.
7. A photovoltaic panel cleaning path planning method based on vision according to claim 5, It is characterized in that In step S23, the specific processing process is as follows: S231: The acceleration of the photovoltaic cleaning robot on the x-axis and y-axis is measured by an accelerometer, denoted as a x (t) and a y (t); S232: Integrate the acceleration on the x-axis and y-axis respectively to obtain the velocity υ in the corresponding direction x (t) and v y (t): Among them, x0 and y0 is the initial velocity; S233: Integrate the velocities on the x-axis and y-axis respectively to obtain the displacement s in the corresponding direction x (t) and s y (t): Among them, s x0 and y0 is the initial displacement, the default is 0; S324: obtained by x With s y The length of a single photovoltaic panel is L, and the width is W. By dividing the displacement information by the length and width of the photovoltaic panel and rounding the result, the current coordinates of the photovoltaic cleaning robot are obtained: Among them, round represents the rounding function.
8. A photovoltaic panel cleaning path planning method based on vision according to claim 5, It is characterized in that In step S24, each node Node in the two-dimensional linked list contains the following attributes: float x,y: the coordinates of the current photovoltaic panel, relative to the origin; bool isAccessible: indicates whether the next node is accessible. Whether the next node is accessible is determined based on the boundary type information. int dir: indicates direction, where a value of 1 indicates east, a value of 1 indicates south, a value of 1 indicates west, and a value of 1 indicates north. int s: the distance from the current node to the previous node, i.e. displacement information; next, prev, down, up pointers: point to the nodes on the right, left, bottom, and top respectively.
9. A photovoltaic panel cleaning path planning method based on vision according to claim 8, It is characterized in that In step S2, when the boundary type information is detected, the following processing is performed: When only the left boundary is detected, set isAccessible = 0 of the left node of the current node in the two-dimensional linked list, and set the prev pointer of the current node to null; When only the right boundary is detected, set isAccessible = 0 of the right node of the current node in the two-dimensional linked list, and set the next pointer of the current node to null; When only the top boundary is detected, the isAccessible of the node below the current node in the two-dimensional linked list is set to 0, and the down pointer of the current node is set to null, and the photovoltaic cleaning robot rotates 90 degrees to the right; When the left boundary and the top boundary are detected at the same time: set isAccessible=0 for the nodes on the left and above the current node, and set the prev and up pointers to null, the photovoltaic cleaning robot rotates 90 degrees to the right, updates the direction dir, and records the displacement information; When the right boundary and the top boundary are detected at the same time: set isAccessible = 0 for the nodes to the right and above the current node, and set the next and up pointers to null, the photovoltaic cleaning robot rotates 90 degrees to the left, updates the direction dir, and records the displacement information; When the left and right boundaries are detected at the same time: set isAccessible = 0 for the nodes on the left and right of the current node, and set the prev and next pointers to null, keep moving in the current direction, and record the displacement information; When the left boundary, right boundary and top boundary are detected at the same time: isAccessible of the nodes on the left, right and above the current node is set to 0, and the prev, next and up pointers are set to null, the photovoltaic cleaning robot reverses direction and moves back, updates the direction dir, and records the displacement information; When only the normal area is detected, set isAccessible=1 for the nodes on the left, right, and above the current node, and point the prev pointer of the current node to the next pointer of the left node, the next pointer to the prev pointer of the right node, and the down pointer to the up pointer of the node below; The photovoltaic cleaning robot moves forward normally, and the direction dir remains unchanged, which is equal to the dir of the previous node.
10. A photovoltaic panel cleaning path planning method based on vision according to claim 9, It is characterized in that In step S2, the edge priority strategy is that when the photovoltaic cleaning robot is at any position in a row or a column, it determines the next turning position, turns along the nearest edge first, and cleans according to the current traversal mode path.
Citation Information
Cited By
Control method and device of photovoltaic cleaning robot, electronic equipment, storage medium and program product
CN119567276A
IMU-based vehicle driving speed dynamic monitoring method
CN119636762A
