A photovoltaic module grasping method based on point cloud and RGBD image

Abstract

This invention relates to the field of automated robotics, specifically a photovoltaic module grasping method based on point cloud and RGBD images. This method uses a two-step approach to identify the location of photovoltaic panels and guide a robotic arm to automatically grasp and place them. This improves the automation level, accuracy, and efficiency of photovoltaic module installation. By combining the high-precision spatial information of point cloud data with the color and depth information of RGBD images, the present invention enables rapid identification, positioning, and accurate grasping of photovoltaic modules, thereby reducing manual intervention and improving the safety and reliability of the installation process.

Description

Technical Field

[0001] The present invention relates to the technical field of automated robots, and in particular to a photovoltaic component grasping method based on point cloud and RGBD images. Background Art

[0002] With the rapid development of renewable energy, the demand for photovoltaic (PV) panel installation is growing. Traditional PV panel installation methods rely primarily on manual labor, which is labor-intensive, inefficient, and lacks precision. While the application of automation technology in the manufacturing industry has been increasing in recent years, automated handling technology remains challenging in PV panel installation due to the diverse and irregular nature of the components.

[0003] Automated handling systems can significantly speed up the production and installation of photovoltaic modules, reducing labor costs. Compared to traditional manual methods, automated handling equipment can handle and install large numbers of modules in a shorter time. Automatic systems can set the optimal gripping force and speed to ensure that each module is properly handled, reducing damage and waste. Automated systems can also better meet the growing market demand for photovoltaic modules and ensure timely delivery. Therefore, automated handling methods for photovoltaic modules are extremely important.

[0004] Currently, the most common automated grasping methods for photovoltaic modules rely on 2D image processing technology. However, these methods often fail to achieve the desired accuracy and robustness when processing photovoltaic modules with complex surfaces and spatial structures. In addition, the lack of depth information also limits the formulation and execution of grasping strategies. Summary of the Invention

[0005] To address the technical problem that automated grasping methods rely on 2D image processing technology, which often struggles to achieve ideal accuracy and robustness when processing photovoltaic modules with complex surfaces and spatial structures, the present invention aims to provide a photovoltaic module grasping method based on point clouds and RGBD images. The technical solutions employed are as follows:

[0006] Generate initial environmental point cloud data and use a 3D target recognition algorithm to identify the location of the material transport vehicle;

[0007] Guide the gripper at the end of the robotic arm to the position above the material transport vehicle so that the photovoltaic panels on the vehicle enter the field of view of the RGBD camera on the gripper;

[0008] Using the RGBD camera on the end-gripper of the robotic arm, the pose estimation algorithm predicts the pose of the photovoltaic module in real time. Based on the predicted pose of the photovoltaic module, the end-gripper of the robotic arm is guided to move toward the photovoltaic module, grasping and lifting the photovoltaic module.

[0009] Generate environmental point cloud data again and identify the locations of components to be installed on the photovoltaic rack;

[0010] Guide the end gripper of the robotic arm to the position above the component to be installed, so that the purlin on the bracket is within the field of view of the RGBD camera on the gripper;

[0011] Using the RGBD camera on the end gripper of the robotic arm, the pose estimation algorithm is used to predict the pose of the purlin in real time. Based on the predicted purlin pose, the end gripper of the robotic arm is guided toward the purlin, aligning the position of the photovoltaic module and the purlin, and then the gripper is released to complete the photovoltaic panel installation.

[0012] Preferably, the real-time prediction of the posture of the photovoltaic assembly by using a posture estimation algorithm includes:

[0013] Design a grasping pose estimation network model: The grasping pose estimation network model uses RGB-D input and adopts an encoding-decoding structure to perform end-to-end training on the grasping pose estimation network model.

[0014] Preferably, the encoder-decoder structure is used to perform end-to-end training on the grasping pose estimation network model, including:

[0015] The encoder consists of a downsampling module and a residual module, and a depth-wise separable convolution residual module is added after the downsampling module;

[0016] Depthwise separable convolution decomposes the traditional convolution into depthwise convolution and pointwise convolution operations;

[0017] The calculation formula for the standard convolution parameter p1 and the calculation amount F1 is:

[0018] P1=H k ×W k ×C in ×C out ;

[0019] F1=H k ×W k ×C in ×H out ×W out ×C out ;

[0020] Among them, H k is the height of the convolution kernel; W k is the width of the convolution kernel; C in is the number of input feature map channels; C out is the number of channels of the output feature map; H out is the height of the output feature map; W out is the width of the output feature map;

[0021] The calculation formula for the depth-separable convolution parameter amount P2 and the computation amount F2 is:

[0022] P2=H k ×W k ×C in ×1+1×1×C in ×C out ;

[0023] F2=H k ×W k ×C in ×H out ×W out ×1+H out ×W out ×C in ×C out ×1×1;

[0024] The ratio of the number of parameters of depth-wise separable convolution and standard convolution R P , the ratio of computational effort R F The formula is as follows:

[0025]

[0026] Preferably, the design of the depthwise separable convolutional residual module includes:

[0027] The standard convolution in the traditional residual module is replaced by depthwise separable convolution; the 3×3 depthwise convolution performs convolution operations on each input channel with different convolution kernels to generate multiple output channel feature maps, and then stacks the results of multiple channels together to form the output tensor;

[0028] After the 3×3 depth convolution, a 1×1 point-by-point convolution is added to perform inter-channel information fusion. Each pixel in each input feature map is multiplied by the weight matrix and accumulated to obtain each output pixel.

[0029] Preferably, a BN layer and a ReLU activation function are added to the residual module.

[0030] Preferably, the grasping detection when grasping the photovoltaic module includes:

[0031] Predict the target's grasping rectangle on the image and obtain the grasping pose: g = (p, w, θ), where p is the grasping coordinate; w is the width of the gripper's opening; and θ is the grasping angle at each graspable point.

[0032] Preferably, the loss function of the grasping pose estimation network model includes:

[0033]

[0034]

[0035] in, is the loss function of the estimated network model for grasping pose, is the true value; is the predicted value; N is the number of grasping rectangles; i is the serial number corresponding to the grasping rectangle; p is the grasping coordinate; w is the width of the gripper of the gripper; θ is the grasping angle at each graspable point; cos(2θ) and sin(2θ) are the postures of the grasping angle; m i is the set of grasping coordinates and grasping angles of the graspable points in the i-th grasping rectangle.

[0036] The embodiments of the present invention have at least the following beneficial effects:

[0037] High-precision positioning and identification: Through a two-step approach of point cloud positioning followed by RGBD pose estimation, high-precision positioning and identification of material transport vehicles and photovoltaic brackets are achieved, ensuring the accuracy of robotic arm operation.

[0038] Real-time pose prediction: Using the depth and color information obtained by the RGBD camera, the pose estimation algorithm can predict the pose of photovoltaic modules and purlins in real time, improving the response speed and flexibility of the robot arm operation.

[0039] Automated operation: The entire process from identification and positioning to grabbing, lifting and placement is automated, reducing manual intervention, lowering the labor intensity of operators, and improving operational efficiency and safety.

[0040] Improved installation accuracy: The pose estimation algorithm and precise control of the robotic arm ensure the positional accuracy of PV panels during installation, helping to improve the installation quality and performance of the entire PV system. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A flowchart of a photovoltaic module grabbing method based on point cloud and RGBD image provided by one embodiment of the present invention;

[0043] Figure 2 A schematic diagram of the network structure of PointNet++ provided by one embodiment of the present invention;

[0044] Figure 3 A schematic diagram of the result of depthwise separable convolution provided by one embodiment of the present invention;

[0045] Figure 4 A schematic diagram of the network structure of a convolutional residual module provided by one embodiment of the present invention;

[0046] Figure 5 A schematic diagram of a grabbing detection process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] To further illustrate the technical means and effectiveness of the present invention in achieving its intended objectives, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of a photovoltaic module grasping method based on point clouds and RGBD images. In the following description, references to different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0048] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0049] The following describes in detail a specific solution of a photovoltaic module grabbing method based on point cloud and RGBD image provided by the present invention with reference to the accompanying drawings.

[0050] See also Figure 1 , which shows a flowchart of a photovoltaic module grabbing method based on point cloud and RGBD image provided by one embodiment of the present invention, the method includes the following steps:

[0051] Step S100: Generate initial environmental point cloud data and use a three-dimensional target recognition algorithm to identify the position of the material transport vehicle.

[0052] A Lidar camera is used to generate environmental point cloud data, and a 3D object recognition algorithm is used to identify the position of the material transport vehicle. It should be noted that identifying the position of the target object using a 3D object recognition algorithm is a well-known technique for those skilled in the art and will not be described in detail here.

[0053] See also Figure 2 , Figure 2A schematic diagram of the PointNet++ network architecture. "Sampling & grouping" refers to sampling and grouping; "pointnet" is the model name; "Hierarchical point set feature learning" refers to the hierarchical point set feature learning model; "Segmentation" refers to segmentation; "Interpolate" refers to interpolation; "fully connected layers" refers to fully connected layers; and "skip link concatenation" refers to skip connections. It focuses on local neighborhood feature information. By sampling the point cloud at different scales and then using PointNet++ to extract local features at each scale, it captures the local structural information of the point cloud at multiple scales, thereby improving the model's robustness.

[0054] The backbone network contains multiple SA modules (set abstraction), which are sampling-grouping structures. The sampling layer (sampling) is the farthest sampling point mentioned above. Grouping is to select several central points as local points and use the PointNet++ network to extract features from these local key points. Through multi-level and multi-scale feature learning, the network can obtain multi-level local point cloud features, similar to the concept of receptive field in convolutional networks. This enables the model to learn features at different scales in the point cloud, thereby improving the robustness and accuracy of the model. There are three points worth noting:

[0055] (1) Local feature extraction:

[0056] A distance-based farthest point sampling (FPS) selection strategy is introduced. The execution process of this process includes:

[0057] 1. In the initial stage, any point in the point cloud is selected as the starting point;

[0058] 2. Calculate the Euclidean distance between the starting point and the remaining points in the data set, and select the point with the farthest Euclidean distance to add to the point set as the second point;

[0059] 3. For the existing selected point set, find the maximum distance from the selected point to the remaining points, and find the remaining points corresponding to the maximum distance from the selected point, and use them as the new selected point set;

[0060] 4. Continue to execute step 3 until the number of points in the selected point set reaches the predetermined number of points.

[0061] After the above steps of the farthest sampling point selection strategy based on distance are processed, a uniformly distributed point set will eventually be formed.

[0062] In a local area, the PointNet++ network is used to extract local geometric features. Then, the neighborhood range is gradually expanded to extract multi-scale and multi-level features until the global features of the entire point cloud are obtained.

[0063] (2) Multi-scale features:

[0064] In order to solve the problem of uneven distribution of point clouds, two solutions are adopted, namely multi-scale grouping (MSG) and multi-resolution grouping (MRG).

[0065] The MSG method performs grouping processing at different scales. First, a specific center point is selected. If multiple scales exist, a region is defined for each scale and fed into separate PointNet++ models for feature extraction. The extracted features are then fused to form a multi-scale holistic feature of the center point. This process is equivalent to keeping the center point fixed but varying the size of the regions to accommodate different scales, thereby merging features from these different scales.

[0066] The MRG method groups and extracts features at different resolution levels, merging features from different levels. Due to scale differences, low-level data is typically first fed into the PointNet++ model for processing, and the resulting features are then merged with high-level features for further analysis. Therefore, this method does not incur significant computational overhead compared to MSG.

[0067] (3) Decoding principle:

[0068] The decoding process redistributes the information aggregated from the local space to the entire point cloud, thereby assigning features to each initial point. The core of this process is to utilize an upsampling mechanism to propagate high-level point features downward to lower levels until they reach the level of the initial point cloud. This process primarily uses interpolation techniques to achieve the diffusion of point features, and skip connections between layers integrate feature information from corresponding encoding layers.

[0069] Step S200: guide the end gripper of the robotic arm to the position above the material transport vehicle so that the photovoltaic components on the material transport vehicle enter the field of view of the RGBD camera on the gripper.

[0070] In step S300, the RGBD camera on the end gripper of the robotic arm is used to predict the pose of the photovoltaic module in real time through a pose estimation algorithm; based on the predicted pose of the photovoltaic module, the end gripper of the robotic arm is guided to move toward the photovoltaic module to grab and lift the photovoltaic module.

[0071] Designing a grasping pose estimation network model: The grasping pose estimation network model uses RGB-D input, fusing color and depth information. It is trained end-to-end using an encoder-decoder architecture.

[0072] Among them, the encoder is responsible for extracting features and is mainly composed of a downsampling module and a residual module, that is, the encoder consists of a downsampling module and a residual module. The downsampling module can reduce the amount of calculation and reduce the redundant information of the feature map, but it will also reduce the image resolution. However, since the downsampling module will lose information, adding a depthwise separable convolution residual module after the downsampling module can help the network better retain features, thereby improving the accuracy of the model. The depthwise separable convolution residual module can bridge the feature maps between the previous and next two consecutive blocks, and allow shallower layers to better utilize deep features, thereby improving the network's expressive power. The main task of the decoder is to restore the high-level feature information extracted by the encoder to an output space of the same size as the original input.

[0073] Among them, a depth-wise separable convolution residual module is added after the downsampling module;

[0074] Depthwise separable convolution decomposes the traditional convolution into depthwise convolution and pointwise convolution.

[0075] The first step is depth convolution, which uses a convolution kernel for each channel without changing the channel depth, and then concatenates the outputs of all convolution kernels to get the final output;

[0076] The second step of point-by-point convolution is actually a 1×1 standard convolution, which fuses the features extracted by the depth convolution to preserve the feature information as much as possible. Figure 3 , Figure 3 Schematic diagram of the result of depth-wise separable convolution.

[0077] The calculation formula for the standard convolution parameter P1 and the calculation amount F1 is:

[0078] P1=H k ×W k ×C in ×C out ;

[0079] F1=H k ×W k ×C in ×H out ×W out ×C out ;

[0080] Among them, H k is the height of the convolution kernel; W k is the width of the convolution kernel; C in is the number of input feature map channels; C out is the number of channels of the output feature map; W out is the height of the output feature map; W out is the width of the output feature map;

[0081] The calculation formula for the depth-separable convolution parameter amount P2 and the computation amount F2 is:

[0082] P2=H k ×W k ×C in ×1+1×1×C in ×C out ;

[0083] F2=H k ×W k ×C in ×H out ×W out ×1+H out ×W out ×C in ×C out ×1×1;

[0084] The ratio of the number of parameters of depth-wise separable convolution and standard convolution R P , the ratio of computational effort R F The formula is as follows:

[0085]

[0086]

[0087] Going further, we design a depth-wise separable convolutional residual module:

[0088] Depthwise separable convolution is used to replace the standard convolution in the traditional residual module; 3×3 depthwise convolution convolves each input channel with different convolution kernels to generate multiple output channel feature maps, and then stacks the results of multiple channels together to form an output tensor; 1×1 point-by-point convolution is added after the 3×3 depthwise convolution to perform inter-channel information fusion, multiplying each pixel in each input feature map with the weight matrix and accumulating them to obtain each output pixel.

[0089] The traditional residual module is optimized and the standard convolution in the traditional residual module is replaced by the depth-separable convolution. The depth-separable convolution residual block (DSCRB) is designed. Figure 4, Figure 4 This is a schematic diagram of the convolutional residual module network structure. The standard convolution Conv in the traditional residual structure is replaced by a depthwise separable convolution. The 3×3 depthwise convolution (Dw Conv) performs a convolution operation on each input channel with a different convolution kernel to generate multiple output channel feature maps, and then stacks the results of multiple channels together to form an output tensor. The 3×3 depthwise convolution can not only capture local features, but also avoid the problems of gradient disappearance and gradient explosion, and can reduce the computational burden and number of parameters of the model without affecting the performance of the model. However, this method only performs convolution in the channel dimension, not in the spatial dimension, so a 1×1 point-by-point convolution is added after the 3×3 depthwise convolution.

[0090] The 1×1 point-by-point convolution is considered a fully connected layer in the channel dimension. It performs inter-channel information fusion, multiplying each pixel in each input feature map by the weight matrix and accumulating the sum to obtain each output pixel. While the number of input channels remains unchanged, the depth and complexity of the network are changed by adjusting the number of output channels, ensuring that the model extracts more effective features.

[0091] Finally, adding a batch normalization layer and Relu activation function to the residual module accelerates network convergence and improves model generalization. The batch normalization layer normalizes the inputs of convolutional or fully connected layers, effectively accelerating neural network training while preserving the shape of the data distribution. The Relu activation function introduces nonlinearity, enabling sparse connections while also supporting faster implementation and better gradient descent.

[0092] Furthermore, the photovoltaic modules are inspected during grabbing, specifically:

[0093] The purpose of grasp detection is to predict the grasping rectangle of the target on the image and obtain the grasping pose. Figure 5 , Figure 5 To obtain and mark the schematic diagram of the target area from the captured image, realize the input of the original image information, obtain the algorithm flow chart of the area to be grasped, and finally output the pose of the object to be grasped; Figure 5 RGB+depth is the color map + depth map, which is RGBD; Qulily is the image quality information (target position); Angle is the target angle; Width is the target width.

[0094] The grasping position is located as follows: g = (p, w, θ), where p is the grasping coordinate; w is the width of the gripper of the gripper; θ is the grasping angle at each graspable point, and the value range of the grasping angle is In order to solve the angle regression To solve the discontinuity problem, we encode the angles as cos(2θ) and sin(2θ), and the formula is

[0095] Furthermore, the grasping pose estimation problem can be regarded as a regression problem, and a grasping pose estimation network model can be constructed. L1 The function serves as the loss function for network model training.

[0096] The loss function of the grasping pose estimation network model is:

[0097]

[0098] in, is the loss function of the estimated network model for grasping pose, is the true value; is the predicted value; N is the number of grasping rectangles; i is the serial number corresponding to the grasping rectangle; p is the grasping coordinate; w is the width of the gripper of the gripper; θ is the grasping angle at each graspable point; cos(2θ) and sin(2θ) are the postures of the grasping angle; m i is the set of grasping coordinates and grasping angles of the graspable points in the i-th grasping rectangle.

[0099] Step S400: Generate environmental point cloud data again and identify the positions of components to be installed on the photovoltaic support.

[0100] The Lidar camera is used again to generate environmental point cloud data, and a 3D object recognition algorithm is used to identify the location of the components to be installed on the photovoltaic rack. It should be noted that the method for generating environmental point cloud data again is the same as the method in step S100, and the method for identifying the location of the components to be installed on the photovoltaic rack is the same as the method for identifying the location of the material transport vehicle in step S100.

[0101] Step S500: guide the end gripper of the robot arm to the position above the component to be installed, so that the purlin on the bracket enters the field of view of the RGBD camera on the gripper.

[0102] In step S600, the RGBD camera on the end gripper of the robotic arm is used to predict the pose of the purlin in real time through a pose estimation algorithm. Based on the predicted pose of the purlin, the end gripper of the robotic arm is guided toward the purlin, aligned with the position of the photovoltaic module and the purlin, and the gripper is released to install the photovoltaic panel.

[0103] It should be noted that the method of predicting the posture of the purlin in real time by using the posture estimation algorithm is the same as the method of predicting the posture of the photovoltaic module in real time by using the posture estimation algorithm in step S300.

[0104] This invention aims to provide a photovoltaic module grasping method based on point cloud and RGBD images. This method uses a two-step approach to identify the location of photovoltaic panels and guide a robotic arm to automatically grasp and place the panels. This method can improve the automation level, accuracy, and efficiency of photovoltaic module installation. By combining the high-precision spatial information of point cloud data with the color and depth information of RGBD images, this invention can achieve rapid identification, positioning, and accurate grasping of photovoltaic modules, thereby reducing manual intervention and improving the safety and reliability of the installation process.

[0105] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0106] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A photovoltaic module grabbing method based on point cloud and RGBD image, characterized in that: The method comprises the following steps: Generate initial environmental point cloud data and use a 3D target recognition algorithm to identify the location of the material transport vehicle; Guide the gripper at the end of the robotic arm to the position above the material transport vehicle so that the photovoltaic panels on the vehicle enter the field of view of the RGBD camera on the gripper; Using the RGBD camera on the end-gripper of the robotic arm, the pose estimation algorithm predicts the pose of the photovoltaic module in real time. Based on the predicted pose of the photovoltaic module, the end-gripper of the robotic arm is guided to move toward the photovoltaic module, grasping and lifting the photovoltaic module. Generate environmental point cloud data again and identify the locations of components to be installed on the photovoltaic rack; Guide the end gripper of the robotic arm to the position above the component to be installed, so that the purlin on the bracket enters the field of view of the RGBD camera on the gripper; Using the RGBD camera on the end gripper of the robotic arm, the pose estimation algorithm is used to predict the pose of the purlin in real time. Based on the predicted purlin pose, the end gripper of the robotic arm is guided toward the purlin, aligning the position of the photovoltaic module and the purlin, and then the gripper is released to complete the photovoltaic panel installation.

2. The photovoltaic module grabbing method based on point cloud and RGBD image according to claim 1 is characterized in that: The real-time prediction of the posture of the photovoltaic module by the posture estimation algorithm includes: Design a grasping pose estimation network model: The grasping pose estimation network model uses RGB-D input and adopts an encoding-decoding structure to perform end-to-end training on the grasping pose estimation network model.

3. The photovoltaic module grabbing method based on point cloud and RGBD image according to claim 2, characterized in that: The encoder-decoder structure is used to perform end-to-end training on the grasping pose estimation network model, including: The encoder consists of a downsampling module and a residual module, and a depth-wise separable convolution residual module is added after the downsampling module; Depthwise separable convolution decomposes the traditional convolution into depthwise convolution and pointwise convolution operations; The calculation formula for the standard convolution parameter p1 and the calculation amount F1 is: P1=H k ×W k ×C in ×C out ; F1=H k ×W k ×C in ×H out ×W out ×C out ; Among them, H k is the height of the convolution kernel; W k is the width of the convolution kernel; C in is the number of input feature map channels; C out is the number of channels of the output feature map; H out is the height of the output feature map; W out is the width of the output feature map; The calculation formula for the depth-separable convolution parameter amount P2 and the computation amount F2 is: P2=H k ×W k ×C in ×1+1×1×C in ×C out ; F2=H k ×W k ×C in ×H out ×W out ×1+H out ×W out ×C in ×C out ×1×1; The ratio of the number of parameters of depth-wise separable convolution and standard convolution R P , the ratio of computational effort R F The formula is as follows:

4. The photovoltaic module grabbing method based on point cloud and RGBD image according to claim 3 is characterized in that: The design of the depth-wise separable convolutional residual module includes: The standard convolution in the traditional residual module is replaced by depthwise separable convolution; the 3×3 depthwise convolution performs convolution operations on each input channel with different convolution kernels to generate multiple output channel feature maps, and then stacks the results of multiple channels together to form the output tensor; After the 3×3 depth convolution, a 1×1 point-by-point convolution is added to perform inter-channel information fusion. Each pixel in each input feature map is multiplied by the weight matrix and accumulated to obtain each output pixel.

5. The photovoltaic module grabbing method based on point cloud and RGBD image according to claim 3, characterized in that: Add BN layer and Relu activation function to the residual module.

6. The photovoltaic module grabbing method based on point cloud and RGBD image according to claim 1, characterized in that: Grasping detection when grasping photovoltaic panels, including: Predict the target's grasping rectangle on the image and obtain the grasping pose: g = (p, w, θ), where p is the grasping coordinate; w is the width of the gripper's opening; and θ is the grasping angle at each graspable point.

7. The photovoltaic module grabbing method based on point cloud and RGBD image according to claim 6, characterized in that: The loss function of the grasp pose estimation network model includes: in, is the loss function of the estimated network model for grasping pose, is the true value; is the predicted value; N is the number of grasping rectangles; i is the serial number corresponding to the grasping rectangle; p is the grasping coordinate; w is the width of the gripper of the gripper; θ is the grasping angle at each graspable point; cos(2θ) and sin(2θ) are the postures of the grasping angle; m i is the set of grasping coordinates and grasping angles of the graspable points in the i-th grasping rectangle.

Images