Photovoltaic module grabbing method based on point cloud and RGBD image

Abstract

The invention relates to the technical field of automatic robots, in particular to a photovoltaic module grabbing method based on point cloud and RGBD images. According to the method, the position of the photovoltaic panel is recognized in a two-step walking mode, and a mechanical arm is guided to automatically grab and place the photovoltaic panel. And the automation level, precision and efficiency of photovoltaic module installation are improved. By combining the high-precision space information of the point cloud data and the color and depth information of the RGBD image, the photovoltaic module can be quickly identified, positioned and accurately grabbed, so that the manual intervention is reduced, and the safety and reliability of the installation process are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of automated robots, and in particular to a photovoltaic component grabbing method based on point cloud and RGBD images. Background Art

[0002] With the rapid development of renewable energy, the demand for photovoltaic module installation is growing. Traditional photovoltaic module installation methods mainly rely on manual operation, which has problems such as high labor intensity, low efficiency, and low installation accuracy. In recent years, the application of automation technology in the manufacturing industry has gradually increased, but in the field of photovoltaic module installation, due to the diversity and irregularity of components, automated grasping technology still faces challenges.

[0003] The automatic gripping system can significantly increase the production and installation speed of photovoltaic modules and reduce labor costs. Compared with traditional manual methods, the automatic gripping equipment can complete the handling and installation of a large number of modules in a shorter time. The automatic system can set the optimal gripping force and speed to ensure that each module is properly handled, reduce damage and waste, and the automatic system can better meet the market's growing demand for photovoltaic modules and ensure timely delivery, so the automatic gripping method of photovoltaic modules is extremely important.

[0004] Currently, the most common automated grasping methods for photovoltaic modules rely on 2D image processing technology. However, these methods often fail to achieve the desired accuracy and robustness when dealing with photovoltaic modules with complex surfaces and spatial structures. In addition, the lack of depth information also limits the formulation and execution of grasping strategies. Summary of the invention

[0005] In order to solve the technical problem that the automated grasping method relies on 2D image processing technology, which is often difficult to achieve ideal accuracy and robustness when processing photovoltaic modules with complex surfaces and spatial structures, the purpose of the present invention is to provide a photovoltaic module grasping method based on point cloud and RGBD image, and the technical solution adopted is as follows:

[0006] Generate initial environmental point cloud data and use 3D target recognition algorithm to identify the location of the material transport vehicle;

[0007] Guide the end gripper of the robot arm to the position above the material transport vehicle so that the photovoltaic modules on the material transport vehicle enter the field of view of the RGBD camera on the gripper;

[0008] The RGBD camera on the end gripper of the robotic arm is used to predict the pose of the photovoltaic module in real time through the pose estimation algorithm. According to the predicted pose of the photovoltaic module, the end gripper of the robotic arm is guided to move toward the photovoltaic module to grab and lift the photovoltaic module.

[0009] Generate environmental point cloud data again and identify the locations of components to be installed on the photovoltaic bracket;

[0010] Guide the end gripper of the robot arm to the position above the component to be installed, so that the purlin on the bracket enters the field of view of the RGBD camera on the gripper;

[0011] The RGBD camera on the gripper at the end of the robotic arm is used to predict the pose of the purlin in real time through the pose estimation algorithm. According to the predicted pose of the purlin, the gripper at the end of the robotic arm is guided to move toward the purlin, align the position of the photovoltaic module and the purlin, and release the gripper to install the photovoltaic panel.

[0012] Preferably, the real-time prediction of the posture of the photovoltaic assembly by a posture estimation algorithm includes:

[0013] Design a grasping pose estimation network model: The grasping pose estimation network model uses RGB-D input and adopts an encoding-decoding structure to perform end-to-end training on the grasping pose estimation network model.

[0014] Preferably, the end-to-end training of the grasping pose estimation network model using an encoder-decoder structure includes:

[0015] The encoder consists of a downsampling module and a residual module, and a depth-wise separable convolution residual module is added after the downsampling module;

[0016] Depth-wise separable convolution decomposes the traditional convolution into depth-wise convolution and point-wise convolution.

[0017] The calculation formula for the standard convolution parameter p1 and the calculation amount F1 is:

[0018] P1=H k ×W k ×C in ×C out ;

[0019] F1=H k ×W k ×C in ×H out ×W out ×C out ;

[0020] Among them, H k is the height of the convolution kernel; W k is the width of the convolution kernel; C in is the number of input feature map channels; C out is the number of channels of the output feature map; H out is the height of the output feature map; W out is the width of the output feature map;

[0021] The calculation formula for the depth-separable convolution parameter amount P2 and the calculation amount F2 is:

[0022] P2=H k ×W k ×C in ×1+1×1×C in ×C out ;

[0023] F2=H k ×W k ×C in ×H out ×W out ×1+H out ×W out ×C in ×C out ×1×1;

[0024] The ratio of the number of parameters of depthwise separable convolution and standard convolution is R P , the ratio of the amount of calculation R F The formula is as follows:

[0025]

[0026] Preferably, the design of the depth-separable convolutional residual module includes:

[0027] The standard convolution in the traditional residual module is replaced by depthwise separable convolution. The 3×3 depthwise convolution convolves each input channel with a different convolution kernel to generate multiple output channel feature maps, and then stacks the results of multiple channels together to form an output tensor.

[0028] After the 3×3 depth convolution, a 1×1 point-by-point convolution is added to perform inter-channel information fusion. Each pixel in each input feature map is multiplied by the weight matrix and accumulated to obtain each output pixel.

[0029] Preferably, a BN layer and a Relu activation function are added to the residual module.

[0030] Preferably, the grasping detection when grasping the photovoltaic module includes:

[0031] Predict the target's grasping rectangle on the image and obtain the grasping posture: g = (p, w, θ), where p is the grasping coordinate; w is the width of the gripper's opening; and θ is the grasping angle at each graspable point.

[0032] Preferably, the loss function of the estimated network model for grasping posture includes:

[0033]

[0034]

[0035] in, is the loss function of the estimated network model for grasping pose, is the true value; is the predicted value; N is the number of grasping rectangles; i is the serial number corresponding to the grasping rectangle; p is the grasping coordinate; w is the width of the gripper of the gripper; θ is the grasping angle at each graspable point; cos(2θ) and sin(2θ) are the postures of the grasping angle; m i is the set of postures of the grasping coordinates and grasping angles of the graspable points in the i-th grasping rectangle.

[0036] The embodiments of the present invention have at least the following beneficial effects:

[0037] High-precision positioning and identification: Through a two-step approach of point cloud positioning followed by RGBD pose estimation, high-precision positioning and identification of material transport vehicles and photovoltaic brackets are achieved, ensuring the accuracy of robotic arm operation.

[0038] Real-time pose prediction: Through the depth information and color information obtained by the RGBD camera, the pose estimation algorithm can predict the pose of photovoltaic modules and purlins in real time, improving the response speed and flexibility of the robot arm operation.

[0039] Automation of operation: The whole process from identification and positioning to grabbing, lifting and placing is automated, which reduces manual intervention, reduces the labor intensity of operators, and improves operation efficiency and safety.

[0040] Improve installation accuracy: The pose estimation algorithm and precise control of the robotic arm ensure the position accuracy of the PV panels during installation, which helps to improve the installation quality and performance of the entire PV system. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 A method flow chart of a photovoltaic component grabbing method based on point cloud and RGBD image provided by one embodiment of the present invention;

[0043] Figure 2 A schematic diagram of a network structure of PointNet++ provided by an embodiment of the present invention;

[0044] Figure 3 A schematic diagram of the result of depthwise separable convolution provided by one embodiment of the present invention;

[0045] Figure 4 A schematic diagram of a convolutional residual module network structure provided by an embodiment of the present invention;

[0046] Figure 5 A schematic diagram of a grab detection process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the photovoltaic component grabbing method based on point cloud and RGBD image proposed by the present invention, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0048] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0049] The following is a detailed description of a photovoltaic component grabbing method based on point cloud and RGBD image provided by the present invention in conjunction with the accompanying drawings.

[0050] See also Figure 1 , which shows a flowchart of a photovoltaic component grabbing method based on point cloud and RGBD image provided by an embodiment of the present invention, the method comprising the following steps:

[0051] Step S100, generating initial environmental point cloud data, and using a three-dimensional target recognition algorithm to identify the position of the material transport vehicle.

[0052] The environmental point cloud data is generated using a Lidar camera, and the position of the material transport vehicle is identified using a three-dimensional target recognition algorithm. It should be noted that identifying the position of the target object using a three-dimensional target recognition algorithm is a well-known technology for those skilled in the art and will not be described in detail here.

[0053] See also Figure 2 , Figure 2The diagram is a schematic diagram of the network structure of PointNet++. Sampling & grouping is sampling grouping; pointnet is the model name; Hierarchical point set feature learning is the hierarchical point set feature learning model; Segmentation is segmentation; Interpolate is interpolation; fully connected layers is the fully connected layer; skip link concatenation is skip connection. It focuses on the local neighborhood feature information. By sampling the point cloud at different scales, and then using PointNet++ at each scale to extract local features. This can capture the local structural information of the point cloud at multiple scales, thereby improving the robustness of the model.

[0054] The backbone network contains multiple SA modules (set abstraction), i.e., sampling-grouping structures. The sampling layer (sampling) is the farthest sampling point mentioned above. Grouping is to select several center points as local points, and use the PointNet++ network to extract features from these local key points. Through multi-level and multi-scale feature learning, the network can obtain multi-level local point cloud features, similar to the concept of receptive field in convolutional networks. This enables the model to learn features of different scales in the point cloud, thereby improving the robustness and accuracy of the model. There are three points worth noting:

[0055] (1) Local feature extraction:

[0056] A distance-based farthest point sampling (FPS) selection strategy is introduced. The execution flow of this process includes:

[0057] 1. In the initial stage, a point in the point cloud is randomly selected as the starting point;

[0058] 2. Calculate the Euclidean distance between the starting point and the remaining points in the data set, and select the point with the farthest Euclidean distance to add to the point set to become the second point;

[0059] 3. For the existing selected point set, find the maximum distance from the selected point to the remaining points, and find the remaining points corresponding to the maximum distance of the selected points, and use them as the new selected point set;

[0060] 4. Continue to execute step 3 until the number of points in the selected point set reaches the predetermined number of points.

[0061] After the above steps of the farthest sampling point selection strategy based on distance are processed, a uniformly distributed point set will eventually be formed.

[0062] In the local area, the PointNet++ network is used to extract local geometric features. Secondly, the neighborhood range is gradually expanded to extract multi-scale and multi-level features until the global features of the entire point cloud are obtained.

[0063] (2) Multi-scale features:

[0064] In order to solve the problem of uneven distribution of point clouds, two solutions are adopted, namely multi-scale grouping (MSG) and multi-resolution grouping (MRG).

[0065] The MSG method performs group processing at different scales. First, a specific center point is selected. If there are multiple different scales, a region is defined for each scale, and these regions are input into independent PointNet++ models to extract features. The extracted features are fused to form the multi-scale overall features of the center point. This process is equivalent to keeping the center point fixed, but changing the size of the region to adapt to different scales, and merging the features of these different scales.

[0066] The MRG method performs grouping and feature extraction at different resolution levels, and merges features at different levels. Due to the difference in scale, low-level data is usually first input into the PointNet++ model for processing, and then the obtained features are merged with high-level features for further analysis. Therefore, this method does not increase excessive computational consumption compared to MSG.

[0067] (3) Decoding principle:

[0068] The decoding process redistributes the information aggregated from the local space to the entire point cloud set, thereby assigning features to each initial point. The core is to use the upsampling mechanism to pass high-level point features down to lower levels until the level of the initial point cloud is reached. This process mainly uses interpolation technology to achieve the diffusion of point features, and integrates feature information from the corresponding encoding layer through jump connections between layers.

[0069] Step S200, guiding the gripper at the end of the robot arm to reach above the position of the material transport vehicle, so that the photovoltaic components on the material transport vehicle enter the field of view of the RGBD camera on the gripper.

[0070] Step S300, using the RGBD camera on the end gripper of the robotic arm to predict the pose of the photovoltaic module in real time through a pose estimation algorithm; according to the predicted pose of the photovoltaic module, guide the end gripper of the robotic arm to move toward the photovoltaic module, grab and lift the photovoltaic module.

[0071] Design of the grasping pose estimation network model: The grasping pose estimation network model uses RGB-D input, that is, it fuses color information and depth information. The grasping pose estimation network model is trained end-to-end using the encoder-decoder structure.

[0072] Among them, the encoder is responsible for extracting features, which is mainly composed of a downsampling module and a residual module, that is, the encoder consists of a downsampling module and a residual module. The downsampling module can reduce the amount of calculation and reduce the redundant information of the feature map, while also reducing the image resolution. However, since the downsampling module will lose information, adding a depth-separable convolution residual module after the downsampling module can help the network better retain features, thereby improving the accuracy of the model. The depth-separable convolution residual module can bridge the feature maps between the previous and next two consecutive blocks, and allow the shallower layers to better utilize the deep features, thereby improving the network's expressive power. The main task of the decoder is to restore the high-level feature information extracted by the encoder to an output space of the same size as the original input.

[0073] Among them, a depth-wise separable convolution residual module is added after the downsampling module;

[0074] Depthwise separable convolution decomposes the traditional convolution into depthwise convolution and pointwise convolution.

[0075] The first step is depth convolution, which uses a convolution kernel for each channel without changing the channel depth, and then concatenates the outputs of all convolution kernels to get the final output;

[0076] The point-by-point convolution in the second step is actually a 1×1 standard convolution, which fuses the features extracted by the deep convolution to preserve the feature information as much as possible. Figure 3 , Figure 3 Schematic diagram of the result of depth-wise separable convolution.

[0077] The calculation formula for the standard convolution parameter P1 and the calculation amount F1 is:

[0078] P1=H k ×W k ×C in ×C out ;

[0079] F1=H k ×W k ×C in ×H out ×W out ×C out ;

[0080] Among them, H k is the height of the convolution kernel; W k is the width of the convolution kernel; C in is the number of input feature map channels; C out is the number of channels of the output feature map; W out is the height of the output feature map; W out is the width of the output feature map;

[0081] The calculation formula for the depth-separable convolution parameter amount P2 and the calculation amount F2 is:

[0082] P2=H k ×W k ×C in ×1+1×1×C in ×C out ;

[0083] F2=H k ×W k ×C in ×H out ×W out ×1+H out ×W out ×C in ×C out ×1×1;

[0084] The ratio of the number of parameters of depthwise separable convolution and standard convolution is R P , the ratio of the amount of calculation R F The formula is as follows:

[0085]

[0086]

[0087] Furthermore, we design a depth-wise separable convolutional residual module:

[0088] Depthwise separable convolution is used to replace the standard convolution in the traditional residual module. The 3×3 depthwise convolution convolves each input channel with different convolution kernels to generate multiple output channel feature maps, and then stacks the results of multiple channels together to form an output tensor. After the 3×3 depthwise convolution, a 1×1 point-by-point convolution is added to fuse information between channels. Each pixel in each input feature map is multiplied by the weight matrix and accumulated to obtain each pixel of the output.

[0089] The traditional residual module is optimized, and the standard convolution in the traditional residual module is replaced by the depth separable convolution. The depth separable convolution residual block (DSCRB) is designed. Please refer to Figure 4, Figure 4 This is a schematic diagram of the network structure of the convolutional residual module. The standard convolution Conv in the traditional residual structure is replaced by a depthwise separable convolution. The 3×3 depthwise convolution (Dw Conv) performs convolution operations on each input channel with different convolution kernels to generate multiple output channel feature maps, and then stacks the results of multiple channels together to form an output tensor. The 3×3 depthwise convolution can not only capture local features, but also avoid the gradient disappearance and gradient explosion problems, and can reduce the computational burden and number of parameters of the model without affecting the performance of the model. However, this method only performs convolution in the channel dimension, not in the spatial dimension, so a 1×1 point-by-point convolution is added after the 3×3 depthwise convolution.

[0090] The 1×1 point-by-point convolution is regarded as a fully connected layer in the channel dimension, which performs information fusion between channels, multiplies each pixel in each input feature map by the weight matrix and accumulates them to obtain each pixel in the output. When the number of input channels remains unchanged, the depth and complexity of the network are changed by adjusting the number of output channels to ensure that the model extracts more effective features.

[0091] Finally, the BN layer and Relu activation function are added to the residual module to speed up the convergence of the network and increase the generalization ability of the model. The BN layer can normalize the input of the convolutional layer or the fully connected layer, effectively accelerating the training of the neural network while keeping the shape of the data distribution unchanged. The Relu activation function can introduce nonlinear characteristics, which can support faster implementation speed and better gradient descent while achieving sparse connections.

[0092] Furthermore, the photovoltaic module is grasped and detected, specifically:

[0093] The purpose of grasp detection is to predict the grasping rectangle of the target on the image and obtain the grasping posture. Figure 5 , Figure 5 To obtain and mark the schematic diagram of the target area from the captured image, realize the input of the original image information, obtain the algorithm flow chart of the area to be grasped, and finally output the pose of the object to be grasped; Figure 5 RGB+depth is the color map + depth map, that is, RGBD; Qulily is the image quality information (target position); Angle is the target angle; Width is the target width.

[0094] The grasping position is located as follows: g = (p, w, θ), where p is the grasping coordinate; w is the width of the gripper of the gripper; θ is the grasping angle at each graspable point, and the value range of the grasping angle is In order to solve the angle regression To solve the discontinuity problem, the angles are encoded as cos(2θ) and sin(2θ), and the formula is

[0095] Furthermore, the grasping posture estimation problem can be regarded as a regression problem, and a grasping posture estimation network model can be constructed. L1 The function is used as the loss function for network model training.

[0096] The loss function of the estimated network model for grasping posture is:

[0097]

[0098] in, is the loss function of the estimated network model for grasping pose, is the true value; is the predicted value; N is the number of grasping rectangles; i is the serial number corresponding to the grasping rectangle; p is the grasping coordinate; w is the width of the gripper of the gripper; θ is the grasping angle at each graspable point; cos(2θ) and sin(2θ) are the postures of the grasping angle; m i is the set of postures of the grasping coordinates and grasping angles of the graspable points in the i-th grasping rectangle.

[0099] Step S400, generating environmental point cloud data again and identifying the positions of components to be installed on the photovoltaic support.

[0100] The Lidar camera is used again to generate environmental point cloud data, and the three-dimensional target recognition algorithm is used to identify the position of the components to be installed on the photovoltaic bracket. It should be noted that the method of generating environmental point cloud data again is the same as the method in step S100, and the method of identifying the position of the components to be installed on the photovoltaic bracket is the same as the method of identifying the position of the material transport vehicle in step S100.

[0101] Step S500, guiding the end gripper of the robot arm to reach above the position of the component to be installed, so that the purlin on the bracket enters the field of view of the RGBD camera on the gripper.

[0102] Step S600, using the RGBD camera on the end gripper of the robotic arm, the pose of the purlin is predicted in real time through the pose estimation algorithm; according to the predicted pose of the purlin, the end gripper of the robotic arm is guided to move toward the purlin, align the position of the photovoltaic module and the purlin, and release the gripper to install the photovoltaic panel.

[0103] It should be noted that the method of predicting the posture of the purlin in real time by using the posture estimation algorithm is the same as the method of predicting the posture of the photovoltaic module in real time by using the posture estimation algorithm in step S300.

[0104] The present invention aims to provide a photovoltaic module grasping method based on point cloud and RGBD images, which identifies the position of photovoltaic panels in a two-step manner and guides the robotic arm to automatically grasp and place photovoltaic panels. It can improve the automation level, accuracy and efficiency of photovoltaic module installation. By combining the high-precision spatial information of point cloud data and the color and depth information of RGBD images, the present invention can realize the rapid identification, positioning and accurate grasping of photovoltaic modules, thereby reducing manual intervention and improving the safety and reliability of the installation process.

[0105] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0106] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A photovoltaic module grabbing method based on point cloud and RGBD image, characterized in that: The method comprises the following steps: Generate initial environmental point cloud data and use 3D target recognition algorithm to identify the location of the material transport vehicle; Guide the end gripper of the robot arm to the position above the material transport vehicle so that the photovoltaic modules on the material transport vehicle enter the field of view of the RGBD camera on the gripper; The RGBD camera on the end gripper of the robotic arm is used to predict the pose of the photovoltaic module in real time through the pose estimation algorithm. According to the predicted pose of the photovoltaic module, the end gripper of the robotic arm is guided to move toward the photovoltaic module to grab and lift the photovoltaic module. Generate environmental point cloud data again and identify the locations of components to be installed on the photovoltaic bracket; Guide the end gripper of the robot arm to the position above the component to be installed, so that the purlin on the bracket enters the field of view of the RGBD camera on the gripper; The RGBD camera on the gripper at the end of the robotic arm is used to predict the pose of the purlin in real time through the pose estimation algorithm. According to the predicted pose of the purlin, the gripper at the end of the robotic arm is guided to move toward the purlin, align the position of the photovoltaic module and the purlin, and release the gripper to install the photovoltaic panel.

2. The photovoltaic component grabbing method based on point cloud and RGBD image according to claim 1 is characterized in that: The real-time prediction of the posture of the photovoltaic module by the posture estimation algorithm includes: Design a grasping pose estimation network model: The grasping pose estimation network model uses RGB-D input and adopts an encoding-decoding structure to perform end-to-end training on the grasping pose estimation network model.

3. The photovoltaic component grabbing method based on point cloud and RGBD image according to claim 2 is characterized in that: The encoder-decoder structure is used to perform end-to-end training on the grasping pose estimation network model, including: The encoder consists of a downsampling module and a residual module, and a depth-wise separable convolution residual module is added after the downsampling module; Depth-wise separable convolution decomposes the traditional convolution into depth-wise convolution and point-wise convolution. The calculation formula for the standard convolution parameter p1 and the calculation amount F1 is: P1=H k ×W k ×C in ×C out ; F1=H k ×W k ×C in ×H out ×W out ×C out ; Among them, H k is the height of the convolution kernel; W k is the width of the convolution kernel; C in is the number of input feature map channels; C out is the number of channels of the output feature map; H out is the height of the output feature map; W out is the width of the output feature map; The calculation formula for the depth-separable convolution parameter amount P2 and the calculation amount F2 is: P2=H k ×W k ×C in ×1+1×1×C in ×C out ; F2=H k ×W k ×C in ×H out ×W out ×1+H out ×W out ×C in ×C out ×1×1; The ratio of the number of parameters of depthwise separable convolution and standard convolution is R P , the ratio of the amount of calculation R F The formula is as follows:

4. The photovoltaic component grabbing method based on point cloud and RGBD image according to claim 3 is characterized in that: The design of the depth-separable convolutional residual module includes: The standard convolution in the traditional residual module is replaced by depthwise separable convolution. The 3×3 depthwise convolution convolves each input channel with a different convolution kernel to generate multiple output channel feature maps, and then stacks the results of multiple channels together to form an output tensor. After the 3×3 depth convolution, a 1×1 point-by-point convolution is added to fuse information between channels. Each pixel in each input feature map is multiplied by the weight matrix and accumulated to obtain each output pixel.

5. The photovoltaic component grabbing method based on point cloud and RGBD image according to claim 3 is characterized in that: Add BN layer and Relu activation function to the residual module.

6. The photovoltaic component grabbing method based on point cloud and RGBD image according to claim 1, characterized in that: Grasping detection when grasping photovoltaic modules, including: Predict the target's grasping rectangle on the image and obtain the grasping posture: g = (p, w, θ), where p is the grasping coordinate; w is the width of the gripper's opening; and θ is the grasping angle at each graspable point.

7. The photovoltaic component grabbing method based on point cloud and RGBD image according to claim 6, characterized in that: The loss function of the network model for estimating the grasping posture includes: in, is the loss function of the estimated network model for grasping pose, is the true value; is the predicted value; N is the number of grasping rectangles; i is the serial number corresponding to the grasping rectangle; p is the grasping coordinate; w is the width of the gripper of the gripper; θ is the grasping angle at each graspable point; cos(2θ) and sin(2θ) are the postures of the grasping angle; m i is the set of postures of the grasping coordinates and grasping angles of the graspable points in the i-th grasping rectangle.

Images