Calculation method for disordered grabbing of 3D point cloud of photovoltaic junction box

Abstract

The invention discloses a calculation method for disordered grabbing of 3D point clouds of a photovoltaic junction box. The calculation method comprises the following steps: constructing a 2D image and a 3D point cloud model of an arc feature region of a standard photovoltaic junction box; obtaining an overall 3D point cloud containing the storage frame and all the photovoltaic junction boxes; a local 3D point cloud is extracted from the overall 3D point cloud according to the 2D image, and the local 3D point cloud is the 3D point cloud of the arc feature area of the photovoltaic junction box; registering the local 3D point cloud and the 3D point cloud model to obtain a registered point cloud; selecting an optimal grabbing point coordinate from the registered point cloud; and the grabbing posture of the manipulator is calculated by fitting the plane and calculating the rotation matrix. According to the method, the grabbing efficiency can be improved, a large amount of 3D point cloud data can be quickly processed, and the grabbing period of the robot is greatly shortened; the grabbing accuracy is improved, and the risks of product damage and production delay caused by grabbing errors are reduced; the robot can adapt to complex scenes, and reliable grabbing is achieved; the automation degree is improved, and the labor cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of machine vision, and particularly to a calculation method for disorderly grasping of 3D point clouds of photovoltaic junction boxes. Background Art

[0002] With the rise of 3D vision technology, 3D point cloud data can more comprehensively describe the three-dimensional information of an object, bringing new opportunities for solving the problem of disorderly grasping of photovoltaic junction boxes. By obtaining the 3D point cloud data of a photovoltaic junction box and combining advanced algorithms, the accurate recognition of its position and posture can be achieved, and then the robot can be guided to complete the grasping action. In the prior art, there have been many studies and practices on the application of 3D point cloud technology in the grasping of photovoltaic junction boxes. Some enterprises have also made preliminary attempts on the production line and achieved certain results. However, in practical applications, challenges such as the processing speed of point cloud data, the recognition accuracy in complex scenarios, and the grasping stability still exist. Summary of the Invention

[0003] According to an embodiment of the present invention, there is provided a calculation method for disorderly grasping of 3D point clouds of photovoltaic junction boxes, including the following steps: Construct a 2D image and a 3D point cloud model of the arc feature region of a standard photovoltaic junction box; Obtain the overall 3D point cloud including the storage box and all photovoltaic junction boxes; Extract the local 3D point cloud from the overall 3D point cloud according to the 2D image, where the local 3D point cloud is the 3D point cloud of the arc feature region of the photovoltaic junction box; Register the local 3D point cloud and the 3D point cloud model to obtain the registered point cloud; Select the coordinates of the best grasping point in the registered point cloud; Calculate the grasping posture of the manipulator by fitting a plane and calculating the rotation matrix.

[0004] Furthermore, a structured light 3D sensor is used to obtain the overall 3D point cloud including the storage box and all photovoltaic junction boxes.

[0005] Furthermore, extracting the local 3D point cloud from the overall 3D point cloud according to the 2D image, where the local 3D point cloud is the 3D point cloud of the arc feature region of the photovoltaic junction box includes the following steps: Separate the peripheries of the photovoltaic junction box and the storage box in the overall 3D point cloud; According to the 2D image, initially locate the arc feature region of the photovoltaic junction box by 2D template matching, then accurately locate the arc feature region, and crop out the corresponding 3D point cloud.

[0006] Furthermore, separating the peripheries of the photovoltaic junction box and the storage box in the overall 3D point cloud includes the following steps: In the overall 3D point cloud, set a height threshold to identify the four peripheral edges of the storage box; Use the gray-scale region growing algorithm to extract the four peripheral edges of the storage box; Utilize gray-scale image morphology to separate the edge of the storage box and the photovoltaic junction box.

[0007] Furthermore, the precise positioning of the arc feature region includes the following steps: Expand the 2D template matching region; In the expanded 2D template matching region, extract the arc feature points and fit the 2D contour.

[0008] Furthermore, the extraction of the arc feature points and the fitting of the 2D contour include the following steps: Convert the overall 3D point cloud into a gray-scale image; In the gray-scale image, generate a measurement rectangle for the edge according to the arc feature; In the measurement rectangle, extract the feature points according to the set parameters; Use the feature points to fit the 2D contour of the arc feature region.

[0009] Furthermore, the extraction of the feature points according to the set parameters includes the following steps: Set the amplitude to 10, the polarity to 'positive', and the feature point order to'max' for the measurement rectangle of the inner arc; Set the amplitude to 30, the polarity to 'negative', and the feature point order to'max' for the measurement rectangle of the outer arc; Find all the feature points of the inner and outer arcs.

[0010] Furthermore, the selection of the best grasping point coordinates in the registered point cloud includes the following steps: In the registered point cloud, select the area where the bottom does not exist as the grasping area; Calculate the position of the point cloud within the grasping area; When there are multiple grasping positions, compare the Z coordinates of each grasping position and select the point cloud with the largest Z value as the final grasping position.

[0011] Furthermore, through fitting a plane and calculating the rotation matrix, it includes the following steps: Fit a plane according to the 3D point cloud data of the arc feature region of the photovoltaic junction box; Define that the plane normal of the plane is consistent with the Z-axis direction of the manipulator coordinate system; Calculate the rotation matrix of the manipulator from the current pose to the target pose through matrix cross product; Convert the rotation matrix into Euler angles to ensure that the end of the manipulator is always perpendicular to the plane.

[0012] Further, convert the rotation matrix into Euler angles and adjust the range of Euler angles to equivalent the rotation range of the manipulator, ensuring that the end of the manipulator is always perpendicular to the plane.

[0013] According to the calculation method for disordered grasping of the 3D point cloud of the photovoltaic junction box according to the embodiment of the present invention, the grasping efficiency is improved, a large amount of 3D point cloud data can be processed quickly, the grasping cycle of the robot is greatly shortened. Compared with the traditional grasping method, the number of grasps per unit time can be significantly increased, and the production efficiency is improved; the grasping accuracy is increased. By using advanced point cloud feature extraction and matching algorithms, key features such as the edges and corners of the photovoltaic junction box can be accurately identified. Even in a complex disordered stacking environment, its spatial position and attitude can be accurately calculated, greatly improving the accuracy of the robot's grasping and reducing the risks of product damage and production delays caused by grasping mistakes; it can adapt to complex scenarios. Since a structured light sensor is used, the sensor has strong robustness to different lighting conditions, background interference, and different materials and surface textures of the junction box. Whether in an outdoor production environment with strong light or in a dim indoor warehouse, it can stably acquire and process 3D point cloud data to achieve reliable grasping, effectively expanding the applicable scenarios; the degree of automation is improved. It is highly integrated with the automated production line and the robot control system, and can realize the full process automation from point cloud data acquisition to grasping action execution, reducing manual intervention, lowering labor costs, and at the same time improving the stability and consistency of the production process.

[0014] It is to be understood that both the foregoing general description and the following detailed description are exemplary and are intended to provide further explanation of the claimed technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flowchart of the calculation method for disordered grasping of the 3D point cloud of the photovoltaic junction box according to the embodiment of the present invention; Figure 2 is a schematic diagram of the photovoltaic junction box according to the embodiment of the present invention; Figure 3 is a schematic diagram of the arc feature area of the photovoltaic junction box according to the embodiment of the present invention; Figure 4 is an overall 3D point cloud schematic diagram of the calculation method for disordered grasping of the 3D point cloud of the photovoltaic junction box according to the embodiment of the present invention; Figure 5 is a schematic diagram after separating the storage box of the calculation method for disordered grasping of the 3D point cloud of the photovoltaic junction box according to the embodiment of the present invention; Figure 6 is a schematic diagram of initially positioning the arc feature area of the photovoltaic junction box in the calculation method for disordered grasping of the 3D point cloud of the photovoltaic junction box according to the embodiment of the present invention; Figure 7Schematic diagram for expanding the 2D template matching area in the calculation method of 3D point cloud unordered grasping of a photovoltaic junction box according to an embodiment of the present invention; Figure 8 Schematic diagram of the inner arc feature points in the calculation method of 3D point cloud unordered grasping of a photovoltaic junction box according to an embodiment of the present invention; Figure 9 Schematic diagram of the outer arc feature points in the calculation method of 3D point cloud unordered grasping of a photovoltaic junction box according to an embodiment of the present invention; Figure 10 Schematic diagram of fitting a 2D contour in the calculation method of 3D point cloud unordered grasping of a photovoltaic junction box according to an embodiment of the present invention; Figure 11 Schematic diagram of the local 3D point cloud in the calculation method of 3D point cloud unordered grasping of a photovoltaic junction box according to an embodiment of the present invention; Figure 12 Schematic diagram of point cloud registration in the calculation method of 3D point cloud unordered grasping of a photovoltaic junction box according to an embodiment of the present invention; Figure 13 For Figure 12 Magnified detail view; Figure 14 Schematic diagram of selecting the best grasping point in the calculation method of 3D point cloud unordered grasping of a photovoltaic junction box according to an embodiment of the present invention; Figure 15 Schematic diagram of fitting a plane in the calculation method of 3D point cloud unordered grasping of a photovoltaic junction box according to an embodiment of the present invention. Detailed implementation manners

[0016] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, and the present invention will be further elaborated.

[0017] First, in combination with Figures 1 to 15 Describe the calculation method of 3D point cloud unordered grasping of a photovoltaic junction box according to an embodiment of the present invention, which is used in the fields of machine vision and industrial robots and has a wide range of application scenarios.

[0018] As Figures 1 to 15As shown in the figure, the calculation method for disordered grasping of the 3D point cloud of the photovoltaic junction box according to the embodiment of the present invention includes the following steps: constructing a 2D image and a 3D point cloud model of the arc feature area of the standard photovoltaic junction box to provide a reference benchmark for subsequent point cloud recognition, segmentation, and registration. In the experiment of scanning stacked products using a structured light 3D sensor, it is found that even if the point cloud is intricate, the grasping position of the junction box can be determined using the arc area feature of the photovoltaic junction box, thereby realizing point cloud recognition and segmentation as well as point cloud registration, and then calculating the pose of the photovoltaic junction box to be grasped; for this arc feature area, through optimized algorithms and parallel computing technologies, a large amount of 3D point cloud data can be processed quickly, and the position and pose of the photovoltaic junction box can be recognized within a short time, thus greatly shortening the grasping cycle of the robot. Compared with the traditional grasping method, the number of grasps per unit time can be significantly increased, and the production efficiency can be improved. Obtain the overall 3D point cloud containing the storage box and all photovoltaic junction boxes. Specifically, a structured light 3D sensor is used to obtain this overall 3D point cloud. This sensor has strong robustness to different lighting conditions, background interference, and different materials and surface textures of the junction box. Whether in an outdoor production environment with strong light or in a dim indoor warehouse, it can stably obtain and process 3D point cloud data to achieve reliable grasping, effectively expanding the applicable scenarios; according to the 2D image, extract the local 3D point cloud from the overall 3D point cloud. The local 3D point cloud is the 3D point cloud of the arc feature area of the photovoltaic junction box, reducing the amount of data processing, improving the calculation efficiency, and at the same time ensuring that the grasping system focuses on the key feature area; register the local 3D point cloud and the 3D point cloud model to obtain the registered point cloud. Through registration, the system can accurately calculate the coordinates of the grasping point and the grasping pose of the manipulator, improving the grasping accuracy, as Figures 12 to 13 shown, where the gray 3D point cloud model and the green is the local 3D point cloud; select the best grasping point coordinates from the registered point cloud; calculate the grasping pose of the manipulator by fitting a plane and calculating the rotation matrix to ensure that the manipulator can grasp the junction box in the optimal pose, reducing the grasping failure rate and improving the grasping efficiency. It is found in actual experiments that in a medium-scale photovoltaic module production line, after applying this calculation method, the grasping efficiency of the junction box can be increased by 50%.

[0019] Furthermore, as Figures 1 to 15As shown, in this embodiment, according to the 2D image, the local 3D point cloud is extracted from the overall 3D point cloud. The local 3D point cloud is the 3D point cloud of the arc feature area of the photovoltaic junction box, and the following steps are included: separating the photovoltaic junction box and the four peripheral edges of the storage frame in the overall 3D point cloud, removing the area of the four peripheral edges of the storage frame in the overall 3D point cloud, providing a reference surface for edge obstacle avoidance, reducing the interference of irrelevant point clouds, and reducing the amount of data to be processed; according to the 2D image, using 2D template matching to preliminarily locate the arc feature area of the photovoltaic junction box. The 2D template matching algorithm has a fast calculation speed, can quickly narrow the range of the target area, reduce the calculation amount of 3D point cloud processing, and then accurately locate the arc feature area. The arc feature is the key feature of the photovoltaic junction box, and accurate positioning can effectively distinguish the junction box from other objects, ensure subsequent grasping calculations, and crop the corresponding 3D point cloud.

[0020] Further, as Figures 1 to 15 shown, in this embodiment, separating the photovoltaic junction box and the four peripheral edges of the storage frame in the overall 3D point cloud includes the following steps: since the heights of the four peripheral edges of the storage frame are the same, in the overall 3D point cloud, the four peripheral edges of the storage frame can be identified by setting a height threshold; using the gray region growing algorithm to extract the four peripheral edges of the storage frame; using gray image morphology to separate the storage frame edge and the photovoltaic junction box. Through the above algorithms, the edges of the photovoltaic junction box and the storage frame can be quickly and accurately separated.

[0021] Further, as Figures 1 to 15 shown, in this embodiment, accurately positioning the arc feature area includes the following steps: expanding the 2D template matching area. Since the position and attitude of the 3D point cloud also need to be calculated, the 2D matching area needs to be expanded to a certain extent to ensure that the arc feature area is completely included; in the expanded 2D template matching area, extracting arc feature points and fitting the 2D contour, which can accurately describe the geometric shape of the arc feature area and provide a reliable reference for subsequent 3D point cloud registration.

[0022] Further, as Figures 1 to 15As shown, in this embodiment, extracting arc feature points and fitting the 2D contour includes the following steps: converting the overall 3D point cloud into a grayscale image to facilitate rapid feature extraction; in the grayscale image, generating a measurement rectangle for the edge according to the arc feature, which can limit the range of feature extraction near the arc feature area and avoid interference from irrelevant areas; in the measurement rectangle, extracting feature points according to the set parameters. In this embodiment, extracting feature points according to the set parameters includes the following steps: setting the amplitude to 10, the polarity to 'positive', and the feature point order to'max' for the measurement rectangle of the inner arc; setting the amplitude to 30, the polarity to 'negative', and the feature point order to'max' for the measurement rectangle of the outer arc; finding all the feature points of the inner and outer arcs; using the feature points to fit the 2D contour of the arc feature area.

[0023] Further, as Figures 1 to 15 shown, in this embodiment, selecting the best grasping point coordinates in the registered point cloud includes the following steps: in the registered point cloud, selecting the area where the bottom does not exist as the grasping area. The bottom area usually contacts the storage box or other objects. Excluding the bottom area can avoid collisions or interferences when the manipulator grasps. Selecting a non-bottom area (such as the top or side) is easier to achieve stable grasping. Since it contacts the support surface, the point cloud in the bottom area may be relatively dense and flat, and there may even be some missing point clouds (occluded). The non-bottom area usually has the complete geometric features of the object (such as arcs, edges, etc.), and the point cloud distribution is relatively uniform; calculating the position of the point cloud in the grasping area, that is, the point cloud coordinates, to determine the possible grasping points; when there are multiple grasping positions, comparing the Z coordinates of each grasping position, and selecting the point cloud with the largest Z value as the final grasping position. The point with the largest Z value is usually located in the top area of the object. The top area is easier to be grasped by the manipulator, and the grasping stability is higher. By selecting the point with the largest Z value, it is ensured that the manipulator grasps from above and avoids collisions with the objects below (such as the storage box or other junction boxes).

[0024] Further, as Figures 1 to 15As shown, in this embodiment, the steps of fitting a plane and calculating a rotation matrix include the following: Since the end of the gripper of the robot manipulator is always required to be perpendicular to the plane where the grabbing area of the junction box is located during grabbing, it is necessary to adjust the attitude of the grabbing area. First, a plane is fitted based on the 3D point cloud data of the arc feature area of the photovoltaic junction box; it is defined that the plane normal of the plane is consistent with the Z-axis direction of the manipulator coordinate system. Since the downward movement of the Z-axis of the manipulator is negative, the downward plane normal is defined as negative; through matrix cross multiplication, the rotation matrix from the current attitude to the target attitude of the manipulator is calculated. The rotation matrix describes the transformation relationship from the current attitude to the target attitude, providing a mathematical basis for the attitude adjustment of the manipulator; the rotation matrix is converted into Euler angles. Euler angles are a commonly used attitude representation method in the manipulator control system, which is convenient for directly controlling the movement of the manipulator to ensure that the end of the manipulator is always perpendicular to the plane. In this embodiment, the rotation matrix is converted into Euler angles, and the range of Euler angles is adjusted to be equivalent to the rotation range of the manipulator to ensure that the end of the manipulator is always perpendicular to the plane. Usually, the range of Euler angles of the manipulator is -180 degrees to 180 degrees. If the range of Euler angles exceeds 180 degrees, the exceeded part is converted to the corresponding range between -180 and 0 degrees. Through the calculation of the rotation matrix and Euler angles, the complex spatial attitude problem is transformed into an angle control that the manipulator can understand.

[0025] As described above, with reference to Figures 1 to 15 The calculation method for disordered grabbing of the 3D point cloud of the photovoltaic junction box according to the embodiment of the present invention is described above. It improves the grabbing efficiency, can quickly process a large amount of 3D point cloud data, greatly shortens the grabbing cycle of the robot. Compared with the traditional grabbing method, it can significantly increase the number of grabs per unit time and improve the production efficiency; it increases the grabbing accuracy. By using advanced point cloud feature extraction and matching algorithms, it can accurately identify key features such as the edges and corners of the photovoltaic junction box. Even in a complex disordered stacking environment, it can accurately calculate its spatial position and attitude, greatly improving the accuracy rate of the robot's grabbing and reducing the risks of product damage and production delays caused by grabbing mistakes; it can adapt to complex scenarios. Since a structured light sensor is used, this sensor has strong robustness to different lighting conditions, background interferences, and different materials and surface textures of the junction box. Whether in an outdoor production environment with strong light or in a dim indoor warehouse, it can stably acquire and process 3D point cloud data to achieve reliable grabbing, effectively expanding the applicable scenarios; it improves the degree of automation. It is highly integrated with the automated production line and the robot control system, and can realize the full process automation from point cloud data acquisition to grabbing action execution, reducing manual intervention, lowering the labor cost, and at the same time improving the stability and consistency of the production process.

[0026] It should be noted that in this specification, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0027] Although the content of the present invention has been described in detail through the above preferred embodiments, it should be recognized that the above description should not be considered as a limitation of the present invention. After those skilled in the art have read the above content, various modifications and alternatives to the present invention will be obvious. Therefore, the protection scope of the present invention should be defined by the appended claims.

Claims

1. A calculation method for disorderly capturing 3D point clouds of photovoltaic junction boxes, characterized in that: The following steps are included: Construct 2D images and 3D point cloud models of the arc-shaped feature areas of a standard photovoltaic junction box; Obtain the overall 3D point cloud including the storage frame and all photovoltaic junction boxes; Extracting a local 3D point cloud from the overall 3D point cloud according to the 2D image, wherein the local 3D point cloud is a 3D point cloud of an arc-shaped feature area of ​​a photovoltaic junction box; Registering the local 3D point cloud and the 3D point cloud model to obtain a registered point cloud; Selecting the best grabbing point coordinates in the registered point cloud; The grasping posture of the manipulator is calculated by fitting the plane and calculating the rotation matrix.

2. The calculation method for disorderly capturing 3D point cloud of photovoltaic junction box according to claim 1 is characterized in that: A structured light 3D sensor is used to obtain the overall 3D point cloud including the storage frame and all photovoltaic junction boxes.

3. The calculation method for disorderly capturing 3D point cloud of photovoltaic junction box according to claim 1 is characterized in that: According to the 2D image, extracting a local 3D point cloud from the overall 3D point cloud, wherein the local 3D point cloud is a 3D point cloud of an arc-shaped feature area of ​​a photovoltaic junction box, comprises the following steps: Separating the photovoltaic junction box and the edges around the storage frame in the overall 3D point cloud; According to the 2D image, 2D template matching is used to preliminarily locate the arc feature area of ​​the photovoltaic junction box, and then the arc feature area is accurately located, and the corresponding 3D point cloud is cropped.

4. The calculation method for disorderly capturing 3D point cloud of photovoltaic junction box according to claim 3 is characterized in that: The separation of the photovoltaic junction box and the edges around the storage frame in the overall 3D point cloud comprises the following steps: In the overall 3D point cloud, a height threshold is set to identify the edges around the storage frame; The grayscale region growing algorithm is used to extract the edges around the storage frame; Grayscale image morphology is used to separate the edge of the storage frame and the photovoltaic junction box.

5. The calculation method for disorderly capturing 3D point cloud of photovoltaic junction box according to claim 3, characterized in that: The method of accurately locating the arc-shaped feature area comprises the following steps: Expand the 2D template matching area; In the expanded 2D template matching area, arc feature points are extracted and the 2D contour is fitted.

6. The calculation method for disorderly capturing 3D point cloud of photovoltaic junction box according to claim 5, characterized in that: The method of extracting arc feature points and fitting 2D contours comprises the following steps: Converting the entire 3D point cloud into a grayscale image; In the grayscale image, the measurement rectangle of the edge is generated according to the arc features; In the measurement rectangle, feature points are extracted according to the set parameters; The 2D contour of the arc-shaped feature area is fitted using feature points.

7. The calculation method for disorderly capturing 3D point cloud of photovoltaic junction box according to claim 6, characterized in that: The step of extracting feature points according to the set parameters comprises the following steps: For the inner arc measurement rectangle, set the amplitude to 10, polarity to 'positive', and feature point order to 'max'; For the outer arc measurement rectangle, set the amplitude to 30, polarity to 'negative', and feature point order to 'max'; Find all the feature points of the inner and outer arcs.

8. The calculation method for disorderly capturing 3D point cloud of photovoltaic junction box according to claim 1, characterized in that: The step of selecting the best grab point coordinates in the registered point cloud comprises the following steps: In the registered point cloud, a bottom non-existent area is selected as a grasping area; Calculating the point cloud position within the grasping area; When there are multiple grasping positions, the Z coordinates of each grasping position are compared, and the point cloud with the largest Z value is selected as the final grasping position.

9. The calculation method for disorderly capturing 3D point cloud of photovoltaic junction box according to claim 1, characterized in that: The steps of fitting a plane and calculating a rotation matrix include: Fitting a plane according to the 3D point cloud data of the arc-shaped feature area of ​​the photovoltaic junction box; The plane normal of the plane is defined to be consistent with the Z-axis direction of the manipulator coordinate system; By matrix cross multiplication, the rotation matrix of the manipulator from the current posture to the target posture is calculated; Convert the rotation matrix to Euler angles to ensure that the end of the robot is always perpendicular to the plane.

10. The calculation method for disorderly capturing 3D point cloud of photovoltaic junction box according to claim 9, characterized in that: The rotation matrix is ​​converted into Euler angles, and the range of the Euler angles is adjusted to be equivalent to the rotation range of the manipulator, ensuring that the end of the manipulator is always perpendicular to the plane.

Images