A three-dimensional automatic scanning device and scanning method for a robotic arm

Abstract

The present application discloses a three-dimensional automatic scanning device and scanning method for a robotic arm, which are used for automatically three-dimensionally scanning an object to be scanned. The device includes a computer and a robotic arm. The computer is controllably connected to the robotic arm. A three-dimensional scanner and a depth camera are provided at the moving end of the robotic arm. The three-dimensional scanner and the depth camera are communicatively connected to the computer. By combining with the depth camera for path analysis, a completely automatic trajectory path generation is realized, ensuring the high-quality reliability of the scanning trajectory. For any object with a design model, the present application can achieve automatic path planning and repeatedly and circularly scan components of the same model. By adding a degree of freedom to the robotic arm through a moving driving member, the scanning range is expanded, scanning blind spots are avoided, and the limitation on the size of the object to be scanned is reduced.

Description

Technical Field

[0001] The present application relates to a three-dimensional automatic scanning device and a scanning method for a robotic arm, belonging to the technical field of three-dimensional scanning. Background Art

[0002] Three-dimensional scanning technology is a non-contact measurement technology used to obtain and analyze the shape and contour of physical objects. Using three-dimensional scanning technology, three-dimensional reconstruction of the object to be scanned can be carried out, thereby creating a three-dimensional model of the actual object. The reconstructed three-dimensional model can be widely applied in the fields of industrial design, reverse engineering, medical materials, bioinformatics, digital cultural relics collection, game creation, etc.

[0003] Robotic arm control is a very mature automatic control technology at present, and there are mature application cases in industrial fields such as automatic grinding and material handling. In the field of automated modeling, there are few applications using robotic arms to complete autonomous modeling. With the development of robotic arm technology and scientific and technological progress, robotic arm control technology is more mature, and at the same time, the demand for automated modeling is becoming stronger. Using a robotic arm in cooperation with a three-dimensional scanner to achieve automated modeling will improve the speed of automated modeling. At the same time, the modeling quality is related to the quality of the robotic arm movement trajectory and the positioning accuracy. Mature robotic arm products can fully meet the requirements of the trajectory and accuracy, making high-efficiency automated modeling possible. Using a robotic arm to hold a three-dimensional scanner to work automatically has become the mainstream to improve the scanning speed and modeling accuracy of three-dimensional scanning technology.

[0004] At present, according to the existing patent documents, CN202010414281.0 discloses a three-dimensional scanning device and a working method for a robot, which expounds the function of a robotic arm holding a three-dimensional scanning and measuring device for on-line measurement; CN108332660B discloses a three-dimensional scanning system and a scanning method for a robot, which can automatically perform scanning without manual intervention; CN108340405B discloses a three-dimensional scanning system and method for a robot, providing a robot system and an automatic scanning method for three-dimensional scanning of an object without any manual intervention, capable of automatic scanning and mainly based on three-dimensional scanning; the CN112659117A patent discloses a three-dimensional scanning method based on a three-dimensional scanner, a robot and a turntable; the CN114248086B patent discloses a flexible three-dimensional vision-guided robot alignment system and method. In the above-mentioned documents, the combined use of a robotic arm and a three-dimensional scanner is described. However, the problem of automatically planning the scanning path has not been solved, and it is necessary to manually plan the scanning path and intervene in the measurement results. Summary of the Invention

[0005] In view of the defects or deficiencies in the above-mentioned background art, the purpose of the present invention is to provide a three-dimensional automatic scanning device and scanning method for a robotic arm, which can automatically perform path planning and execute scanning work. This method overcomes the constraints of traditional methods on the type and size of the scanning object, and at the same time realizes the automatic execution of scanning work, improving the scanning efficiency.

[0006] To solve the above technical problems, the technical solution of the present application is to provide a three-dimensional automatic scanning device for a robotic arm, which is used for automatically three-dimensional scanning of an object to be scanned, including a computer and a robotic arm. The computer is control-connected to the robotic arm. A three-dimensional scanner and a depth camera are provided at the moving end of the robotic arm. The three-dimensional scanner and the depth camera are communicatively connected to the computer;

[0007] The depth camera obtains the spatial three-dimensional coordinates of the photographed surface of the object to be scanned, and converts them to obtain the local point cloud data of the photographed surface of the object to be scanned;

[0008] The local point cloud data is fitted with the design model of the object to be scanned, and the rough position data of the object to be scanned in the world coordinate system is calculated;

[0009] According to the rough position data of the object to be scanned in the world coordinate system, the boundary points at both ends of the three direction coordinate axes are calculated, and a cuboid envelope surface that includes the object to be scanned is generated using these boundary points;

[0010] Discrete coordinate points at a specific safety distance d around the cuboid envelope surface are generated, and the order of arrival of these discrete points is marked to generate the movement path of the robotic arm;

[0011] The robotic arm drives the three-dimensional scanner to perform a three-dimensional scanning action according to the generated movement path above, and carries the three-dimensional scanner to perform a complete scan of the object to be scanned.

[0012] Preferably, it further includes a moving driving member for driving the relative movement of the robotic arm and the object to be scanned; the moving driving member is set as a surrounding driving slide rail. The computer is control-connected to the surrounding driving slide rail. The robotic arm is arranged on the surrounding driving slide rail, and the object to be scanned is arranged inside the surrounding driving slide rail. The surrounding driving slide rail drives the robotic arm to rotate around the object to be scanned. Further, the surrounding driving slide rail is set as a U-shaped slide rail, and the object to be scanned is arranged on a placement table, and the placement table is located at the center of the U-shaped slide rail.

[0013] Preferably, it further includes a moving driving member for driving the relative movement of the robotic arm and the object to be scanned; the moving driving member is set as an AGV cart. The computer is control-connected to the AGV cart. The robotic arm is arranged on the AGV cart, and the AGV cart drives the robotic arm to move relative to the object to be scanned.

[0014] The present application also provides a three-dimensional automatic scanning method for a robotic arm, including the following steps:

[0015] Step 1: Use a depth camera to take pictures of the object to be scanned, obtain the spatial three-dimensional coordinates of the photographed surface of the object to be scanned, and convert them to obtain the local point cloud data of the photographed surface of the object to be scanned;

[0016] Step 2: Preprocess the local point cloud data obtained in Step 1 to remove noise, and then fit the processed local point cloud data with the design model of the object to be scanned to calculate the rough position of the object to be scanned in the world coordinate system;

[0017] Step 3: Calculate the boundary points at both ends of the three direction coordinate axes of the position of the object to be scanned obtained in Step 2 in the world coordinate system, and use these boundary points to generate a cuboid envelope surface that includes the object to be scanned;

[0018] Step 4: Generate discrete coordinate points at a specific safety distance d around the cuboid envelope surface, mark the order of arrival of these discrete points, and generate the movement path of the robotic arm;

[0019] Step 5: The robotic arm drives the 3D scanner to perform a 3D scanning action according to the movement path generated in Step 4, and carries the 3D scanner to perform a complete scan of the object to be scanned.

[0020] The advantages of this application are as follows. Compared with the traditional handheld scanning method, it reduces a large amount of manual operations. By combining a depth camera for path analysis, it realizes the generation of a completely automatic trajectory path, ensuring the high-quality reliability of the scanning trajectory. For any object with a design model, this application can achieve automatic path planning and repeatedly scan the same type of components in a loop. By adding a degree of freedom to the robotic arm through a mobile drive, it expands the scanning range, avoids scanning blind spots, and reduces the size limitations of the object to be scanned. Brief Description of the Drawings

[0021] Figure 1 It is a schematic diagram of a three-dimensional automatic scanning system including a surrounding drive slide rail;

[0022] Figure 2 It is a flowchart of the three-dimensional automatic scanning method of the robotic arm provided by this application;

[0023] Figure 3 It is a schematic diagram of a three-dimensional automatic scanning system including an AGV cart. Detailed Description of the Preferred Embodiment

[0024] To make this application more obvious and understandable, the preferred embodiments are described in detail below in conjunction with the accompanying drawings.

[0025] Embodiment 1

[0026] This embodiment provides a three-dimensional automatic scanning system based on a robotic arm. Refer to Figure 1 andFigure 3 for three-dimensional scanning of the object 7 to be scanned. The object 7 to be scanned can be a symmetric object, an asymmetric object with an uneven surface, or an object group formed by stacking multiple sub-objects.

[0027] Specifically, the three-dimensional automatic scanning system includes:

[0028] A computer 1, which is used to perform calculations and control the entire system to perform scanning. As known to those skilled in the art, the computer 1 can be a personal computer, an industrial control computer with sufficient computing power, or a workstation.

[0029] An industrial control computer 2, which is connected to each instrument and can be used for data acquisition, preprocessing of data, and implementation of simple calculation tasks. Of course, the industrial control computer 2 can also be omitted, and each instrument can be directly connected to the computer 1.

[0030] A robotic arm 3, which performs three-dimensional scanning actions. A three-dimensional scanner 6 and a depth camera 8 are arranged at the moving end of the robotic arm 3. Both the three-dimensional scanner 6 and the depth camera 8 are commercially available instruments. The three-dimensional scanner 6 is used to obtain the point cloud of the geometric surface of the object 7 to be scanned, and the depth camera 8 is used to obtain the three-dimensional spatial coordinates of each point in the image; both the three-dimensional scanner 6 and the depth camera 8 are communicatively connected to the computer 1.

[0031] A placement table 5, which is used to place the object 7 to be scanned.

[0032] A tracker 9, as a component of a spherical three-dimensional scanner. Existing spherical three-dimensional scanners generally have a laser scanning head and a tracker. When performing scanning work, the tracker tracks the position of the scanning head, and the scanning head obtains the data of the object to be scanned. Through coordinate transformation, the position of the object in the world coordinates is obtained.

[0033] Using the above three-dimensional automatic scanning system, this embodiment also provides a method for realizing three-dimensional automatic scanning to obtain the point cloud data of the object surface. See Figure 2 , and the specific steps are as follows:

[0034] Step 1, use the depth camera 8 to take pictures of the object 7 to be scanned, obtain the three-dimensional spatial coordinates of the photographed surface of the object 7 to be scanned, and convert them to obtain the local point cloud data of the photographed surface of the object 7 to be scanned.

[0035] Step 2: Preprocess the local point cloud data obtained in Step 1 to remove noise, and then fit the processed local point cloud data with the design model of the object 7 to be scanned, and calculate the position of the object 7 to be scanned in the world coordinate system. Since there is a camera accuracy error in the depth camera 8, and when fitting the local point cloud data with the design model of the object 7 to be scanned, the physical object may have been deformed relative to the design model, which results in a fitting accuracy error. Therefore, the position of the object 7 to be scanned in the world coordinate system calculated in Step 2 is a rough position.

[0036] Step 3: Calculate the boundary points at both ends of the three-direction coordinate axes of the position of the object 7 to be scanned obtained in Step 2, and use these boundary points to generate a cuboid envelope surface that encloses the object 7 to be scanned.

[0037] Step 4: Generate discrete coordinate points at a specific safety distance d around the cuboid envelope surface, and mark the order of arrival of these discrete points to generate the motion path of the robotic arm 3. Currently, the software supporting commercially available robotic arms can generate the motion path according to the order of arrival of the discrete points. Therefore, only the position information of the discrete points to be reached needs to be input to generate the motion path of the robotic arm.

[0038] Step 5: The robotic arm 3 drives the 3D scanner 6 to perform a 3D scanning action according to the motion path generated in Step 4, and carries the 3D scanner 6 to perform a complete scan of the object 7 to be scanned.

[0039] The scanned point cloud data is transmitted to the computer 1 for automatic saving. When implementing the above method; the robotic arm 3D scanning software is running on the computer 1, the computer 1 sends a signal to the industrial control computer 2, and the industrial control computer 2 controls the depth camera 8 to take pictures and collect data, and the calculation process is executed by the computer 1 or the industrial control computer 2.

[0040] In a preferred embodiment, the robotic arm 3 is a six-axis robotic arm, and the 3D scanner 6 is a spherical 3D laser scanner.

[0041] Example 2

[0042] See Figure 1, since the movement range of the robotic arm 3 is limited, when the size of the object 7 to be scanned is large or the outer contour is not standard, the robotic arm 3 may not be able to complete all the scanning actions. Therefore, in this embodiment, on the basis of Embodiment 1, a surrounding drive slide rail 4 is provided. The surrounding drive slide rail 4 is arranged around the placement table 5. The robotic arm 3 is installed on the surrounding drive slide rail 4. When the robotic arm 3 performs the scanning action, it is necessary to keep the 3D scanner 6 and / or the depth camera 8 facing the object 7 to be scanned. The surrounding drive slide rail 4 can drive the robotic arm 3 to move around the object 7 to be scanned on the placement table 5, so as to more conveniently perform all the scanning actions. In a preferred embodiment, the surrounding drive slide rail 4 is a U-shaped slide rail, and the placement table 5 is located at the center of the U-shaped slide rail.

[0043] Specifically, when performing the method of three-dimensional automatic scanning to obtain the point cloud data of the object surface, the robotic arm 3 is arranged on the surrounding drive slide rail 4, the object 7 to be scanned is placed on the placement table 5, and the placement table 5 is located at the center of the surrounding drive slide rail 4; the surrounding drive slide rail 4 assists the robotic arm 3 to move around the object 7 to be scanned, so as to better perform the three-dimensional scanning action.

[0044] Embodiment 3

[0045] See Figure 3 , in this embodiment, on the basis of Embodiment 1, an AGV cart 11 is provided. The robotic arm 3 is arranged on the AGV cart 11, and the AGV cart 11 can move around the placement table 5; when the robotic arm 3 performs the scanning action, it is necessary to keep the 3D scanner 6 and / or the depth camera 8 facing the object 7 to be scanned. The AGV cart 11 can drive the robotic arm 3 to move around the object 7 to be scanned on the placement table 5, so as to more conveniently perform all the scanning actions.

[0046] Specifically, when performing the method of three-dimensional automatic scanning to obtain the point cloud data of the object surface, the robotic arm 3 is arranged on the AGV cart 11, and the object 7 to be scanned is placed on the placement table 5; the AGV cart 11 assists the robotic arm 3 to move around the object 7 to be scanned, so as to better perform the three-dimensional scanning action.

Claims

1. A three-dimensional automatic scanning device for a robotic arm, characterized in that, For automatically performing three-dimensional scanning on an object (7) to be scanned, including a computer (1) and a robotic arm (3), the computer (1) is controllably connected to the robotic arm (3), a three-dimensional scanner (6) and a depth camera (8) are provided at the moving end of the robotic arm (3), and the three-dimensional scanner (6) and the depth camera (8) are communicatively connected to the computer (1); The depth camera (8) acquires the spatial three-dimensional coordinates of the photographed surface of the object (7) to be scanned, and converts them to obtain the local point cloud data of the photographed surface of the object (7) to be scanned; The local point cloud data is fitted with the design model of the object (7) to be scanned, and the rough position of the object (7) to be scanned in the world coordinate system is calculated; According to the position of the object (7) to be scanned in the world coordinate system, the boundary points at both ends of the three-direction coordinate axes are calculated, and a cuboid envelope surface that encloses the object (7) to be scanned is generated using these boundary points; Discrete coordinate points at a specific safety distance d from the cuboid envelope surface are generated, and the sequential arrival order of these discrete points is marked to generate the movement path of the robotic arm (3); The robotic arm (3) drives the three-dimensional scanner (6) to perform a three-dimensional scanning action according to the movement path, and carries the three-dimensional scanner (6) to perform a complete scan of the object (7) to be scanned.

2. The three-dimensional automatic scanning device for a robotic arm according to claim 1, characterized in that, It further includes a moving drive member for driving the relative movement of the robotic arm (3) and the object (7) to be scanned; the moving drive member is provided as a surrounding drive slide rail (4), the computer (1) is controllably connected to the surrounding drive slide rail (4), the robotic arm (3) is provided on the surrounding drive slide rail (4), the object (7) to be scanned is provided inside the surrounding drive slide rail (4), and the surrounding drive slide rail (4) drives the robotic arm (3) to rotate around the object (7) to be scanned.

3. A three-dimensional automatic scanning device for a robotic arm according to claim 2, characterized in that, The surrounding drive slide rail (4) is provided as a U-shaped slide rail, the object (7) to be scanned is provided on a placement table (5), and the placement table (5) is located at the center of the U-shaped slide rail.

4. A three-dimensional automatic scanning device for a robotic arm according to claim 1, characterized in that, It further includes a moving drive member for driving the relative movement of the robotic arm (3) and the object (7) to be scanned; the moving drive member is provided as an AGV cart (11), the computer (1) is controllably connected to the AGV cart (11), the robotic arm (3) is provided on the AGV cart (11), and the AGV cart (11) drives the robotic arm (3) to move relative to the object (7) to be scanned.

5. A three-dimensional automatic scanning method for a robotic arm, characterized in that, It includes the following steps: Step 1: Use the depth camera to take a photo of the object to be scanned, acquire the spatial three-dimensional coordinates of the photographed surface of the object to be scanned, and convert them to obtain the local point cloud data of the photographed surface of the object to be scanned; Step 2: Preprocess the local point cloud data obtained in Step 1 to remove noise, and then fit the processed local point cloud data with the design model of the object to be scanned to calculate the rough position of the object to be scanned in the world coordinate system; Step 3: Calculate the boundary points at both ends of the three-direction coordinate axes of the position of the object to be scanned obtained in Step 2 in the world coordinate system, and generate a cuboid envelope surface that encloses the object to be scanned using these boundary points; Step 4: Generate discrete coordinate points at a specific safety distance d from the cuboid envelope surface, and mark the sequential arrival order of these discrete points to generate the movement path of the robotic arm; Step 5: The robotic arm drives the 3D scanner to perform a 3D scanning action according to the motion path generated in Step 4, and carries the 3D scanner to perform a complete scan of the object to be scanned.

Images