Self-Planning Method Based on 3D Point Camera Shooting

Abstract

The present invention relates to the field of vision algorithms, specifically a self-planning method based on 3D point cameras, and its specific steps are as follows: S1, initial shooting pose; S2, shooting space; S3, observation pose; S4, input the observation pose; S5, create an octree map; S6, update the Octomap; S7, whether the shooting is completed; S8, calculate the shooting result; S9, evaluate each shooting pose; S10, sort to obtain the optimal shooting pose; The present invention optimizes the problem that in industrial robot vision tasks, the camera shooting position requires manual operation and the manual operation is cumbersome through the observation pose generation method and the generation process. To a certain extent, this method can effectively reduce the cost of manual teaching; at the same time, through the observation pose evaluation method and related formulas and calculations, this method does not have excessive requirements for specific workpieces and scenarios, has good flexibility, and can adapt to different working environments.

Description

Technical Field

[0001] The present invention relates to the field of vision algorithms, specifically a self-planning method based on 3D point camera shooting. Background Art

[0002] With the rise and maturity of vision algorithms, a large number of related algorithms have started to be implemented in the industrial field. Compared with traditional methods, vision algorithms have brought more efficient automation processes to the field of industrial robots. However, the results of vision algorithms directly depend on the imaging results captured by the camera. For example, in the visual grasping task of an industrial robot, if the camera cannot accurately capture the object, especially in the case of a large field of view scene, the entire vision algorithm will become ineffective. Currently, for such problems, it is necessary to roughly determine the shooting position through simulation or manual teaching of the robot, and repeatedly try to determine the optimal shooting effect. At the same time, due to the limitation of the shooting position, the algorithm cannot handle flexible motion scenarios. Summary of the Invention

[0003] To solve the above problems, the present invention proposes a self-planning method based on 3D point camera shooting.

[0004] The self-planning method based on 3D point camera shooting is as follows:

[0005] S1. Initial shooting pose: Set an initial shooting position that can capture the object.

[0006] S2. Shooting space: Set a shooting space range that can completely contain the object to be captured, and directly configure a rectangular space or a spherical space by the user to enclose the area p to be captured.

[0007] S3. Observation pose: Generate an observation pose according to the shooting space in step S2.

[0008] S4. Input observation pose: Uniformly sample the positions to be captured in the space and calculate the normal vector at the position:

[0009] a. Observation position Observation pose

[0010] b. Each observation point

[0011] c. Finally, calculate the shooting poses of all sampled points and save them in the variable Positions.

[0012] S5. Create an octree map: Create an octree map, i.e., Octomap, for the shooting space to record and update the state of the current shooting space.

[0013] a. The map can divide the space into cube regions of different sizes according to the scale resolution;

[0014] b. After inputting the captured point cloud, the regions in the space will be divided into several parts;

[0015] S6. Update Octomap: Update Octomap and use the captured point cloud and camera position to update the regions marked in step S4;

[0016] S7. Determine if shooting is complete: Determine whether the current shooting process is complete:

[0017] a. Is there a shooting pose? If not, end;

[0018] b. Has the Octomap not changed for N consecutive times? If not, end;

[0019] S8. Calculate the shooting result: If the shooting has not exited, use the RayCasting method to simulate taking pictures and calculate the shooting result for each shooting pose Position i :

[0020] a. Using the camera as the initial direction vector, generate different direction vectors Vectors i according to the viewing angle parameter fov and the interpolation spacing step;

[0021] b. Simulate the camera's light rays Camera through the position of the camera and the direction vector Rays = {Ray0, Ray1,... Ray i ... Ray n}, where

[0022] c. Calculate the intersection situation of each light ray in Camera Rays with the Octomap by the RayCasting algorithm;

[0023] d. Count the number N Rays of light rays in Camera free that intersect the free space, the number N unknown that intersect the unknown space, and the number N occupied that intersect the occupied space respectively;

[0024] S9. Evaluate each shooting pose: Give each shooting pose a score. The specific scoring method for each pose is:

[0025] a. Evaluate the regionality of the shooting position; set the light target ratio α for each shooting pose free , α occupied , α unkown , calculate f area ;

[0026] b. The distance between the shooting position and the current position of the robot; set the parameter ρ, representing the desired proportion of the robot's movement, 0 means no movement, 1 means moving to the farthest distance, and calculate f navigation ; where P camera represents the position where the current camera is located;

[0027] c. Calculate the score for each pose through the formula f score = f area * f navigation Calculate the score for each pose;

[0028] d. For each shooting pose, judge whether the robot can reach it by robot kinematics;

[0029] S10. Sort to obtain the optimal shooting pose and take the maximum value among all shooting poses:

[0030] a. If there is, then the shooting pose corresponding to the maximum score will be executed next, that is, start from step S5;

[0031] b. If not, then enter the condition in a of step S6 and exit the shooting.

[0032] The normal vector in step S2 is specifically the position P of each sampling point i , the normal vector normal i and the shooting distance distance from the camera to the shooting object.

[0033] In step S2, b can be divided into the area without point cloud, that is, free.

[0034] In step S2, b can be divided into the blank area, that is, unknown.

[0035] In step S2, b can be divided into the area with point cloud, that is, occupied.

[0036] In step S2, b can be divided into shooting point cloud: the mobile robot moves to the shooting position and executes shooting.

[0037] In step S9, d sets the score of the unreachable shooting pose to 0.

[0038] In step S10, this maximum value needs to be greater than the minimum threshold score min .

[0039] The beneficial effects of the present invention are as follows: By means of the pose generation method and the optimization of the generation process, the present invention solves the problem that in the vision task of industrial robots, the camera shooting position requires manual operation, and the manual operation is cumbersome. To a certain extent, this method can effectively reduce the cost of manual teaching; at the same time, through the observation pose evaluation method and related formulas and calculations, this method does not have excessive requirements for specific workpieces and scenarios, has good flexibility, and can adapt to different working environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The present invention will be further described below with reference to the drawings and embodiments.

[0041] Figure 1 It is a schematic structural diagram of the process of the present invention;

[0042] Figure 2 It is a schematic diagram of the observation position formula of the present invention;

[0043] Figure 3 It is a schematic diagram of the distance calculation formula between the shooting position and the current position of the robot in the present invention;

[0044] Figure 4 It is a schematic diagram of the generated observation posture structure of the present invention;

[0045] Figure 5 It is a schematic diagram of creating an octree map of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] In order to make the technical means, creative features, achieved purposes and functions of the present invention easy to understand, the present invention will be further described below.

[0047] As Figures 1 to 5 shown, based on the self-planning method for 3D point camera shooting, the specific steps are as follows:

[0048] S1. Initial shooting posture: Set an initial shooting position that can capture the object;

[0049] S2. Shooting space: Set a shooting space range that can completely contain the object to be shot, and the user directly configures a rectangular space or a spherical space to enclose the area p to be shot;

[0050] S3. Observation posture: Generate an observation posture according to the shooting space in step S2, as Figure 4 shown, the arrow indicates the camera observation direction;

[0051] S4. Input the observation posture: Uniformly sample the positions to be shot in the space and calculate the normal vector at the position:

[0052] a. Observation position Observation posture such as Figure 2 ;

[0053] b. For each observation point

[0054] c. Finally, calculate the shooting poses of all sampling points and save them in the variable Positions;

[0055] S5. Create an octree map: Create an octree map, i.e., Octomap, in the shooting space to record and update the state of the current shooting space, such as Figure 5 ;

[0056] a. This map can divide the space into cube regions of different sizes according to the scale resolution;

[0057] b. After inputting the shooting point cloud, the regions in this space will be divided into several parts;

[0058] S6. Update Octomap: Update Octomap and use the captured point cloud and camera position to update the regions marked in step S4;

[0059] S7. Determine if shooting is complete: Determine if the current shooting process is complete:

[0060] a. Is there a shooting pose? If not, end;

[0061] b. Has Octomap not changed for N consecutive times? If not, end;

[0062] S8. Calculate the shooting result: If the shooting has not exited, use the RayCasting method to simulate taking pictures and calculate the shooting result for each shooting pose Position i :

[0063] a. Using the camera as the initial direction vector, generate different direction vectors Vectors according to the viewing angle parameter fov and the interpolation spacing step i ;

[0064] b. Simulate the camera's light rays Camera ={Ray0,Ray1,...Ray Rays ...Ray i ...Ray n} through the position of the camera

[0065] c. By the RayCasting algorithm, calculate the intersection of each light ray in Camera Rays with the Octomap;

[0066] d. Count the number N of the rays in the Camera Rays intersecting the free space, free the number N of the rays intersecting the unknown space, unknown and the number N of the rays intersecting the Occupied space occupied ;

[0067] S9. Evaluate each shooting pose: Give each shooting pose a score. The specific scoring method for each pose is as follows:

[0068] a. Evaluate the regionality of the shooting position; Set the target ratio α of the light rays for each shooting pose free α occupied α unkown which is calculated by the formula shown in Figure 2 f; area ;

[0069] b. The distance between the shooting position and the current position of the robot; Set the parameter ρ, which represents the desired ratio of the robot's movement, 0 means no movement, and 1 means moving to the farthest distance; Calculate f according to Figure 3 ; where P navigation represents the position where the current camera is located; camera

[0070] c. Calculate the score of each pose through the formula f score = f area * f navigation ;

[0071] d. For each shooting pose, judge whether the robot can reach by robot kinematics;

[0072] S10. Sort to obtain the optimal shooting pose and take the maximum value among all shooting poses:

[0073] a. If there is one, then the shooting pose corresponding to the maximum score will be executed next, that is, start from step S5;

[0074] b. If not, then enter the condition in a of step S6 to exit the shooting.

[0075] The normal vector of step S2 described above is specifically the position P of each sampling point i , the normal vector normal i and the shooting distance distance from the camera to the shooting object.

[0076] ​Through steps S1 and S2, in view of the problem of cumbersome shooting positions for the above industrial robot in cooperation with the camera, the present invention proposes a self-planning method for shooting positions based on 3D point clouds. Under this method, the robot can determine the positions to be shot according to the settings and shooting conditions, realizing the simulation generation, evaluation, and selection of the shooting positions.

[0077] b in step S2 can be divided into areas without point clouds, that is, free.

[0078] b in step S2 can be divided into blank areas, that is, unknown.

[0079] b in step S2 can be divided into areas with point clouds, that is, occupied.

[0080] b in step S2 can be divided into shooting point clouds, moving the robot to the shooting position, and performing the shooting.

[0081] Through steps S3 and S4, the pose generation method and the generation process are observed to optimize the problem that the camera shooting position requires manual operation and the manual operation is cumbersome in the visual tasks of industrial robots. To a certain extent, this method can effectively reduce the cost of manual teaching.

[0082] In step S9, d sets the score of unreachable shooting poses to 0.

[0083] In step S10, this maximum value needs to be greater than the minimum threshold score min 。

[0084] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art of this industry should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification is only the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A self - planning method based on 3D point camera shooting, characterized in that: The specific steps are as follows: S1. Initial shooting pose: Set an initial shooting position from which the object can be shot. S2. Shooting space: Set a shooting space range that can completely contain the object to be shot. The user directly configures a rectangular space or a spherical space to enclose the area p to be shot. S3. Observation pose: Generate an observation pose according to the shooting space in step S2. S4. Input the observation pose: Uniformly sample the positions to be shot in the space and calculate the normal vectors of the positions. a. Observation position Observation posture b. Each observation point c. Finally, calculate the shooting poses of all sampled points and save them in the variable Positions. S5. Create an octree map: Create an octree map, i.e., Octomap, for the shooting space to record and update the state of the current shooting space. a. This map can divide the space into cube regions of different sizes according to the scale resolution. b. After inputting the shooting point cloud, the regions in this space will be divided into several parts. S6. Update Octomap: Update Octomap, and use the captured point cloud and the camera position to update the regions marked in step S4. S7. Determine whether the shooting is completed: Determine whether the current shooting process is completed. a. Whether there is a shooting pose. If not, end. b. Whether Octomap has not changed for N consecutive times. If not, end. S8, Calculate the shooting result: If the shooting has not exited and the RayCasting method is used to simulate taking pictures, calculate the shooting result of each shooting pose Position i of the shooting result: a. With the camera as the initial direction vector, different direction vectors Vectors are generated according to the viewing angle parameter fov and the interpolation spacing step i ; b. Through the position where the camera is located and the direction vector to simulate the light of the camera Camera Rays ={Ray0, Ray1, … Ray i … Ray n}, where c. Calculate the intersection of each ray in the Camera with the Octomap using the RayCasting algorithm Rays for each ray in the Camera d. Count the number of lights in the Camera Rays intersecting with the free space as N free , the number of intersections with the unknown space as N unknown and the number of intersections with the occupied space as N occupied ; S9. Evaluate each shooting pose: Give each shooting pose a score. The specific scoring method for each pose is as follows: a. Evaluate the regionality of the shooting position; set the light target ratio α for each shooting pose free , α occupied , α unkown , calculate f area ; b. The distance between the shooting position and the current position of the robot; Set a parameter ρ, which represents the expected proportion of the robot's movement. 0 means no movement, and 1 means moving to the farthest distance, and calculate f navigation ; where P camera represents the position where the current camera is located; c. Calculate the score for each posture through the formula f score = f area * f navigation ​ d. For each shooting pose, judge whether the robot can reach it by robot kinematics. S10. Sort to obtain the optimal shooting pose and take the maximum value among all shooting poses. a. If there is one, next, execute the shooting pose corresponding to the maximum score, that is, start from step S5. b. If not, enter the condition in a of step S6 and exit the shooting.

2. The self - planning method based on 3D point camera shooting according to claim 1, characterized in that: The normal vector of step S2 is specifically the position P of each sampling point i , the normal vector normal i and the shooting distance distance from the camera to the object being photographed.

3. The self - planning method based on 3D point camera shooting according to claim 1, characterized in that: The b in step S2 can be divided into a region without point cloud, which is free.

4. The self - planning method based on 3D point camera shooting according to claim 1, characterized in that: The b in step S2 can be divided into a blank region, which is unknown.

5. The self - planning method based on 3D point camera shooting according to claim 1, characterized in that: The b in step S2 can be divided into a region with point cloud, which is occupied.

6. The self - planning method based on 3D point camera shooting according to claim 1, wherein: The b in step S2 can be divided into shooting point cloud, the mobile robot moves to the shooting position and executes the shooting.

7. The self - planning method based on 3D point camera shooting according to claim 1, wherein: In d of step S9, set the score of the unreachable shooting pose to 0.

8. The self - planning method based on 3D point camera shooting according to claim 1, wherein: In the said step S10, the maximum value needs to be greater than the minimum threshold score min .

Images