Self-adaptive multi-view photographing point planning algorithm and system for large casting grinding

Abstract

The invention belongs to the technical field of polishing track planning, and particularly discloses a self-adaptive multi-view photographing point planning algorithm and system for large casting polishing, and the algorithm comprises the following steps: S1, collecting a casting image, processing the casting image, and recognizing a to-be-polished region containing a marker in the casting image; s2, on the basis of the marker recognized in the step S1, all axis directions of a photographing coordinate system under a camera coordinate system are set, and the posture controllability of the photographing mechanical arm reaching the camera photographing point is improved; s3, determining the position of the origin of coordinates of the photographing coordinate system in the camera coordinate system according to the direction of each axis of the photographing coordinate system in the step S2; s4, based on the steps S2 and S3, calculating to obtain a camera photographing point pose; and S5, optimizing the pose of the camera photographing point based on a particle swarm optimization algorithm, and obtaining the pose of the optimized reachable camera photographing point.

Description

Technical Field

[0001] The invention belongs to the technical field of grinding trajectory planning, and in particular relates to an adaptive multi-viewpoint photo point planning algorithm and system for large casting grinding. Background Art

[0002] Existing automated polishing robots without perception capabilities lack precise force control and error adaptability, and their versatility and applicability are often poor. In order to improve the level of autonomous intelligent polishing of robots, it is crucial to introduce perception technology into the robot system.

[0003] Force control sensing technology can effectively deal with the problem of grinding error compensation, but it cannot solve the problem of poor autonomous adaptability of grinding robots caused by teaching or offline programming of grinding trajectories; the development of vision technology has enabled robots to be more capable of perceiving the environment, and use vision technology to perceive the positions on the workpiece that need to be polished. There is no need for teaching or offline programming of grinding position positioning guidance, and the combination of vision and force control can further enhance the autonomous intelligent capabilities of grinding equipment.

[0004] Wang Rong used visual inspection technology to extract the position information of the contour to be processed on the workpiece, and assisted in guiding the robotic arm to perceive the grinding position. However, the inspection process and the grinding trajectory planning process were carried out independently; Liang Haigang proposed a global model and local feature registration positioning method based on the generalized closest point iteration algorithm to locate the grinding area position. However, this method failed when there was a deviation in the casting model; Tian proposed a key technical method to identify rust as the rust removal process. Rust detection was completed through visual perception. This method fixed the camera outside the robot, and the limited field of view made this solution unsuitable for large steel castings; Joshi et al. used machine vision methods to acquire and identify the surface image texture features of the surface to be polished, and further calculated the polishing roughness based on the regression model, but did not perform autonomous polishing trajectory planning.

[0005] With the continuous progress of machine vision, the verification process of active grinding trajectory planning solutions for robots with visual perception capabilities has been promoted; the existing active visual recognition of welds can assist in solving the trajectory planning problem of weld grinding scenarios; and in order to use robots for grinding operations, domestic and foreign workers have designed a variety of vision-based grinding path planning solutions; for example: Ren used a Keyence laser scanner to obtain workpiece point cloud data and proposed a grinding trajectory planning method in the point cloud data; Feng et al. proposed a curved weld grinding trajectory planning method for the problem of curved surfaces that are difficult to handle; Xie et al. obtained mirror surfaces. An improved Prim grinding path planning algorithm was proposed based on surface shape information; Ding Xiaotian proposed a method for autonomous trajectory planning of large and complex component grinding robots based on three-dimensional point clouds, and developed autonomous trajectory planning software in combination with the OpenCascade geometry engine; Wang Xiangfei designed a point cloud post-processing algorithm for scanned parts to automatically identify the welds to be polished and generate grinding trajectories; Zhang Jiangtao performed point cloud registration based on the point cloud model and the standard workpiece model, and reconstructed the complex table in combination with greedy triangulation projection, and extracted grinding path points based on non-uniform rational B-spline curves; Tai Chengyue used visual guidance to plan the pipeline weld grinding trajectory.

[0006] Existing visual perception grinding technology is mostly used in grinding position positioning. However, when grinding large castings, how to quickly perceive the range of the area to be polished and obtain complete information of the area to be polished is a difficult problem that needs to be solved urgently. Therefore, the present invention proposes an adaptive multi-viewpoint photo point planning algorithm for grinding large castings. Summary of the invention

[0007] In order to solve the above technical problems, the present invention provides an adaptive multi-viewpoint photo point planning algorithm and system for large casting grinding.

[0008] Based on the above purpose, the present invention is achieved through the following technical solutions:

[0009] The present invention provides an adaptive multi-viewpoint photo point planning algorithm for large casting grinding, comprising the following steps:

[0010] S1. Collecting a casting image, processing the casting image, and identifying a region to be polished containing a marker in the casting image.

[0011] S2. Based on the marker identified in step S1, the directions of the axes of the photographing coordinate system in the camera coordinate system are set to improve the controllability of the posture of the photographing robot arm when it reaches the camera photographing point.

[0012] S3. According to the directions of the axes of the photographing coordinate system in step S2, the position of the origin of the photographing coordinate system in the camera coordinate system is determined.

[0013] S4. Based on steps S2 and S3, the camera shooting point position is calculated.

[0014] S5. Optimize the camera shooting point pose based on the particle swarm optimization algorithm to obtain the optimized reachable camera shooting point pose.

[0015] According to the above-mentioned adaptive multi-viewpoint photo point planning algorithm for large casting grinding, preferably, in step S1, the specific steps of collecting the casting image are: using the 2D image acquisition function of the structured light camera to capture the casting image of the area to be polished containing the worker markers, and then using the existing deep learning-based method to train and deploy the marker detection model to identify the markers in the casting image, and correspond the identified marker pixel positions to the 3D space.

[0016] According to the above-mentioned adaptive multi-viewpoint photo point planning algorithm for large casting grinding, preferably, in step S1, the marker detection model uses the YOLOv5 network to identify the markers in the casting image, in preparation for guiding the camera mounted on the photographing robot arm to collect point cloud data; after sensing the grinding area, it is necessary to further adaptively plan the camera photo points from multiple angles to completely collect the three-dimensional detail data of the steel casting surface; the markers include four white "×"-shaped marks made by the quality inspection workers in the area to be ground of the casting, and the workers perform grinding in the area formed by the marks.

[0017] According to the above-mentioned adaptive multi-viewpoint photo point planning algorithm for large casting grinding, preferably, in step S1, during the camera shooting process, the camera should be approximately perpendicular to the workpiece surface; and during the rotation of the photo robot arm, the camera should be driven to shoot at the minimum rotation angle of its end to prevent the photo robot arm and the camera from colliding, and to collect data with the camera's maximum effective field of view.

[0018] According to the above-mentioned adaptive multi-viewpoint photo point planning algorithm for large casting grinding, preferably, in step S2, the step of setting the directions of each axis of the photo coordinate system under the camera coordinate system includes: S201, setting the Z-axis direction of the photo coordinate system; S202, setting the X-axis direction of the photo coordinate system; S203, setting the Y-axis direction of the photo coordinate system.

[0019] In step S201, the Z-axis direction is the key to ensure that the camera shoots the workpiece vertically. For the three-dimensional space coordinates of the four marks, any three points of the four marks are selected in combination for plane fitting. After the direction vectors of the plane formed by the three points are obtained, the average vector is taken as the Z-axis direction of the photographing coordinate system; after the photographing robot arm mounts the camera to the set photographing point, the Z-axis direction of the coordinate system of the end of the photographing robot arm, the Z-axis direction of the camera coordinate system, and the Z-axis direction of the photographing coordinate system must be ensured to be in the same direction, and the positive and negative directions of the Z-axis of the photographing coordinate system relative to the camera coordinate system are corrected accordingly to ensure that the camera is facing correctly.

[0020] According to the above-mentioned adaptive multi-viewpoint photo point planning algorithm for large casting grinding, preferably, in step S202, the X-axis direction of the photo coordinate system is consistent with the long side direction of the rectangular area formed by the four marks, and ensure that the X-axis direction of the photo coordinate system is in an acute angle relationship with the X-axis direction of the end of the photo robotic arm; the X-axis direction of the end of the photo robotic arm on which the camera is mounted is installed in an opposite manner to the X-axis of the camera coordinate system. Therefore, the X-axis direction of the photo coordinate system in the camera coordinate system at this time needs to be in an obtuse angle relationship with the X-axis of the current camera coordinate system to ensure that the camera reaches the photo point with the minimum rotation angle.

[0021] According to the right-hand coordinate system principle, the Y-axis direction of the camera coordinate system is set to

[0022] in, Represents the vector of the Z axis of the camera coordinate system under the camera; Represents the vector of the X-axis of the camera coordinate system under the camera.

[0023] At this point, the rotation matrix R formed by the direction of the photographic coordinate system is obtained c for:

[0024]

[0025] Among them, SO(n) represents the special orthogonal group, and when n=3, it represents the rotation in three-dimensional space.

[0026] According to the above-mentioned adaptive multi-viewpoint photo point planning algorithm for large casting grinding, preferably, in step S3, the step of determining the position of the origin of the photo coordinate system in the camera coordinate system is as follows: in the grinding area detection, the pixel positions of the four marks and the central pixel position composed of the four marks are identified, and the five pixel positions are mapped to the 3D space, which is expressed as S i ;exist Lifting h above the casting surface c The height is set to the position of the camera's photo point.

[0027]

[0028] in, Indicates the locations of the five camera photo points.

[0029] Through the five camera shooting points planned for adaptive shooting height and angle, detailed data of the polishing area can be obtained from multiple perspectives.

[0030] According to the above-mentioned adaptive multi-viewpoint photo point planning algorithm for large casting grinding, preferably, in step S4, multiple photo coordinate systems under the camera coordinate system are defined, and the expressions are:

[0031]

[0032] in, is a special Euclidean group, the R matrix is ​​the rotation matrix, and the T matrix is ​​called the transformation matrix. is the translation matrix,

[0033] According to the current end position of the camera robot arm, the camera coordinate system is transformed to the camera robot arm coordinate system, and its expression is:

[0034]

[0035] Among them, Pose i Indicates the end position in the coordinate system of the camera robot arm; R x , R y , R z Represents the rotation matrix around the X, Y, and Z axes respectively; T cal Represents the "eye-on-hand" hand-eye calibration matrix between the structured light camera and the flange at the end of the slave robot arm.

[0036] According to the Euler angle rotation mode of the camera robot, formulas (3) and (4) are converted into the six-degree-of-freedom representation method of Euler angle. According to the sequence of the camera robot, the sequence is the order of the three rotations of Euler angle, and the sequence is converted into Euler angle form. When the sequence of the camera robot is ZYX, that is, it rotates around the Z axis first, then around the Y axis, and finally around the X axis, the transformation matrix is ​​converted into the six-degree-of-freedom Euler angle form, and its expression is:

[0037]

[0038] Among them, P pose (i) indicates the camera photo point; P pose (i)∈R 6×1 It is the six-degree-of-freedom pose sent to the camera robot arm, which means the pose of the camera point set; (X i ,Y i ,Z i )T Indicates the end position of the camera robot arm; (α i ,β i ,γ i ) T Represents the Euler angle of rotation around the X, Y, and Z axes; the function atan2(x,y) represents the inverse tangent.

[0039] According to the above-mentioned adaptive multi-viewpoint photo point planning algorithm for large casting grinding, preferably, in step S5, due to the limitation of the grinding mechanism, the photo robot arm is a collaborative robot arm with a small arm span, which leads to many cases where the camera photo point pose calculated in step S4 is unreachable. Therefore, the origin of the photo coordinate system calculated in step S4 is adaptively corrected by the particle swarm optimization algorithm (PSO), and combined with the inverse solution state code of the photo robot arm, it is determined whether the currently given photo coordinate system pose is reachable.

[0040] The particle swarm optimization algorithm is designed based on the predation behavior of bird flocks. This algorithm is a type of swarm intelligence algorithm. It is known that there is a piece of food in a certain area. During the search process, each bird in the entire flock of birds determines whether it has found the food by sharing its own location information and the food information shared by the flock. Based on this implementation, the entire flock of birds will eventually gather near the food, that is, the optimal solution to the target is found. The particle swarm optimization algorithm is widely used in target optimization due to its advantages of high precision, fast convergence and easy implementation.

[0041] The basic principle of particle swarm algorithm is to simplify the simulation of birds in a flock by using a massless particle. The particle has two properties, namely speed and position. The direction and distance of the particle in the next iteration are represented by speed, while the position represents the current position of the particle.

[0042] Press P to take a photo with the camera pose (i) Set the food as the target value of the optimization problem, and use the particle swarm optimization algorithm to find a particle set near the target value, which represents the potential camera shooting point; Set it as the position of the potential photo point, and determine whether the potential photo point is reachable based on the inverse state code of the photo robot arm, filter out the reachable postures of the photo robot arm, and select a particle closest to the target value as the optimized photo point.

[0043] Among the unreachable photo points, the particle swarm algorithm is used to continuously search for reachable points, so that the photo robot arm can adapt to the position of the photo point, thereby improving the reachability of the photo robot arm to reach the target position.

[0044] According to the adaptive planning, the camera robot adaptively captures the point cloud information of the casting from multiple angles to completely cover the area to be polished.

[0045] The invention discloses an adaptive multi-viewpoint photo point planning system for large casting polishing, comprising a photo module, a communication module and a main control module; the photo module is a photo robot arm equipped with a structured light camera, which has a wireless communication function and is used to photograph casting images and plan camera photo point positions; the communication module has a wireless communication function and is used to obtain casting images and the photo robot arm positions collected by the camera; the main control module is a hardware device equipped with a microprocessor, which has a wireless communication function and is used to process casting images and output optimized camera photo point positions; the main control module collects casting images through the structured light cameras of the communication module and the photo module, uses a YOLOv5 network to identify markers in the casting images, sets the axis direction of a photo coordinate system under a camera coordinate system according to the markers, determines the position of the origin of the photo coordinate system under the camera coordinate system, obtains the optimized reachable camera photo point positions, uses the photo robot arm to plan the camera photo point positions, guides the camera to collect point cloud data, realizes the posture of multi-angle shooting of casting point cloud information, completely covers the area to be polished, and executes the above-mentioned adaptive multi-viewpoint photo point planning algorithm for large casting polishing.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The present invention collects casting images through a structured light camera, uses the YOLOv5 network to identify markers in the casting image and the area to be polished formed by the markers, and reconstructs the information of the area to be polished to completely cover the area to be polished. This can solve the problem of insufficient camera field of view coverage and meet the needs of perceiving a larger area to be polished for large castings and obtaining complete data, thereby achieving the purpose of quickly perceiving the range of the area to be polished and obtaining complete information of the area to be polished. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the marked grinding area;

[0049] Figure 2 This is a visualization diagram of the camera’s photo point posture and direction, where the red, green, and blue arrows represent the photo coordinate system.

[0050] Figure 3 This is a visualization simulation diagram of the camera’s photo point position;

[0051] Figure 4 This is a schematic diagram of the combination of particle swarm and camera points. The pink dots represent camera camera points, and the red dots represent potential camera camera points.

[0052] Figure 5 It is a visualization simulation diagram of the optimized reachable camera photo point pose. DETAILED DESCRIPTION

[0053] The present invention is further described in detail below through specific examples, but the scope of the present invention is not limited thereto.

[0054] Example 1

[0055] An adaptive multi-view camera point planning algorithm for large casting grinding. Figure 1 As shown, the following steps are included;

[0056] S1. Collecting a casting image, processing the casting image, and identifying a region to be polished containing a marker in the casting image.

[0057] The specific steps for collecting casting images are as follows: use the 2D image acquisition function of the structured light camera to capture the image of the casting area to be polished containing the worker's markers, and then use the existing deep learning-based method to train and deploy the marker detection model to identify the markers in the casting image and correspond the identified marker pixel positions to the 3D space.

[0058] The marker detection model uses the YOLOv5 network to identify markers in the casting image, preparing to guide the camera mounted on the camera robot arm to collect point cloud data; after sensing the polishing area, it is necessary to further adaptively plan the camera shooting points from multiple angles to fully collect the three-dimensional detail data of the steel casting surface; the markers include four white "×" marks made by the quality inspection worker in the area to be polished, such as Figure 1 As shown, the worker is grinding in the area where the mark is formed.

[0059] During the camera shooting process, the camera should be approximately perpendicular to the workpiece surface; and during the rotation of the camera robot arm, the camera should be driven to shoot at the minimum rotation angle at its end to prevent the camera robot arm and the camera from colliding, and to collect data with the camera's maximum effective field of view.

[0060] S2. Based on the marker identified in step S1, the directions of the axes of the photographing coordinate system in the camera coordinate system are set to improve the controllability of the posture of the photographing robot arm when it reaches the camera photographing point.

[0061] The step of setting the directions of the axes of the photographing coordinate system under the camera coordinate system includes: S201, setting the Z-axis direction of the photographing coordinate system.

[0062] The Z-axis direction is the key to ensure that the camera shoots the workpiece vertically. For the three-dimensional space coordinates of the four marks, any three points of the four marks are selected in combination for plane fitting. After obtaining the direction vectors of the plane formed by the three points, the average vector is taken as the Z-axis direction of the photographing coordinate system. After the photographing robot arm mounts the camera to the set photographing point, the Z-axis direction of the coordinate system of the end of the photographing robot arm, the Z-axis direction of the camera coordinate system, and the Z-axis direction of the photographing coordinate system must be guaranteed to be in the same direction. Based on this, the positive and negative directions of the Z-axis of the photographing coordinate system relative to the camera coordinate system are corrected to ensure the correct orientation of the camera.

[0063] S202, setting the X-axis direction of the photographing coordinate system; the X-axis direction of the photographing coordinate system is consistent with the long side direction of the rectangular area formed by the four marks, and ensures that the X-axis direction of the photographing coordinate system is in an acute angle relationship with the X-axis direction of the end of the photographing robotic arm; the X-axis direction of the end of the photographing robotic arm on which the camera is mounted is installed in an opposite manner to the X-axis of the camera coordinate system. Therefore, the X-axis direction of the photographing coordinate system in the camera coordinate system at this time needs to be in an obtuse angle relationship with the X-axis of the current camera coordinate system to ensure that the camera reaches the photographing point with the minimum rotation angle.

[0064] S203, setting the Y-axis direction of the photographing coordinate system; according to the right-hand coordinate system principle, the Y-axis direction of the photographing coordinate system is set to

[0065] in, Represents the vector of the Z axis of the camera coordinate system under the camera; Represents the vector of the X-axis of the camera coordinate system under the camera.

[0066] At this point, the rotation matrix R formed by the direction of the photographic coordinate system is obtained c for:

[0067]

[0068] Among them, SO(n) represents the special orthogonal group, and when n=3, it represents the rotation in three-dimensional space.

[0069] The camera coordinate system direction is set as Figure 2 As shown, The arrows are red, green and blue respectively.

[0070] S3. Determine the position of the origin of the photographing coordinate system in the camera coordinate system according to the directions of the axes of the photographing coordinate system in step S2.

[0071] The steps to determine the position of the origin of the photographing coordinate system in the camera coordinate system are as follows: in the polishing area detection, the pixel positions of the four marks and the central pixel position formed by the four marks are identified, and the five pixel positions are mapped to the 3D space, which is represented as S i ;exist Lifting h above the casting surface c The height is set to the position of the camera's photo point.

[0072]

[0073] in, Indicates the locations of the five camera photo points.

[0074] Through the five camera shooting points planned for adaptive shooting height and angle, detailed data of the polishing area can be obtained from multiple perspectives.

[0075] S4. Based on steps S2 and S3, the camera shooting point position is calculated.

[0076] Define multiple camera coordinate systems under the camera coordinate system, and the expressions are as follows:

[0077]

[0078] in, is a special Euclidean group, the R matrix is ​​the rotation matrix, and the T matrix is ​​called the transformation matrix. is the translation matrix,

[0079] According to the current end position of the camera robot arm, the camera coordinate system is transformed to the camera robot arm coordinate system, and its expression is:

[0080]

[0081] Among them, Pose i Indicates the end position in the coordinate system of the camera robot arm; R x , R y , R z Represents the rotation matrix around the X, Y, and Z axes respectively; T cal Represents the "eye-on-hand" hand-eye calibration matrix between the structured light camera and the flange at the end of the slave robot arm.

[0082] According to the Euler angle rotation mode of the camera robot, formulas (3) and (4) are converted into the six-degree-of-freedom representation method of Euler angle. According to the sequence of the camera robot, the sequence is the order of the three rotations of Euler angle, and the sequence is converted into Euler angle form. When the sequence of the camera robot is ZYX, that is, it rotates around the Z axis first, then around the Y axis, and finally around the X axis, the transformation matrix is ​​converted into the six-degree-of-freedom Euler angle form, and its expression is:

[0083]

[0084] Among them, P pose (i)∈R 6×1It is the six-degree-of-freedom pose sent to the camera robot arm, which means the pose of the camera point set; (X i ,Y i ,Z i ) T Indicates the end position of the camera robot arm; (α i ,β i ,γ i ) T Represents the Euler angles of rotation around the X, Y, and Z axes; the function atan2(x, y) represents the inverse tangent; the poses of the five planned camera points are visualized and simulated, such as Figure 3 shown.

[0085] S5. Optimize the camera shooting point pose based on the particle swarm optimization algorithm to obtain the optimized reachable camera shooting point pose.

[0086] Due to the limitation of the grinding mechanism, the photographing robot arm is a collaborative robot arm with a small arm span, which leads to the fact that the camera photographing point pose calculated in step S4 is often unreachable. Therefore, the origin of the photographing coordinate system calculated in step S4 is adaptively corrected through the particle swarm optimization algorithm (PSO), and combined with the inverse solution state code of the photographing robot arm, it is judged whether the currently given photographing coordinate system pose is reachable.

[0087] The particle swarm optimization algorithm is designed based on the predation behavior of bird flocks. This algorithm is a type of swarm intelligence algorithm. It is known that there is a piece of food in a certain area. During the search process, each bird in the entire flock of birds determines whether it has found the food by sharing its own location information and the food information shared by the flock. Based on this implementation, the entire flock of birds will eventually gather near the food, that is, the optimal solution to the target is found. The particle swarm optimization algorithm is widely used in target optimization due to its advantages of high precision, fast convergence and easy implementation.

[0088] The basic principle of particle swarm algorithm is to simplify the simulation of birds in a flock by using a massless particle. The particle has two properties, namely speed and position. The direction and distance of the particle in the next iteration are represented by speed, and the position represents the current position of the particle. The main characteristics and description of the particle are shown in Table 1.

[0089] Table 1: Main characteristics and description of particles

[0090]

[0091] like Figure 4 As shown, point the camera to take a photo. pose (i) Set as food Figure 4 The pink point in the middle is the target value of the optimization problem. The particle swarm optimization algorithm is used to find the particle set near the target value, which represents the potential camera shooting point. Figure 4 Middle red point; the optimal solution of the particle Set it as the position of the potential photo point, and determine whether the potential photo point is reachable based on the inverse state code of the photo robot arm, filter out the poses reachable from the photo robot arm, and select a particle closest to the target value as the optimized photo point.

[0092] In the unreachable photo points, the particle swarm algorithm is used to continuously search for reachable points, so that the camera robot can adapt to the position of the photo points, thereby improving the reachability of the camera robot to reach the target position; the optimized pose of the reachable camera photo points is visualized and simulated, such as Figure 5 shown.

[0093] According to the adaptive planning, the camera robot adaptively captures the point cloud information of the casting from multiple angles to completely cover the area to be polished.

[0094] Example 2

[0095] An adaptive multi-viewpoint photo point planning system for large casting grinding comprises a photo module, a communication module and a main control module.

[0096] The photographing module is a photographing robot arm equipped with a structured light camera and has a wireless communication function, and is used to photograph casting images and plan the camera photographing point positions.

[0097] The communication module has a wireless communication function and is used to obtain the casting image taken by the camera and the posture of the camera robot arm.

[0098] The main control module is a hardware device equipped with a microprocessor and has a wireless communication function, which is used to process the casting image and output the optimized camera shooting point posture; the main control module collects the casting image through the structured light camera of the communication module and the shooting module, adopts the YOLOv5 network to identify the markers in the casting image, sets the axis direction of the shooting coordinate system under the camera coordinate system according to the markers, determines the position of the origin of the shooting coordinate system under the camera coordinate system, obtains the optimized reachable camera shooting point posture, uses the shooting robot arm to plan the camera shooting point posture, guides the camera to collect point cloud data, realizes the posture of multi-angle shooting of casting point cloud information, completely covers the area to be polished, and executes the planning algorithm described in Example 1 of the present invention.

[0099] The specific process of the planning system implementing the planning algorithm of Example 1 is as follows:

[0100] (1) The camera robot arm of the camera module uses the 2D image acquisition function of the structured light camera to photograph the area of ​​the casting to be polished containing the worker's mark.

[0101] (2) The communication module obtains the casting image captured by the camera and the position of the camera robot arm.

[0102] (3) The main control module uses the YOLOv5 network to identify the markers in the casting image transmitted by the communication module, sets the axis direction of the photographing coordinate system in the camera coordinate system according to the markers, determines the position of the origin of the photographing coordinate system in the camera coordinate system, obtains the optimized reachable camera photographing point pose, uses the photographing robot arm to plan the camera photographing point pose, guides the camera to collect point cloud data, and realizes the posture of multi-angle shooting of casting point cloud information, completely covering the area to be polished.

[0103] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. An adaptive multi-viewpoint photo point planning algorithm for large casting grinding, characterized in that: The following steps are involved: S1, collecting casting images, processing the casting images, and identifying the areas to be polished containing markers in the casting images; S2. Based on the marker identified in step S1, the directions of the axes of the photographing coordinate system in the camera coordinate system are set to improve the controllability of the posture of the photographing robot arm when it reaches the camera photographing point; S3, according to the directions of the axes of the photographing coordinate system in step S2, determining the position of the origin of the photographing coordinate system in the camera coordinate system; S4, based on steps S2 and S3, calculate the camera shooting point pose; S5. Optimize the camera shooting point pose based on the particle swarm optimization algorithm to obtain the optimized reachable camera shooting point pose.

2. The adaptive multi-viewpoint photo point planning algorithm for large casting grinding according to claim 1 is characterized in that: In step S1, the specific steps of collecting the casting image are: using the 2D image acquisition function of the structured light camera to capture the casting image of the area to be polished containing the marker, then identifying the marker in the casting image, and corresponding the identified marker pixel position to the 3D space.

3. The adaptive multi-viewpoint photo point planning algorithm for large casting grinding according to claim 2 is characterized in that: In step S1, a YOLOv5 network is used to identify markers in a casting image; the markers include four "×"-shaped marks in the area of ​​the casting to be polished.

4. The adaptive multi-viewpoint photo point planning algorithm for large casting grinding according to claim 1 is characterized in that: In step S1, the camera is perpendicular to the workpiece surface during shooting; and during the rotation of the photographing robot arm, the camera should be driven to shoot at the minimum rotation angle at its end.

5. The adaptive multi-viewpoint photo point planning algorithm for large casting grinding according to claim 1 is characterized in that: In step S2, the steps of setting the directions of the axes of the photographing coordinate system in the camera coordinate system include: S201, setting the direction of the Z axis of the photographing coordinate system; S202, setting the direction of the X axis of the photographing coordinate system; S203, setting the direction of the Y axis of the photographing coordinate system; In step S201, any three points of the four marks are selected in combination for plane fitting. After the direction vectors of the plane formed by the three points are obtained, the average vector is taken as the Z-axis direction of the photographing coordinate system. After the photographing robot arm carrying the camera arrives at the set photographing point, the Z-axis direction of the coordinate system of the end of the photographing robot arm, the Z-axis direction of the camera coordinate system, and the Z-axis direction of the photographing coordinate system are in the same direction. Based on this, the positive and negative directions of the Z-axis of the photographing coordinate system relative to the camera coordinate system are corrected to ensure that the camera is facing correctly.

6. The adaptive multi-viewpoint photo point planning algorithm for large casting grinding according to claim 5 is characterized in that: In step S202, the X-axis direction of the photographing coordinate system is consistent with the long side direction of the rectangular area formed by the four marks, and it is ensured that the X-axis direction of the photographing coordinate system is in an acute angle relationship with the X-axis direction of the end of the photographing mechanical arm; the X-axis direction of the end of the photographing mechanical arm is installed in an opposite manner to the X-axis of the camera coordinate system. Therefore, at this time, the X-axis direction of the photographing coordinate system in the camera coordinate system needs to be in an obtuse angle relationship with the X-axis of the current camera coordinate system to ensure that the camera reaches the photographing point at the minimum rotation angle; According to the right-hand coordinate system principle, the Y-axis direction of the camera coordinate system is set to in, Represents the vector of the Z axis of the camera coordinate system under the camera; Represents the vector of the X-axis of the camera coordinate system under the camera; At this point, the rotation matrix R formed by the direction of the photographing coordinate system is obtained c for: Among them, SO(n) represents the special orthogonal group, and when n=3, it represents the rotation in three-dimensional space.

7. The adaptive multi-viewpoint photo point planning algorithm for large casting grinding according to claim 1 is characterized in that: In step S3, the step of determining the position of the origin of the photographing coordinate system in the camera coordinate system is as follows: in the polishing area detection, the pixel positions of the four marks and the central pixel position formed by the four marks are identified, and the five pixel positions are mapped to the 3D space, which is represented as S i ;exist Lifting h above the casting surface c The height is set to the location of the camera's photo point; in, Indicates the locations of the five camera photo points.

8. The adaptive multi-viewpoint photo point planning algorithm for large casting grinding according to claim 1 is characterized in that: In step S4, multiple photographing coordinate systems under the camera coordinate system are defined, and the expressions are as follows: in, is a special Euclidean group, the R matrix is ​​the rotation matrix, and the T matrix is ​​called the transformation matrix. is the translation matrix, According to the current end position of the camera robot arm, the camera coordinate system is transformed to the camera robot arm coordinate system, and its expression is: Among them, Pose i Indicates the end position in the coordinate system of the camera robot arm; R x , R y , R z Respectively represent the rotation matrices around the X, Y, and Z axes; T cal Represents the "eye-on-hand" hand-eye calibration matrix between the structured light camera and the flange at the end of the slave robot arm; According to the Euler angle rotation mode of the camera robot, formulas (3) and (4) are converted into the six-degree-of-freedom representation method of Euler angles, and converted into the Euler angle form according to the alignment of the camera robot. When the alignment of the camera robot is ZYX, the transformation matrix is ​​converted into the six-degree-of-freedom Euler angle form, and its expression is: Among them, P pose (i) indicates the camera photo point; P pose (i)∈R 6×1 It is the six-degree-of-freedom pose sent to the camera robot arm, which means the pose of the camera point set; (X i , Y i , Z i ) T Indicates the end position of the camera robot arm; (α i , β i , γ i ) T Represents the Euler angle of rotation around the X, Y, and Z axes; the function atan2(x, y) represents the inverse tangent.

9. The adaptive multi-viewpoint photo point planning algorithm for large casting grinding according to claim 1, characterized in that: In step S5, the origin of the photographing coordinate system calculated in step S4 is adaptively corrected by the particle swarm optimization algorithm, and combined with the inverse solution state code of the photographing robot arm, it is determined whether the current photographing coordinate system pose is reachable; Press P to take a photo with the camera pose (i) Set the food as the target value of the optimization problem, and use the particle swarm optimization algorithm to find a particle set near the target value, which represents the potential camera shooting point; Set it as the position of the potential photo point, and determine whether the potential photo point is reachable based on the inverse state code of the photo robot arm, filter out the poses reachable from the photo robot arm, and select a particle closest to the target value as the optimized photo point.

10. An adaptive multi-viewpoint photo point planning system for large casting grinding, characterized in that: It includes a photographing module, a communication module and a main control module; the photographing module is a photographing mechanical arm equipped with a structured light camera, which has a wireless communication function and is used to photograph casting images and plan the camera photographing point posture; the communication module has a wireless communication function and is used to obtain the casting image photographed by the camera and the photographing mechanical arm posture; the main control module is a hardware device equipped with a microprocessor, which has a wireless communication function and is used to process the casting image and output the optimized camera photographing point posture; the main control module collects the casting image through the structured light camera of the communication module and the photographing module, adopts the YOLOv5 network to identify the markers in the casting image, sets the axis direction of the photographing coordinate system under the camera coordinate system according to the markers, determines the position of the coordinate origin of the photographing coordinate system under the camera coordinate system, obtains the optimized reachable camera photographing point posture, uses the photographing mechanical arm to plan the camera photographing point posture, guides the camera to collect point cloud data, realizes the posture of multi-angle photographing of casting point cloud information, completely covers the area to be polished, and executes the planning algorithm described in any one of claims 1-9.

Images