A robotic arm navigation method based on visual recognition and positioning
By adjusting the end of the robot arm parallel to the target position in the building robot, and combining visual recognition and ranging modules to establish a three-dimensional coordinate system, the problem of insufficient positioning accuracy of the robot arm under two-dimensional control in the existing technology is solved, and the problem of insufficient positioning accuracy of the robot arm under two-dimensional control is realized. The accurate navigation and positioning of three-dimensional space has been improved, and the accuracy and efficiency of construction are improved.
Patent Information
- Application Number
- CN202111492837.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-08
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-12-08
AI Technical Summary
The visual recognition technology of existing building robots can only control the robotic arm in two-dimensional coordinates, and cannot accurately locate and navigate the three-dimensional coordinates, making it difficult for the robotic arm to accurately reach a specific position during construction.
By adjusting the end of the robot arm is parallel to the outermost vertical plane of the target position, combining the visual identification module, ranging module and inclination sensor, a plane and three-dimensional coordinate system is established to realize the identification, positioning and navigation of the robot arm on the three-dimensional coordinates.
It realizes the precise positioning and navigation of the robotic arm in three-dimensional space, improves the accuracy and efficiency of construction, and overcomes the position error problem under two-dimensional control.
Smart Images

Figure CN114227674B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of architecture, and in particular to a robot arm navigation method based on visual recognition and positioning. Background Art
[0002] In recent years, the development of the construction industry has been accelerating, and the impact of robots on the new situation has become increasingly greater. Construction robots are based on the current market demand of the engineering machinery construction industry and are accelerating their integration into the transformation and upgrading of the construction industry, thereby continuously promoting automation and intelligent transformation in the construction field.
[0003] When construction is carried out by construction robots, the mechanical arms of the construction robots need to reach certain specific positions when facing different construction processes; although the visual recognition technology in the prior art can identify the required objects, it can only provide a two-dimensional coordinate, and therefore the mechanical arms can only be controlled in two dimensions. Summary of the invention
[0004] The present invention provides a robot arm navigation method based on visual recognition and positioning. By using the scheme of the present invention, the robot arm of a construction robot can realize recognition, positioning and navigation on three-dimensional coordinates, so that the robot arm can move to a specific position.
[0005] To achieve the above object, the technical solution of the present invention is: a robot arm navigation method based on visual recognition and positioning, comprising the following steps:
[0006] S1. Adjust the robotic arm until the end of the robotic arm is parallel to the outermost vertical plane where the target position is located.
[0007] S2. The visual recognition module identifies the two-dimensional coordinates of the center point of the target position.
[0008] Among them, the initial position of the end of the robot arm is used as the origin to establish the plane coordinate axis, and the center point of the camera's viewing range is used as the reference point; the two-dimensional coordinates (X1, Y1) of the target object are obtained according to the coordinate offset between the coordinates of the center point of the target position and the coordinates of the origin (0, 0); the coordinates of the reference point are (X2, Y2).
[0009] S3. Adjust the end of the robot arm until the coordinates of the reference point are the same as the two-dimensional coordinates of the center point of the target position.
[0010] S4. Obtain the distance Z2 between the end of the robotic arm and the target position through the distance measurement module.
[0011] S5. The control module drives the robotic arm to move forward a distance of Z2, so that the robotic arm reaches the target position.
[0012] The above method first adjusts the robotic arm so that the end of the robotic arm is parallel to the outermost vertical plane where the target position is located, thereby correcting the posture of the end of the robotic arm and preventing the accuracy of subsequent actions of the robotic arm from being affected by the error of the end of the robotic arm; then the two-dimensional coordinates of the center point of the target position are obtained through the visual recognition module, and the initial position of the end of the robotic arm is used as the origin to establish the plane coordinate axis, and the center point of the camera's viewing range is used as the reference point; the end of the robotic arm is adjusted to the same coordinates of the reference point as the two-dimensional coordinates of the center point of the target position; the robotic arm can be adjusted based on the plane coordinate system, and after the adjustment, the end of the robotic arm can be accurately located on the same straight line as the center point of the target position, and then the distance Z2 between the end of the robotic arm and the target position is obtained through the ranging module, and the control module drives the robotic arm to maintain the current two-dimensional coordinates and move a distance Z2 toward the center point of the target position, so that the end of the robotic arm can reach the center point of the target position.
[0013] Further, in step S1, the angle perpendicular to the gravity is measured by the inclination sensor to obtain the horizontal angle, and the control module uses the horizontal angle as a reference to determine the positive or negative angle of the inclination sensor feedback angle, thereby adjusting the inclination angle of the end of the robotic arm until the angle feedback from the inclination sensor is parallel to the gravity direction, and the end of the robotic arm is a vertical plane. At this time, the vertical plane where the end of the robotic arm is located is parallel to the outermost vertical plane where the target position is located; in this way, the posture of the end of the robotic arm can be corrected to prevent the accuracy of subsequent movements of the robotic arm from being affected by errors at the end of the robotic arm.
[0014] Furthermore, the step S3 specifically includes: the control module obtains the movement amount of the robot arm on the x-axis and y-axis according to the coordinate offset of the coordinates of the center point of the target position (X1, Y1) and the coordinates of the reference point (X2, Y2), and the actual coordinate distance ratio d; the control module drives the robot arm to move according to the feedback movement amount and movement direction, so that the reference point coordinates (X2, Y2) are equal to the two-dimensional coordinates of the target object (X1, Y1); in this way, the control module can obtain the movement amount between the end of the robot arm and the center point of the target position through the visual recognition module, and convert it into the actual movement amount of the robot arm on the x-axis and y-axis, so as to adjust accurately.
[0015] Furthermore, step S3 specifically includes: S3.1, when the robot arm moves to the center point of the target position, the moving distance on the x-axis is L1 and the moving distance on the y-axis is L2; the ratio of the coordinate value to the actual distance of the coordinate is d; wherein, L1=d·X1; L2=d·Y1; S3.2, the horizontal moving speed of the robot arm is V1, and the vertical moving speed of the robot arm is V2; when the robot arm moves to the center point of the target position, the moving time required on the x-axis is L1 / V1; the moving time required on the y-axis is L2 / V2; S3.3, the robot arm moves L1 / V1 seconds according to the x-axis moving direction fed back by the control module and then stops; the robot arm moves L2 / V2 seconds according to the y-axis moving direction fed back by the control module and then stops.
[0016] Furthermore, in step S4, it specifically includes: when the two-dimensional coordinates of the robotic arm are equal to the two-dimensional coordinates of the target position, obtaining the actual distance Z2 between the end of the robotic arm and the target position through the distance measurement module; establishing a three-dimensional coordinate system, with the origin of the two-dimensional coordinates of the robotic arm in step S3 as the origin of the x-axis and y-axis in the three-dimensional coordinate system, and the center of the current position of the robotic arm as the origin of the z-axis in the three-dimensional coordinate system; according to the product of the distance Z2 and the actual distance ratio of the coordinates, obtaining the coordinate Z1 of the center point of the target position on the z-axis, and then obtaining the three-dimensional coordinates (X1, Y1, Z1) of the target position; the above setting, based on the plane coordinate system of step S3, obtains the actual distance Z2 between the end of the robotic arm and the target position through the distance measurement module, and then establishes a three-dimensional coordinate system, and the offset between the end of the robotic arm and the center point of the target position in the three-dimensional coordinate system can be obtained in real time. If the end of the robotic arm is offset, it can be quickly corrected.
[0017] Further, in step S5, it specifically includes: the forward movement speed of the robot arm is V3; the forward movement time is Z1 / V3; and the robot arm stops moving after moving forward for Z1 / V3 seconds.
[0018] Furthermore, in step S4, if the center point of the target position is hollow, a reflector is set on one side of the center point of the target position; in this way, the problem that the ranging module cannot accurately identify the actual distance between the end of the robotic arm and the target position when the target position is a through hole or a hollow setting can be overcome.
[0019] Furthermore, the robotic arm is provided with a ranging module, a visual recognition module, a control module and an inclination sensor; the ranging module, the visual recognition module and the inclination sensor are respectively arranged at the ends of the robotic arm; the control module is arranged on the robotic arm, and the ranging module, the visual recognition module and the inclination sensor are respectively electrically connected to the control module. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 It is a schematic diagram of the structure of the present invention.
[0021] Figure 2 It is a processing flow chart of the present invention.
[0022] Figure numerals: 1. Robotic arm; 11. End of the robot arm; 2. Distance measurement module; 3. Visual recognition module; 4. Inclination sensor. DETAILED DESCRIPTION
[0023] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0024] like Figure 1-Figure 2 As shown, a robot arm navigation method based on visual recognition and positioning includes the following steps:
[0025] S1. Adjust the robotic arm 1 until the end 11 of the robotic arm is parallel to the outermost vertical plane where the target position is located; wherein, the angle perpendicular to the gravity is measured by the inclination sensor 4 to obtain the horizontal angle, and the control module uses the horizontal angle as a reference to determine the positive or negative angle of the feedback angle of the inclination sensor 4, thereby adjusting the inclination angle of the end 11 of the robotic arm until the angle feedback by the inclination sensor 4 is parallel to the gravity direction, and the end of the robotic arm 1 is a vertical plane. At this time, the vertical plane where the end 11 of the robotic arm is located is parallel to the outermost vertical plane where the target position is located.
[0026] S2. The visual recognition module 3 recognizes the two-dimensional coordinates of the center point of the target position.
[0027] Among them, the initial position of the end 11 of the robot arm is used as the origin to establish the plane coordinate axis, and the center point of the camera's viewing range is used as the reference point; according to the coordinate offset between the coordinates of the center point of the target position and the coordinates of the origin (0, 0), the two-dimensional coordinates (X1, Y1) of the target object are obtained; the coordinates of the reference point are (X2, Y2).
[0028] S3. Adjust the end 11 of the robot arm until the coordinates of the reference point are the same as the two-dimensional coordinates of the center point of the target position; the control module calculates the quotient of the coordinate offset of the center point of the target position (X1, Y1) and the reference point coordinates (X2, Y2) and the actual coordinate distance ratio d to obtain the movement amount of the robot arm 1 on the x-axis and y-axis; the control module drives the robot arm 1 to move according to the feedback movement amount and movement direction, so that the reference point coordinates (X2, Y2) are equal to the two-dimensional coordinates of the target object (X1, Y1).
[0029] S3.1. When the robot arm 1 moves to the center point of the target position, the moving distance on the x-axis is L1 and the moving distance on the y-axis is L2; the ratio of the coordinate value to the actual coordinate distance is d; where L1=d•X1; L2=d•Y1; in this embodiment, the actual coordinate distance ratio d is 1:10, that is, L1 is 10X1 and L2 is 10Y1.
[0030] S3.2, the horizontal moving speed of robot arm 1 is V1, and the vertical moving speed of robot arm 1 is V2; when robot arm 1 moves to the center point of the target position, the moving time required on the x-axis is L1 / V1; the moving time required on the y-axis is L2 / V2.
[0031] S3.3, the robot arm 1 moves L1 / V1 seconds according to the x-axis moving direction fed back by the control module and then stops; the robot arm 1 moves L2 / V2 seconds according to the y-axis moving direction fed back by the control module and then stops.
[0032] S4. Obtain the distance Z2 between the end 11 of the robot arm and the target position through the distance measurement module 2.
[0033] When the two-dimensional coordinates of the robot arm 1 are equal to the two-dimensional coordinates of the target position, the actual distance Z2 between the end 11 of the robot arm and the center point of the target position is obtained by the distance measurement module 2. Among them, if the center point of the target position is hollow, a reflector is set on one side of the center point of the target position. The reflector can be any flexible or rigid material that can reflect the laser emitted by the laser ranging sensor or the sound wave emitted by the ultrasonic ranging sensor; in this way, if it is necessary to perform construction processes such as cast-in-place, the target position can be a pouring port. At this time, the target position is a through hole. By setting a reflector, the distance measurement module 2 can accurately feedback the actual distance between the end 11 of the robot arm and the pouring port; and then the robot arm 1 can drive the pouring pipe used for pouring slurry to pass through the insertion and identify the pouring pipe as the target position, thereby overcoming the problem that the distance measurement module 2 cannot accurately identify the actual distance between the end 11 of the robot arm and the target position when the target position is a through hole or a hollow setting.
[0034] A three-dimensional coordinate system is established, with the two-dimensional coordinate origin of the robot 1 in step S3 as the origin of the x-axis and y-axis in the three-dimensional coordinate system, and the center of the current position of the robot 1 as the z-axis origin in the three-dimensional coordinate system; according to the product of the distance Z2 and the actual distance ratio of the coordinate, the coordinate Z1 of the center point of the target position on the z-axis is obtained, and then the three-dimensional coordinates (X1, Y1, Z1) of the target position are obtained.
[0035] S5, the control module drives the robot arm 1 to move forward a distance of Z2, so that the robot arm 1 reaches the target position; the forward movement speed of the robot arm 1 is V3; the forward movement time is Z1 / V3; the robot arm 1 moves forward for Z1 / V3 seconds and then stops.
[0036] The robotic arm 1 is provided with a distance measuring module 2, a visual recognition module 3, a control module and a tilt sensor 4; the distance measuring module 2, the visual recognition module 3 and the tilt sensor 4 are respectively arranged at the ends of the robotic arm 1; the control module is arranged on the robotic arm 1, and the distance measuring module 2, the visual recognition module 3 and the tilt sensor 4 are respectively electrically connected to the control module; wherein, the robotic arm 1 is provided with a base for rotating the robotic arm 1 at one end away from the distance measuring module 2; the distance measuring module 2 is a laser distance measuring sensor; the visual sensor is a camera; the control module can be a CPU, a PLC or a single-chip microcomputer, etc.
[0037] The robot arm 1 also includes a driving mechanism for driving the robot arm 1 to adjust its position, the driving mechanism including a horizontal motor for driving the robot arm end 11 to move in the x-axis direction, a vertical motor for driving the robot arm end 11 to move in the y-axis direction, and a forward motor for driving the robot arm end 11 to move in the z-axis direction.
Claims
1. A robot arm navigation method based on visual recognition and positioning, characterized in that: The following steps are involved: S1, adjust the robotic arm until the end of the robotic arm is parallel to the outermost vertical plane where the target position is located; S2, the visual recognition module identifies the two-dimensional coordinates of the center point of the target position; The initial position of the end of the robot arm is used as the origin to establish the plane coordinate axis, and the center point of the camera's viewing range is used as the reference point; the two-dimensional coordinates (X1, Y1) of the target object are obtained according to the coordinate offset between the center point of the target position and the coordinates of the origin (0, 0); the coordinates of the reference point are (X2, Y2); S3, adjust the end of the robot arm until the coordinates of the reference point are the same as the two-dimensional coordinates of the center point of the target position; S4, obtaining the distance Z2 between the end of the robotic arm and the target position through the distance measurement module; Establish a three-dimensional coordinate system, with the two-dimensional coordinate origin of the robot arm in step S3 as the origin of the x-axis and y-axis in the three-dimensional coordinate system, and the center of the current position of the robot arm as the origin of the z-axis in the three-dimensional coordinate system; According to the product of the distance Z2 and the actual distance ratio of the coordinates, the coordinate Z1 of the center point of the target position on the z-axis is obtained, and then the three-dimensional coordinates (X1, Y1, Z1) of the target position are obtained; when the two-dimensional coordinates of the robot arm are equal to the two-dimensional coordinates of the target position, the actual distance Z2 between the end of the robot arm and the center point of the target position is obtained through the distance measurement module; wherein, the target position is the pouring port, at this time the center point of the target position is hollow, and a reflector is set on one side of the center point of the target position. By setting the reflector, the distance measurement module can accurately feedback the actual distance between the end of the robot arm and the pouring port; S5. The control module drives the robotic arm to move forward a distance of Z2. The forward movement speed of the robotic arm is V3. The forward movement time is Z1 / V3. The robotic arm stops moving after moving forward for Z1 / V3 seconds.
2. The robot arm navigation method based on visual recognition and positioning according to claim 1, characterized in that: In step S1, the angle perpendicular to the gravity is measured by the inclination sensor to obtain the horizontal angle. The control module uses the horizontal angle as a reference to determine the positive or negative angle of the inclination sensor feedback angle, thereby adjusting the inclination angle of the end of the robotic arm until the angle feedback from the inclination sensor is parallel to the gravity direction and the end of the robotic arm is a vertical plane. At this time, the vertical plane where the end of the robotic arm is located is parallel to the outermost vertical plane where the target position is located.
3. The robot arm navigation method based on visual recognition and positioning according to claim 1, characterized in that: The step S3 specifically includes: the control module obtains the movement amount of the robot arm on the x-axis and y-axis according to the coordinate offset of the coordinates of the center point of the target position (X1, Y1) and the coordinates of the reference point (X2, Y2) and the actual coordinate distance ratio d; the control module drives the robot arm to move according to the feedback movement amount and movement direction so that the reference point coordinates (X2, Y2) are equal to the two-dimensional coordinates of the target object (X1, Y1).
4. The robot arm navigation method based on visual recognition and positioning according to claim 1, characterized in that: In step S3, specifically, it also includes: S3.
1. When the robot moves to the center point of the target position, the moving distance on the x-axis is L1 and the moving distance on the y-axis is L2; the ratio of the coordinate value to the actual distance of the coordinate is d; where L1=d·X1; L2=d·Y1; S3.2, the horizontal moving speed of the robot arm is V1, and the vertical moving speed of the robot arm is V2; when the robot arm moves to the center point of the target position, the moving time required on the x-axis is L1 / V1; the moving time required on the y-axis is L2 / V2; S3.
3. The robot arm moves L1 / V1 seconds according to the x-axis moving direction fed back by the control module and then stops; the robot arm moves L2 / V2 seconds according to the y-axis moving direction fed back by the control module and then stops.
5. The robot arm navigation method based on visual recognition and positioning according to claim 1, characterized in that: The robotic arm is provided with a distance measuring module, a visual recognition module, a control module and an inclination sensor; the distance measuring module, the visual recognition module and the inclination sensor are respectively arranged at the ends of the robotic arm; the control module is arranged on the robotic arm, and the distance measuring module, the visual recognition module and the inclination sensor are respectively electrically connected to the control module.
