A robot trajectory tracking system based on visual data and a method of application thereof

Abstract

This invention relates to the field of trajectory tracking technology, and discloses a robot trajectory tracking system based on visual data and its application method. The robot trajectory tracking system includes a visual camera, a host computer, and a robot; the visual camera is mounted on the robot, and both the visual camera and the robot are communicatively connected to the host computer. The movements of both the visual camera and the robot are controlled by the host computer. This invention can effectively reduce dependence on the environment and reduce deployment costs.

Description

Technical Field

[0001] This invention relates to the field of trajectory tracking technology, and more specifically, to a robot trajectory tracking system based on visual data and its application method. Background Technology

[0002] Industrial robots are widely used due to their advantages of high flexibility, low cost, and high workspace / floor area ratio. When used, robots rely on trajectories to perform processing operations.

[0003] Currently, commonly used trajectory tracking methods mainly plan trajectories by recognizing texture features (such as tile joints, wall tile edges, etc.) and physical markers (such as words, QR codes, etc.). Trajectory planning methods based on texture features are highly dependent on the environment. If there are stains, wear, or occlusions on the object surface, feature recognition is prone to failure. Trajectory planning methods based on physical markers require the pre-laying of physical markers. If the trajectory is changed, the machine needs to be stopped and the physical markers need to be re-laid, resulting in high deployment costs for small and medium batch production and temporary tasks. Summary of the Invention

[0004] The technical objective of this invention is to address the shortcomings of the prior art by providing a vision-based robot trajectory tracking system and its application method that can effectively reduce dependence on the environment and lower deployment costs.

[0005] The first technical solution adopted in this invention is as follows: A robot trajectory tracking system based on visual data, the robot trajectory tracking system comprising a visual camera, a host computer, and a robot; The vision camera is mounted on the robot, and both the vision camera and the robot are communicatively connected to the host computer. The actions of both the vision camera and the robot are controlled by the host computer.

[0006] Furthermore, the robot includes an AGV (Automated Guided Vehicle) and a robotic arm mounted on the AGV; The vision camera is mounted on the robotic arm; The robotic arm and the AGV are respectively connected to the host computer for communication, and the movements of the robotic arm and the AGV are controlled by the host computer.

[0007] The second technical solution adopted in this invention is as follows: An application method for a robot trajectory tracking system based on visual data as described in the above technical solution 1, the application method comprising the following steps: S1. Mark the position of the workpiece to be processed in advance; S2. The host computer controls the vision camera to acquire images of the workpiece at the location to be processed, obtains the original image, and sends it to the host computer. S3. The host computer determines whether the original image contains at least some markers. If so, it extracts the feature information of the markers in the original image and performs preprocessing to obtain multiple data points and their pixel coordinates. If not, the operation ends. S4. Convert the pixel coordinates of each data point into physical coordinates and send them to the robot. The robot performs corresponding operations based on the physical coordinates of each data point. After the operations are completed, return to step S2 until the host computer receives a manual stop command, at which point the operation ends.

[0008] Furthermore, the mark is a dashed mark.

[0009] The above-mentioned technical measures can be completed by drawing lines, which effectively reduces deployment costs and operational difficulties.

[0010] Furthermore, the specific steps for the host computer to extract the marked feature information from the original image are as follows: The host computer extracts the marked feature information in the original image through edge detection to obtain the upper edge feature curve and the lower edge feature curve.

[0011] Furthermore, the preprocessing includes: discretizing the upper edge feature curve and the lower edge feature curve to obtain multiple upper edge points and their pixel coordinates and lower edge points and their pixel coordinates, and sequentially performing jump point processing, median filtering and Kalman filtering on the upper edge points and the lower edge points.

[0012] The above-mentioned technical measures, through jump point processing, median filtering, and Kalman filtering, can improve data quality and achieve smooth tracking of the markers; median filtering can remove noise, and Kalman filtering can perform optimal estimation, effectively improving data accuracy and stability.

[0013] Furthermore, the specific steps of the discretization process are as follows: A pixel coordinate system is established, and the upper edge feature curve and the lower edge feature curve are sampled at a preset interval to obtain multiple upper edge points and their pixel coordinates, and lower edge points and their pixel coordinates.

[0014] Furthermore, the specific steps of the jump point processing are as follows: A standard distance is preset between upper and lower edge points with the same X-axis coordinate value. The difference between the Y-axis coordinate values ​​of upper and lower edge points with the same X-axis coordinate value is calculated. The error value between the difference and the standard distance is calculated. It is determined whether the error value exceeds a preset error range. If so, the corresponding upper and lower edge points jump to a new point, and interpolation processing is performed on the upper and lower edge points that jump to a new point.

[0015] The above-mentioned technical measures calculate the error between the difference and the standard distance to determine whether a jump point has occurred, and then perform interpolation processing to correct the error caused by environmental factors, effectively reducing dependence on the environment.

[0016] Furthermore, the interpolation process includes: taking the upper and lower edge points where the jump point occurs as centers, obtaining the pixel coordinates of adjacent upper and lower edge points respectively, calculating the corrected coordinates of the upper and lower edge points where the jump point occurs through an interpolation algorithm, and replacing the pixel coordinates of the upper and lower edge points where the jump point occurs with the corrected coordinates.

[0017] Furthermore, the specific steps for converting the pixel coordinates of each data point into physical coordinates are as follows: converting the pixel coordinates of each data point into physical coordinates using a coordinate transformation algorithm.

[0018] One or more technical solutions provided by this invention have at least the following technical effects or advantages: This invention supports marking on any plane of a workpiece and generates motion trajectories by recognizing the marks, eliminating the need for pre-laying fixed marks or complex programming, thus effectively improving flexibility.

[0019] This invention allows for setting robot movement trajectories simply by marking lines, making it easy to operate and effectively reducing the technical requirements for technicians. It also reduces deployment costs and time costs for small-batch production and temporary tasks.

[0020] This invention uses a vision camera to capture images multiple times, which can adapt to real-time adjustments of the markers, enabling instant updates and tracking, and effectively improving efficiency.

[0021] This invention corrects errors caused by environmental factors through jump point processing, median filtering, and Kalman filtering, effectively reducing dependence on the environment, improving data quality, and thus increasing the tracking success rate. Attached Figure Description

[0022] The accompanying drawings, which are provided to further illustrate embodiments of the invention and constitute a part of this invention, are not intended to limit the scope of the invention. Figure 1 This is a structural block diagram of the robot trajectory tracking system based on visual data in this invention; Figure 2 This is a flowchart of the application method of the robot trajectory tracking system based on visual data in this invention. Detailed Implementation

[0023] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, where there is no conflict, the embodiments of the present invention and the features thereof can be combined with each other.

[0024] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0025] Reference Figure 1 This embodiment provides a robot trajectory tracking system based on visual data. The robot trajectory tracking system includes a visual camera, a host computer, and a robot. The visual camera is installed on the robot, and both the visual camera and the robot are communicatively connected to the host computer. The movements of both the visual camera and the robot are controlled by the host computer.

[0026] The robot consists of an AGV (Automated Guided Vehicle) and a robotic arm mounted on the AGV. A vision camera is mounted on a robotic arm; The robotic arm and AGV are connected to the host computer for communication, and the movements of the robotic arm and AGV are controlled by the host computer.

[0027] The host computer is an industrial computer or similar equipment.

[0028] Reference Figure 2 This embodiment also proposes an application method for the above-mentioned robot trajectory tracking system based on visual data. The application method includes the following steps: S1. Mark the position of the workpiece to be processed in advance; The marking is a line mark, which can be drawn manually.

[0029] S2. The host computer controls the vision camera to acquire images of the workpiece at the location to be processed, obtains the original image, and sends it to the host computer. The shooting parameters of the vision camera are set according to the actual application scenario to ensure clear and accurate image acquisition.

[0030] S3. The host computer determines whether the original image contains at least some markers. If so, it extracts the feature information of the markers in the original image and performs preprocessing to obtain multiple data points and their pixel coordinates. If not, the operation ends. The specific steps for the host computer to extract the marked feature information from the original image are as follows: The host computer extracts the marked feature information from the original image through edge detection, and obtains the upper edge feature curve and the lower edge feature curve.

[0031] The preprocessing includes: discretizing the upper and lower edge feature curves to obtain multiple upper edge points and their pixel coordinates, and lower edge points and their pixel coordinates, and then performing jump point processing, median filtering, and Kalman filtering on the upper and lower edge points in sequence.

[0032] The specific steps of discretization are as follows: A pixel coordinate system is established, and the upper and lower edge feature curves are sampled at preset intervals to obtain multiple upper edge points and their pixel coordinates, as well as lower edge points and their pixel coordinates. The pixel coordinate system has its origin at the top left corner of the original image, with the X-axis pointing to the right and the Y-axis pointing downwards, and the unit is pixels. In this embodiment, the preferred interval is 10 pixels, 20 pixels, or 80 pixels.

[0033] The specific steps for handling jump points are as follows: A standard distance (i.e., standard line width) is preset between upper and lower edge points with the same X-axis coordinate value. The difference in Y-axis coordinate values ​​between these points is calculated, and the error between this difference and the standard distance is calculated. It is then determined whether the error exceeds a preset error range. If so, the corresponding upper and lower edge points are considered to have skipped a point, and interpolation is performed on these skipped points. The standard distance and error range are determined based on actual needs and are not specifically limited in this embodiment.

[0034] The interpolation process includes: taking the upper and lower edge points where the jump point occurs as centers, obtaining the pixel coordinates of adjacent upper and lower edge points respectively, calculating the corrected coordinates of the upper and lower edge points where the jump point occurs through an interpolation algorithm, and replacing the pixel coordinates of the upper and lower edge points where the jump point occurs with the corrected coordinates.

[0035] The socket algorithm can be determined according to actual needs. In this embodiment, the lines are drawn manually, and the preferred algorithms are segmented cubic Hermit interpolation (PCHIP) and inverse distance weighted interpolation (IDW).

[0036] S4. Convert the pixel coordinates of each data point into physical coordinates and send them to the robot. The robot performs the corresponding operation based on the physical coordinates of each data point. After the operation is completed, return to step S2 until the host computer receives a manual stop command, and the operation ends.

[0037] The host computer receives manual stop commands from manual operators via a touch screen or control panel.

[0038] The specific steps for converting the pixel coordinates of each data point to physical coordinates are as follows: The pixel coordinates of each data point are converted to physical coordinates using a coordinate transformation algorithm. The coordinate transformation algorithm used is an existing algorithm, and the choice depends on the actual needs; this embodiment does not impose any specific limitations.

[0039] In application, the host computer controls the AGV to follow the marker and move the robotic arm to the designated position. After reaching the position, the host computer controls the robotic arm to perform marker-following and cutting. During the marker-following and cutting process, the AGV remains stationary. During the movement of the AGV, the robotic arm remains stationary.

[0040] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0041] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A robot trajectory tracking system based on visual data, characterized in that: the robot trajectory tracking system comprises a visual camera, a host computer and a robot; the visual camera is installed on the robot, and the visual camera and the robot are respectively connected in communication with the host computer, and the actions of the visual camera and the robot are controlled by the host computer. 2.The robot trajectory tracking system based on visual data according to claim 1, characterized in that: the robot comprises an AGV and a mechanical arm installed on the AGV; the visual camera is installed on the mechanical arm; the mechanical arm and the AGV are respectively connected in communication with the host computer, and the actions of the mechanical arm and the AGV are controlled by the host computer. 3.An application method of the robot trajectory tracking system based on visual data according to any one of claims 1 to 2, characterized in that: the application method comprises the following steps: S1, marking a workpiece to-be-processed position in advance; S2, the host computer controls the visual camera to collect an image of the workpiece to-be-processed position, obtains an original image and sends it to the host computer; S3, the host computer judges whether the original image includes at least part of the mark, if yes, extracts the feature information of the mark in the original image, and pre-processes to obtain a plurality of data points and their pixel coordinates; if not, the work is completed; S4, converting the pixel coordinates of each data point into physical coordinates and sending them to the robot, and the robot performs corresponding operations according to the physical coordinates of each data point, and after the execution is completed, returns to step S2 until the host computer receives a manual stop instruction, and the work is completed. 4.The application method of the robot trajectory tracking system based on visual data according to claim 3, characterized in that: the mark is a scribe mark. 5.The application method of the robot trajectory tracking system based on visual data according to claim 3, characterized in that: the specific steps of the host computer extracting the feature information of the mark in the original image are: the host computer extracts the feature information of the mark in the original image through edge detection to obtain an upper edge feature curve and a lower edge feature curve. 6.The application method of the robot trajectory tracking system based on visual data according to claim 5, characterized in that: the pre-processing comprises: discretizing the upper edge feature curve and the lower edge feature curve to obtain a plurality of upper edge points and their pixel coordinates and lower edge points and their pixel coordinates, and sequentially performing skip point processing, median filtering and Kalman filtering on the upper edge points and the lower edge points. 7.The application method of the robot trajectory tracking system based on visual data according to claim 6, characterized in that: the specific steps of the discretization processing are: establishing a pixel coordinate system, and sampling the upper edge feature curve and the lower edge feature curve at a predetermined interval to obtain a plurality of upper edge points and their pixel coordinates and lower edge points and their pixel coordinates. 8.The application method of the robot trajectory tracking system based on visual data according to claim 6, characterized in that: the specific steps of the skip point processing are: A standard distance is preset between upper and lower edge points with the same X-axis coordinate value. The difference between the Y-axis coordinate values ​​of upper and lower edge points with the same X-axis coordinate value is calculated. The error value between the difference and the standard distance is calculated. It is determined whether the error value exceeds a preset error range. If so, the corresponding upper and lower edge points jump to a new point, and interpolation processing is performed on the upper and lower edge points that jump to a new point.

9. The application method of the robot trajectory tracking system based on visual data according to claim 8, characterized in that: The interpolation process includes: taking the upper and lower edge points where the jump point occurs as centers, obtaining the pixel coordinates of adjacent upper and lower edge points respectively, calculating the corrected coordinates of the upper and lower edge points where the jump point occurs through an interpolation algorithm, and replacing the pixel coordinates of the upper and lower edge points where the jump point occurs with the corrected coordinates.

10. The application method of the robot trajectory tracking system based on visual data according to claim 3, characterized in that: The specific steps for converting the pixel coordinates of each data point into physical coordinates are as follows: converting the pixel coordinates of each data point into physical coordinates using a coordinate transformation algorithm.

Images