Teaching method, electronic device and computer readable storage medium

Abstract

The application discloses a teaching method, an electronic device and a computer readable storage medium. The teaching method comprises the following steps: acquiring data collected by an angular velocity sensor and an acceleration sensor in a terminal during movement of the terminal along a teaching track; determining attitude information of each teaching point on the teaching track according to at least the data collected by the angular velocity sensor during movement of the terminal; determining position information of each teaching point according to the data collected by the acceleration sensor during movement of the terminal; and controlling a robot to reproduce the teaching track according to the attitude information and the position information of each teaching point. The teaching method provided by the application can teach any track at low cost and is convenient to operate.

Description

Technical Field

[0001] This application belongs to the field of robot teaching technology, and in particular relates to a teaching method, electronic device and computer-readable storage medium. Background Technology

[0002] Currently, traditional industrial robot teaching methods mainly include general-purpose teach pendant teaching, drag-and-drop teaching, and laser teach pendant teaching. However, these main methods all have various drawbacks. Specifically, when using a teach pendant, the pendant is relatively heavy, making it inconvenient for the user to operate, and it is also relatively expensive. The teach pendant cable is inconvenient to drag and has a limited length. When using drag-and-drop teaching, there are generally two specific implementation methods. The first is through a torque sensor, but this method is expensive. The second is to estimate the torque by converting the current of each axis motor into torque. This method relies on the accurate establishment of the robot's body dynamics model and accurate friction compensation. However, friction compensation is greatly affected by temperature, speed, and lifespan, so the drag-and-drop teaching accuracy of this method is poor. When using a laser teach pendant, dedicated laser teach pendants are expensive, and the laser emitter and receiver of the laser teach pendant need to be set up in advance, with some limitations on the height and angle range of the setup.

[0003] Therefore, in view of the shortcomings of existing technologies, there is an urgent need for a simple and low-cost teaching method. Summary of the Invention

[0004] This application provides a teaching method, an electronic device, and a computer-readable storage medium that enable teaching of arbitrary trajectories at low cost and with convenient operation.

[0005] A first aspect of this application provides a teaching method, the method comprising: acquiring data collected by an angular velocity sensor and an accelerometer in the terminal during the terminal's movement along a teaching trajectory; determining the attitude information of each teaching point on the teaching trajectory based at least on the data collected by the angular velocity sensor during the terminal's movement; determining the position information of each teaching point based on the data collected by the accelerometer during the terminal's movement; and controlling a robot to reproduce the teaching trajectory based on the attitude information and the position information of each teaching point.

[0006] A second aspect of this application provides an electronic device, which includes a processor, a memory, and a communication circuit. The processor is coupled to the memory and the communication circuit, respectively. The memory stores program data, and the processor executes the program data in the memory to implement the steps in the above method.

[0007] A third aspect of this application provides an electronic device, comprising: an acquisition module, configured to acquire data collected by an angular velocity sensor and an accelerometer in the terminal during the terminal's movement along a teaching trajectory; a first determination module, connected to the acquisition module, configured to determine the attitude information of each teaching point on the teaching trajectory based at least on the data collected by the angular velocity sensor during the terminal's movement; a second determination module, connected to the acquisition module, configured to determine the position information of each teaching point based on the data collected by the accelerometer during the terminal's movement; and a reproduction module, connected to the first determination module and the second determination module, configured to control a robot to reproduce the teaching trajectory based on the attitude information and position information of each teaching point.

[0008] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that can be executed by a processor to implement the steps in the above-described method.

[0009] The beneficial effects are as follows: During the teaching process of this application, the angular velocity sensor and accelerometer in the terminal acquire data during the terminal's movement along the teaching trajectory. Then, based on the data collected by the angular velocity sensor during the terminal's movement, the attitude information of each teaching point on the teaching trajectory is determined, and based on the data collected by the accelerometer during the terminal's movement, the position information of each teaching point is determined. Finally, after determining the attitude and position information of each teaching point, the robot can be controlled to reproduce the teaching trajectory. Thus, the entire teaching process can be completed with the help of the terminal, without the need for other hardware devices. It can teach any trajectory at low cost and is easy to operate. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0011] Figure 1 This is a flowchart illustrating one implementation method of the teaching method of this application;

[0012] Figure 2 yes Figure 1 A flowchart illustrating step S130;

[0013] Figure 3 yes Figure 2 A flowchart illustrating step S133;

[0014] Figure 4 This is a schematic diagram captured by the terminal.

[0015] Figure 5 This is a schematic diagram of the structure of one embodiment of the electronic device of this application;

[0016] Figure 6 This is a schematic diagram of another embodiment of the electronic device of this application;

[0017] Figure 7 This is a schematic diagram of one embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0019] It should be noted that the terms "first" and "second" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0020] First, the teaching method of this application can be executed by a robot control cabinet or by other electronic devices with algorithm processing capabilities, such as computers.

[0021] See Figure 1 , Figure 1 This is a flowchart illustrating one embodiment of the teaching method of this application, which includes:

[0022] S110: During the process of the terminal moving along the teaching trajectory, acquire the data collected by the angular velocity sensor and the acceleration sensor in the terminal.

[0023] Specifically, the teaching trajectory is the trajectory that the robot is expected to replicate. This can be achieved by manually controlling the terminal to move along the teaching trajectory, or by using other control devices to control the terminal to move along the teaching trajectory.

[0024] The movement of the terminal along the teaching trajectory means that a certain point on the control terminal moves along or approximately along the teaching trajectory. In other words, during the movement of the terminal, the movement trajectory of a certain point on the terminal is or approximately the teaching trajectory.

[0025] The terminal can be any electronic device that includes both an angular velocity sensor and an accelerometer, such as a mobile phone or a tablet computer. Given the high penetration rate of mobile phones and the fact that most mobile phones include both angular velocity and accelerometer sensors, a mobile phone can be designated as the terminal.

[0026] S120: Determine the attitude information of each teaching point on the teaching trajectory based at least on the data collected by the angular velocity sensor during the terminal's movement.

[0027] Specifically, determining the teaching trajectory essentially means determining the attitude and position information of each teaching point on the trajectory. The attitude information of a teaching point can specifically be attitude angles. It is understandable that different teaching points correspond to different time points.

[0028] Among them, the angular velocity sensor, also known as a gyroscope, applies the Coriolis force principle and has a built-in special ceramic device that can accurately measure angles.

[0029] In one application scenario, the angular velocity sensor itself has the function of detecting the direction of gravity. It can measure the direction of gravity. At this time, the angle measured by the angular velocity sensor is the attitude angle of the terminal. Therefore, the attitude angle collected by the angular velocity sensor at a certain teaching point can be directly determined as the attitude angle of the teaching point.

[0030] In another application scenario, the angular velocity sensor is a standard angular velocity sensor that does not have gravity direction detection capabilities. In this case, the angular velocity sensor does not have a basic zero point, and the angle measured by the angular velocity sensor is only the angle change value of the terminal. In order to determine the attitude information of the teaching point, the terminal is further equipped with a gravity sensor. This gravity sensor determines the gravity direction by measuring the gravitational acceleration. In this case, step S120 specifically includes:

[0031] S121: For each teaching point, determine the attitude information of the teaching point based on the data collected by the gravity sensor and the data collected by the angular velocity sensor.

[0032] Specifically, for any teaching point, the angle collected by the gravity sensor and the angle collected by the angular velocity sensor at the teaching point are added together to obtain the attitude angle corresponding to the teaching point.

[0033] S130: Determine the position information of each teaching point based on the data collected by the accelerometer during the movement of the terminal.

[0034] Specifically, an accelerometer is an instrument that measures the linear acceleration of an object. Accelerometers used in terminals are usually MEMS (Micro-Electro-Mechanical Systems), which include internal vibration structures. As the terminal moves, external acceleration affects the vibration characteristics of the internal vibration structure of the accelerometer, thereby allowing acceleration to be measured. By integrating the measured acceleration, velocity can be obtained, and by integrating it again, position information can be obtained, including the distance and direction of the motion.

[0035] It is understandable that the location information of the teaching starting point of any teaching trajectory is known. Therefore, for any teaching point located after the teaching starting point, in one application scenario, after determining its distance and direction relative to the teaching starting point, the location information of the teaching point can be determined. In another application scenario, after determining its distance and direction relative to the teaching point located before it and whose location information is known, the location information of the teaching point can be determined.

[0036] See Figure 2 In this embodiment, step S130 specifically includes:

[0037] S131: Obtain the acceleration collected by the accelerometer at the second teaching point, where the second teaching point is located after the first teaching point.

[0038] S132: Determine the result of integrating the acceleration over the target time interval twice to obtain the direction and initial distance of the second teaching point relative to the first teaching point, where the target time interval is the time interval between the second teaching point and the first teaching point.

[0039] S133: Determine the position information of the second teaching point based on the position information of the first teaching point and the direction and initial distance of the second teaching point relative to the first teaching point.

[0040] Specifically, the first teaching point can be any teaching point located before the second teaching point. For example, the first teaching point can be the starting point of the teaching trajectory, or it can be a teaching point located before and adjacent to the second teaching point.

[0041] In this system, the first teaching point is denoted as teaching point A, and the second teaching point is denoted as teaching point B. The time interval between teaching point A and teaching point B is T. During the movement of the terminal, it first moves to teaching point A and then to teaching point B. The acceleration of the terminal at teaching point B is obtained through an accelerometer. Then, the integral of this acceleration with respect to T is determined twice to obtain the direction and initial distance of the second teaching point relative to the first teaching point. Specifically, the first result after integrating the acceleration with respect to T once is determined. This first result is the velocity of the terminal during its movement from teaching point A to teaching point B. This velocity is a vector, carrying both a velocity value and a direction. Then, the second result after integrating the first result with respect to T once is determined. This second result is the direction and initial distance of teaching point B relative to teaching point A.

[0042] In one application scenario, step S133 directly determines the initial distance as the final distance between the second teaching point and the first teaching point. At this time, the position information of the second teaching point is determined directly based on the position of the first teaching point, the direction of the second teaching point relative to the first teaching point, and the initial distance.

[0043] In another application scenario, considering that the initial distance between the second teaching point and the first teaching point is obtained through two integration processes, resulting in a large accumulated error, in order to accurately determine the distance between the second teaching point and the first teaching point, and to ensure the accurate determination of the position information of the second teaching point, please refer to... Figure 3 Step S133 specifically includes:

[0044] S1331: Acquire a first image captured by the terminal at the first teaching point, a second image captured at the second teaching point, and a third image captured at the third teaching point during the movement of the terminal. The third teaching point is located after the second teaching point, and the first, second, and third images all include the same target object, and the extension direction of the target object is parallel to the direction of the terminal's movement.

[0045] Specifically, in combination Figure 4 The first image, the second image, and the third image all include the same target object, which is denoted as L, and the extension direction of the target object L is parallel to the direction of the terminal movement.

[0046] S1332: Based on the first image, the second image, and the initial distance between the second teaching point and the first teaching point, and using a preset algorithm, determine the first length of the target object in its extension direction.

[0047] Specifically, the preset algorithm can be a binocular vision algorithm or other algorithms. The following explanation will take the binocular vision algorithm as the preset algorithm.

[0048] Combination Figure 4Let S1 be the initial distance between the second teaching point and the first teaching point. Based on the first image, the second image, and the initial distance S1 between the second teaching point and the first teaching point, the first length L1 of the target object L in its extension direction can be determined using a binocular vision algorithm.

[0049] Binocular vision algorithms refer to observing an object from two or more points, acquiring images from different perspectives, and calculating the offset between pixels based on the matching relationship between pixels in the images using the principle of triangulation, thereby obtaining the three-dimensional information of the object.

[0050] It is understandable that the initial distance between the second teaching point and the first teaching point is obtained through two integration processes. The determined initial distance between the second teaching point and the first teaching point deviates from the actual distance between the second teaching point and the first teaching point. Since the first length L1 is determined based on the initial distance between the second teaching point and the first teaching point, the first length L1 also deviates from the actual length of the target object.

[0051] S1333: Based on the second image, the third image, and the initial distance between the third teaching point and the second teaching point, and using a preset algorithm, determine the second length of the target object in its extension direction.

[0052] Specifically, the first, second, and third teaching points are arranged sequentially on the teaching trajectory. Referring to the method for determining the initial distance and direction of the second teaching point relative to the first teaching point, the initial distance and direction of the third teaching point relative to the second teaching point can be determined. Figure 4 Let S2 be the initial distance between the third teaching point and the second teaching point.

[0053] Similarly, based on the second image, the third image, and the initial distance S2 of the third teaching point relative to the second teaching point, the second length L2 of the target object L in its extension direction can be determined using a binocular vision algorithm. Likewise, the initial distance of the third teaching point relative to the second teaching point deviates from the actual distance between the third teaching point and the second teaching point. Since the second length L2 is determined based on the initial distance of the third teaching point relative to the second teaching point, the second length L2 also deviates from the actual length of the target object.

[0054] S1334: Determine the target length based on the first length and the second length.

[0055] Specifically, the first length and the second length can be weighted and summed to obtain the target length.

[0056] For example, in this embodiment, the average of the first length and the second length is calculated to obtain the target length. It is understood that the first length L1 deviates from the actual length of the target object, and the second length L2 also deviates from the actual length of the target object. However, compared with the first length L1 and the second length L2, the target length is closer to the actual length of the target object.

[0057] It should be noted that in other embodiments, to further improve accuracy, more teaching points can be used to determine the target length. For example, in addition to using the first, second, and third teaching points, a fourth teaching point is also used. Correspondingly, in addition to acquiring the first, second, and third images, a fourth image captured by the terminal during movement at the fourth teaching point is also acquired, and the first, second, third, and fourth images all include the same target object. Referring to the process of determining the first and second lengths of the target object, the third length of the target object is determined based on the fourth image, the third image, and the initial distance of the fourth teaching point relative to the third teaching point. Then, the target length is determined based on the first, second, and third lengths. For example, the average value of the first, second, and third lengths can be used to determine the target length.

[0058] Similarly, more teaching points can be used, such as the fifth teaching point, the sixth teaching point, etc., as long as the same target object is included in all the images used.

[0059] S1335: Based on the first image, the second image, and the target length, and using a preset algorithm, determine the target distance of the second teaching point relative to the first teaching point.

[0060] Specifically, the above uses a binocular vision algorithm to determine the first length of the target object based on the first image, the second image, and the initial distance between the second teaching point and the first teaching point. Here, the same binocular vision algorithm is used to determine the target distance between the second teaching point and the first teaching point based on the first image, the second image, and the target length. This target distance is then determined as the final distance between the second teaching point and the first teaching point.

[0061] Understandably, by using the same method, the target distance of the third teaching point relative to the second teaching point can be determined, which is the final distance of the third teaching point relative to the second teaching point.

[0062] S1336: Determine the location information of the second teaching point based on the location information of the first teaching point, the direction of the second teaching point relative to the first teaching point, and the target distance.

[0063] Specifically, after obtaining the target distance between the second teaching point and the first teaching point, the target distance is used as the final distance between the second teaching point and the first teaching point. Then, based on the position information of the first teaching point and the direction and target distance of the second teaching point relative to the first teaching point, the position information of the second teaching point can be accurately determined.

[0064] It is understandable that, after obtaining the location information of the second teaching point, the location information of the third teaching point can also be accurately determined based on the target distance of the third teaching point relative to the second teaching point.

[0065] S140: Based on the posture and position information of each teaching point, control the robot to reproduce the teaching trajectory.

[0066] Specifically, after determining the pose and position information of each teaching point, the robot can be controlled to move according to the pose and position information of each teaching point, thereby reproducing the teaching trajectory.

[0067] As can be seen from the above, this application can teach a teaching trajectory using only a terminal including an angular velocity sensor and an acceleration sensor, enabling the teaching of any trajectory at low cost and with convenient operation.

[0068] See Figure 5 , Figure 5 This is a schematic diagram of one embodiment of the electronic device of this application. The electronic device 200 includes a processor 210, a memory 220, and a communication circuit 230. The processor 210 is coupled to the memory 220 and the communication circuit 230 respectively. The memory 220 stores program data. The processor 210 executes the program data in the memory 220 to implement the steps in any of the above embodiments. The detailed steps can be found in the above embodiments and will not be repeated here.

[0069] Among them, electronic device 200 can be any device with algorithm processing capabilities, such as a computer, mobile phone, or robot control cabinet, without any restrictions.

[0070] See Figure 6 , Figure 6 This is a schematic diagram of another embodiment of the electronic device of this application. The electronic device 300 includes an acquisition module 310, a first determination module 320, a second determination module 330, and a reproduction module 340.

[0071] The acquisition module 310 is used to acquire data collected by the angular velocity sensor and the acceleration sensor in the terminal during the process of the terminal moving along the teaching trajectory.

[0072] The first determining module 320 is connected to the acquiring module 310 and is used to determine the attitude information of each teaching point on the teaching trajectory based at least on the data collected by the angular velocity sensor during the movement of the terminal.

[0073] The second determining module 330 is connected to the acquiring module 310 and is used to determine the position information of each teaching point based on the data collected by the accelerometer during the movement of the terminal.

[0074] The reproduction module 340 is connected to the first determination module 320 and the second determination module 330, and is used to control the robot to reproduce the teaching trajectory based on the posture information and position information of each teaching point.

[0075] Among them, electronic device 300 can be any device with algorithm processing capabilities, such as a computer, mobile phone, or robot control cabinet, without any restrictions.

[0076] When the electronic device 300 is in operation, it executes the method steps in any of the above embodiments. For detailed method steps, please refer to the relevant content above, which will not be repeated here.

[0077] See Figure 7 , Figure 7 This is a schematic diagram of one embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 400 stores a computer program 410, which can be executed by a processor to implement the steps in any of the above methods.

[0078] Specifically, the computer-readable storage medium 400 can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or a device that can store the computer program 410. Alternatively, it can be a server that stores the computer program 410, which can send the stored computer program 410 to other devices for execution, or it can run the stored computer program 410 itself.

[0079] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A teaching method, characterized in that, The method includes: During the process of the terminal moving along the teaching trajectory, data collected by the angular velocity sensor and acceleration sensor in the terminal are acquired; The attitude information of each teaching point on the teaching trajectory is determined based at least on the data collected by the angular velocity sensor during the movement of the terminal; Based on the data collected by the accelerometer during the movement of the terminal, the position information of each teaching point is determined; Based on the posture information and position information of each teaching point, the robot is controlled to reproduce the teaching trajectory; The step of determining the position information of each teaching point based on the data collected by the accelerometer during the movement of the terminal includes: The acceleration collected by the accelerometer at the second teaching point is obtained, wherein the second teaching point is located after the first teaching point; The result of integrating the acceleration twice over the target time interval is determined to obtain the direction and initial distance of the second teaching point relative to the first teaching point, wherein the target time interval is the time interval between the second teaching point and the first teaching point; The location information of the second teaching point is determined based on the location information of the first teaching point and the direction and initial distance of the second teaching point relative to the first teaching point; The step of determining the position information of the second teaching point based on the position information of the first teaching point and the direction and initial distance of the second teaching point relative to the first teaching point includes: The terminal acquires a first image captured at the first teaching point, a second image captured at the second teaching point, and a third image captured at the third teaching point during the movement of the terminal. The third teaching point is located after the second teaching point, and the first image, the second image, and the third image all include the same target object. The extension direction of the target object is parallel to the direction of the terminal's movement. Based on the first image, the second image, and the initial distance between the second teaching point and the first teaching point, and using a preset algorithm, the first length of the target object in its extension direction is determined; Based on the second image, the third image, and the initial distance between the third teaching point and the second teaching point, and using a preset algorithm, the second length of the target object in its extension direction is determined; Determine the target length based on the first length and the second length; Based on the first image, the second image, and the target length, and using a preset algorithm, the target distance of the second teaching point relative to the first teaching point is determined; The location information of the second teaching point is determined based on the location information of the first teaching point, the direction of the second teaching point relative to the first teaching point, and the target distance.

2. The method according to claim 1, characterized in that, The step of determining the target length based on the first length and the second length includes: The target length is obtained by determining the average of the first length and the second length.

3. The method according to claim 1, characterized in that, The angular velocity sensor has a gravity direction detection function; the step of determining the attitude information of each teaching point on the teaching trajectory based at least on the data collected by the angular velocity sensor during the movement of the terminal includes: The attitude angles collected by the angular velocity sensor at each of the teaching points are determined as the attitude information of each teaching point.

4. The method according to claim 1, characterized in that, The angular velocity sensor does not have a gravity direction detection function; the terminal further includes a gravity sensor; The step of determining the attitude information of each teaching point on the teaching trajectory based at least on data collected by the angular velocity sensor during the movement of the terminal includes: For each teaching point, the attitude information of the teaching point is determined based on the data collected by the gravity sensor and the data collected by the angular velocity sensor corresponding to the teaching point.

5. The method according to claim 1, characterized in that, The terminal is a mobile phone.

6. An electronic device, characterized in that, The electronic device includes a processor, a memory, and a communication circuit. The processor is coupled to the memory and the communication circuit. The memory stores program data. The processor executes the program data in the memory to implement the steps of the method as described in any one of claims 1-5.

7. An electronic device, characterized in that, The electronic device, using the method of any one of claims 1-5, comprises: The acquisition module is used to acquire data collected by the angular velocity sensor and the acceleration sensor in the terminal during the process of the terminal moving along the teaching trajectory; The first determining module, connected to the acquiring module, is used to determine the attitude information of each teaching point on the teaching trajectory based at least on the data collected by the angular velocity sensor during the movement of the terminal. The second determining module, connected to the acquiring module, is used to determine the position information of each teaching point based on the data collected by the accelerometer during the movement of the terminal. The reproduction module, connected to the first determining module and the second determining module, is used to control the robot to reproduce the teaching trajectory based on the posture information and position information of each teaching point.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by a processor to implement the steps of the method as described in any one of claims 1-5.

Images