[0011] In order to make the objectives, technical solutions and advantages of the present invention clearer, the following further describes the present invention in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.
[0012] In the description of the present invention, it should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", The orientation or positional relationship indicated by "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, not It indicates or implies that the pointed device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention.
[0013] In the description of the present invention, "plurality" means two or more than two, unless specifically defined otherwise.
[0014] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and other terms should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection , Or integrated; it can be a mechanical connection, or it can be an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, it can be the internal communication of two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0015] The implementation of this embodiment will be described in detail below with reference to the specific drawings. For ease of description, a spatial coordinate system (x, y, z) is established, where the x-axis and the y-axis are in the horizontal plane and perpendicular to each other, and the z-axis is in the vertical direction.
[0016] This embodiment provides a deep learning tracking collection method based on target posture, which is used for teaching actions of learning targets, including the following steps:
[0017] 101. Select multiple reference points of the teaching action, respectively track the movement of each reference point, and record the teaching tracking data. The target in this embodiment may be the whole of humans, animals, or other mechanical devices, or a specific part, such as a human hand, a bird's wings, and the execution end of other robots. Specifically, in this embodiment, the calligraphy action of a person holding a brush to write a Chinese character is taken as an example. The brush is the target, and the movement of the brush itself during writing is a teaching action. The process of writing a Chinese character while the writer holds the brush , Track the writing action process of the brush from multiple directions, mark multiple reference points on the brush, track each reference point and record it as teaching tracking data. The teaching tracking data should include the movement track and time of the reference point axis. It is easy to understand that in this embodiment, the hand can also be selected as the target, in other embodiments.
[0018] 102. Analyze each teaching tracking data, and fit at least two functions through the relationship between movement and time: a posture function used to describe the change of the target posture over time, and a displacement function used to describe the change of the target position over time. In this step, the teaching tracking data obtained in step 101 is processed and analyzed. In this embodiment, as figure 1 As shown, three points A, B, and C are collected on the brush as reference points, where point B is the center point of the rotation circle of the brush during the writing process, and a certain interval time is set as the analysis unit, for example, t=0.5s, Then analyze the position change information of each reference point every 0.5s, and fit it into at least two functions: attitude function and displacement function. Among them, the change of the target's own posture in the course of the action can be described by the posture function, such as rotating a certain angle in the vertical direction. In the displacement function, the target is regarded as a mass point, and the displacement change of the target is described, such as moving from point A to point B, and then rising to point C. In other embodiments, the number of functions can also be increased, such as action functions: describe output signals at certain specific points in time to perform specified operations, such as welding, pressing, etc. at time t. It should be understood that if there is no posture change during the teaching action of the target, only the displacement changes, the posture function is fitted to a constant function assigned a value of 0, otherwise, there is only posture change and no displacement change in the whole process, and the displacement function is fitted. For a constant function assigned a value of 0, having at least two functions including an attitude function and a displacement function obviously includes these two situations. And record the position information at this time and the corresponding time point at the same time.
[0019] Such as figure 1 with figure 2 As shown, in this embodiment, since point B is the center point of the rotation circle, that is, if the displacement of the brush is ignored, point B is regarded as a static point during the writing process, so the change of point B over time can be used as a displacement function. Through the change of the position of point A in time t, and the relative between point A and point B
[0020] Distance (l 1 The length of ), you can calculate the swing angle, that is, the posture change. There are many specific calculation methods, for example, let the distance between point A and point B be l 1 , The distance between point B and point C is l 2 , The brush taken at interval t is simplified to t 1 And t 2 Two straight lines, overlap the two points B, calculate t 1 And t 2 The distance between points A on X 1 , X 1 And l 1 The angle α can be calculated by the cosine formula, and the change of the angle α with respect to time t is the posture function at that moment. Similarly, t 1 And t 2 The distance between the upper point C X 2 And l 2 , The angle β can be calculated by the cosine formula. The angle β should theoretically be equal to the angle α, which can be used as mutual verification data in the calculation.
[0021] Both the attitude function and the displacement function include the same variables: time. On the one hand, the two can be combined to describe the action of the target. On the other hand, the speed and acceleration at a specific position/time can be known through the increment of unit time. As a reference data for controlling the robot.
[0022] In the process of writing with a brush, the displacement function is used to record the movement of the pen in three coordinate directions in space with the change of time t. The changes in the coordinates on the x and y axes can be used to describe the rough stroke direction when writing text , Font size, writing range and other action data. The coordinate change on the z-axis can be approximated as a function of the thickness of the stroke. Taking the paper as the z coordinate 0 point, the closer the z coordinate is to 0, the higher the compression force of the pen tip, the thicker the stroke, and the greater the writing force at this time. Larger; the larger the z-axis coordinate, the smaller the compression force on the pen tip and the thinner the stroke. The part of the z-axis coordinate in the displacement function that exceeds the threshold indicates that the pen tip has left the paper at this time and is marked as an invalid writing operation, and is recorded as a displacement operation to record the position of the moving pen.
[0023] The posture function is used to record the changes over time t, and the pen itself rotates in the x, y, and z axes. The posture function can be used to describe the posture change of the penholder during the writing process, corresponding to the calligraphy, which can be understood as the posture change of using the pen tip.
[0024] 103. Generate a control program to enable the robot to implement teaching actions according to the posture function and displacement function. The robot can imitate the action according to the driver program and move according to the desired movement mode: the movement of the execution end over time follows the displacement function, and the movement of the execution end itself follows the attitude function during the movement according to the displacement function, thereby simulating the indication of the target. Teach movement.
[0025] It can be seen from the above process that the tracking and acquisition method based on the target posture deep learning provided in this embodiment first determines multiple reference points, and then tracks the movement of the reference points during the teaching action to complete the acquisition, teaching and tracking. The data, and the time variable form two functions to describe the teaching action process of the target. Since the attitude function only records the attitude change of the target itself with respect to time, the displacement function regards the target as a mass point and only records the position change of the target with respect to time. Simplified movement data. The inverse solution is fitted into two functions, and the control program is generated according to the two functions. The robot can run the control program to simulate the operation process of the target. Due to the simplified action data, the amount of calculation for taking points when imitating more complex teaching actions is reduced, and the teaching actions can be imitated with a higher degree of reduction, and the actions are simplified without human involvement in judgment, making the imitation learning The process requires low manual participation and high degree of intelligence.
[0026] Preferably, such as figure 1 As shown, after step 103, the following steps are further included:
[0027] 104. Drive the robot to imitate actions according to the control program.
[0028] 105. Track the process of imitation from multiple directions and record imitation tracking data.
[0029] 106. Compare the imitation tracking data and teaching tracking data, and modify the control program.
[0030] Since the generated control program is only based on data collection and automatic calculation, the actions after execution may not be able to fully meet the requirements of imitation. Therefore, the control program is tested in step 104, and the imitation tracking is recorded according to step 101 during execution. Then compare the simulation tracking data with the teaching tracking data, and then modify the control program to form a closed control loop, that is, the robot learning process.
[0031] There are many specific comparison methods, such as repeating the process from step 101 to step 103, using the execution end of the robot as the collection target of the teaching action, and generating a new displacement function, posture function, and original motion data. Comparing the displacement function and attitude function to find out whether there is a deviation beyond the threshold; or directly through the image comparison method, imitating the image information and the teaching image, adjusting the transparency and superimposing, and comparing the error on the image to judge the similarity degree. If the error is found to exceed the threshold, determine the correction direction and size, and then reverse the correction control program.
[0032] The above steps 103 to 106 can be repeated. After multiple trial runs, collection comparisons, and multiple iterative learning, the action error converges to within the threshold, and the learning process is judged to be completed.
[0033] This embodiment also provides a learning system for controlling the robot to learn and teach actions. The robot includes an action tracking part, a data analysis part, a drive control part, and an execution end. Among them, the motion tracking part has multiple trackers, which can continuously track the reference point and record the teaching tracking data during the teaching motion. The data analysis part receives the teaching tracking data and analyzes it to obtain the motion function of the teaching motion. That is, the displacement function and the posture function above, the drive control part generates a control program after receiving the motion function, and controls the execution end to perform an imitating action.
[0034] The learning system in this embodiment can generate motion functions by collecting teaching actions, self-deconstructing and analyzing, and then generating a control program. After running the control program, the execution end performs imitating actions to imitate the teaching actions. Due to the simplified action data, the amount of calculation for taking points when imitating more complex teaching actions is reduced, and the teaching actions can be imitated with a higher degree of reduction, and the actions are simplified without human involvement in judgment, making the imitation learning The process requires low manual participation and high degree of intelligence.
[0035] Preferably, the tracker is a plurality of tracking cameras, and each tracking camera selects a corresponding reference point for rotation tracking, and its rotation information is recorded as teaching tracking data. The tracking camera is a special camera that can rotate and adjust its own shooting angle. Through control programs and image recognition technology, it can capture the movement of the target so that the target is always within the shooting range. In this embodiment, the reference point is the target that the tracking camera needs to capture. As the teaching action progresses, the tracking camera rotates with the reference point, and the rotation angle and speed are the rotation information, which is recorded as the teaching tracking The data can be restored to the displacement data of the reference point according to the distance between the tracking camera and the target.
[0036] Preferably, the motion tracking part has an effective collection space, and each tracker camera is distributed around the space around the effective collection space, and they are all aligned in the effective collection space. The teaching action should be carried out in the effective collection space to facilitate tracking and obtaining data
[0037] Further, a marker is provided on the target, which is convenient for tracking camera recognition. Markers can be painted dots with special colors, pasted patterns with special shapes, and installed parts capable of emitting special light and special electromagnetic signals. In other embodiments, the reference point can also only be used as a virtual concept, which is processed as digital information in the system, and there is no point actually marked on the target.
[0038] Preferably, the actions of the execution end are always located in the effective collection space. In practical applications, after the teaching action is completed, the execution end can be moved to the effective collection space, or multiple effective collection spaces can be set to perform the teaching action and the imitation action of the execution end respectively.
[0039] Preferably, the learning system further includes a learning part; it is used to modify the control program according to the action of the execution end, and the learning principle is the same as the aforementioned detection, modification, and re-operation process, and will not be repeated.
[0040] In this embodiment, a computer-readable storage medium is also provided, and the computer-readable storage medium stores a computer program. When the computer program is executed by the processor, the steps of the above-mentioned target posture-based deep learning tracking acquisition method are realized.
[0041] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modification, equivalent replacement and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention. Inside.
