Robot guidance method, device, electronic device and storage medium
By loading image acquisition equipment on a multi-axis robot, analyzing and correcting image data to obtain the positional relationship of the target to be operated, the problem of low repositioning accuracy of mobile multi-axis robots is solved, and higher position accuracy and work efficiency are achieved.
Patent Information
- Application Number
- CN202211714189.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-27
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-12-27
AI Technical Summary
The repositioning accuracy of the mobile multi-axis robot is not high, resulting in the inability to accurately reach the target position, affecting work efficiency.
By loading an image acquisition device on the end effector of the multi-axis robot, collecting image data of the target to be operated, analyzing the data to obtain description information of the relative position relationship, and performing error correction, and then performing position guidance to make the end of the multi-axis robot The actuator reaches the operating area where the target to be operated.
The repositioning accuracy of the mobile multi-axis robot is improved, ensuring that the end effector can accurately reach the target position, and improving work efficiency.
Smart Images

Figure CN116038698B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of robot control technology, and in particular to a robot guiding method, device, electronic equipment and storage medium. Background Art
[0002] Multi-axis robots are widely used in industrial production and operation and maintenance. For example, multi-axis robots are used to process products on assembly lines. In order to meet the needs of practical applications, multi-axis robots are usually mounted on mobile platforms to form mobile multi-axis robots, so that the multi-axis robots can work in a mobile manner. For example, the multi-axis robot is mounted on a mobile platform to move the multi-axis robot to the position to be operated, so as to pick up the target. Among them, the mobile platform can be AGV (Automated Guided Vehicle), AMR (Autonomous Mobile Robot), etc., and the mobile platform can move based on laser radar or other navigation methods, such as structured light method, multi-eye vision method, etc.
[0003] However, in actual applications, there will be errors in navigation and in the mechanical linkage between the multi-axis robot and the mobile platform. In addition, the parking accuracy of the chassis motor of the mobile platform itself is not high, and there will be delays in the linkage between the navigation module and the motor. Therefore, the repositioning accuracy of the mobile platform when working is often not high. Therefore, the repositioning accuracy of the mobile multi-axis robot is often not high, and the target position reached by the mobile multi-axis robot may be inaccurate, which may cause the mobile multi-axis robot to be unable to work. Summary of the invention
[0004] The purpose of the embodiments of the present invention is to provide a robot guidance method, device, electronic device and storage medium to improve the repositioning accuracy of a mobile multi-axis robot during operation. The specific technical solution is as follows:
[0005] In a first aspect, an embodiment of the present invention provides a robot guidance method, which is applied to a control device, wherein the control device is communicatively connected to a mobile multi-axis robot; the mobile multi-axis robot comprises: a multi-axis robot and a mobile platform, wherein the multi-axis robot is provided with a mechanical arm of an end effector and an image acquisition device is mounted thereon; the method comprises:
[0006] Controlling the mobile platform to move to a predetermined reference position; wherein the reference position is the position of the target to be operated obtained after preliminary positioning of the target to be operated;
[0007] After the mobile platform reaches the reference position, using the image acquisition device to acquire image data of the target to be operated;
[0008] Performing data analysis on the image data to obtain specified description information; wherein the specified description information is description information used to characterize the relative position relationship between the target to be operated and the multi-axis robot;
[0009] Performing error correction on the specified description information;
[0010] According to the corrected designated description information, the multi-axis robot is positionally guided so that the end effector of the guided multi-axis robot reaches within the operation area for the target to be operated.
[0011] Optionally, the specified description information includes: angular offset information, depth information and line offset information; wherein the angular offset information is the angular offset information from the image plane of the image acquisition device to the target to be operated in the base coordinate system, the depth information is the vertical distance from the end effector to the target to be operated in the base coordinate system, and the line offset information is the height distance and lateral distance from the end effector to the target to be operated in the base coordinate system, and the base coordinate system is a coordinate system centered on the multi-axis robot;
[0012] Performing error correction on the specified description information, comprising:
[0013] According to a predetermined correction order, for each type of information among the angular offset information, the depth information and the line offset information, based on an error correction method corresponding to the type of information, error correction is performed on the type of information;
[0014] The predetermined correction order is a correction order regarding the angular offset information, the depth information and the line offset information.
[0015] Optionally, the predetermined correction sequence includes:
[0016] The error correction is performed on the angular offset information, the depth information and the line offset information in sequence.
[0017] Optionally, the error correction method corresponding to the angular offset information includes:
[0018] Acquire the offset direction of the image plane relative to the target to be operated represented by the angular offset information;
[0019] Obtaining an offset angle represented by the angular offset information as a current offset angle;
[0020] The current offset angle is offset in the opposite direction of the offset direction by a first step length; wherein the first step length is an angle value with respect to the offset angle, and an initial value of the first step length is a preset offset angle;
[0021] Determining whether the offset direction changes after the first offset length;
[0022] If the offset direction has not changed, the offset angle obtained after offsetting the first step is used as the new current offset angle, and the process returns to the step of offsetting the current offset angle in the opposite direction of the offset direction by the first step until the offset direction changes;
[0023] If the offset direction changes, the offset angle obtained after the first offset length is used as the new current offset angle, and the first offset length is proportionally reduced to obtain a new first offset length, and the step of offsetting the current offset angle by the first offset length in the opposite direction of the offset direction is returned until the first offset length is reduced to less than the angle execution accuracy; wherein the angle execution accuracy represents the accuracy of the angle offset information within a reasonable error range;
[0024] The angular offset information of the offset angle obtained by the last offset is determined to obtain the corrected angular offset information.
[0025] Optionally, the error correction method corresponding to the depth information includes:
[0026] Determine two reference image areas adjacent to a designated image area and reference depth information corresponding to the two reference image areas from a pre-collected mapping table of various reference image areas corresponding to the target to be operated, and obtain a first set of data; the designated image area is an image area of the target to be operated in the image data;
[0027] Determine from the mapping table two other reference image areas other than the reference image area in the first set of data, and reference depth information corresponding to the other two reference areas, to obtain a second set of data;
[0028] The constant parameters in the first linear equation are solved by substituting the first set of data and the second set of data into the first linear equation for solving the constant parameters; wherein the first linear equation is a quadratic equation with the image area of any target as the independent variable and the depth information of the any target as the dependent variable, and the equation used when substituting the first set of data and the second set of data into the first linear equation for solving the parameters is as follows:
[0029]
[0030]
[0031]
[0032] Among them, s1 、s 2 are the areas of the two reference images in the first set of data, s 0 、s 3 are the areas of the two reference images in the second set of data, and p 1 is the reference image area s in the mapping table 1 The corresponding reference depth information, p 2 is the reference image area s in the mapping table 2 The corresponding reference depth information, p 3 is the reference image area s in the mapping table 3 The corresponding reference depth information, p 4 is the reference image area s in the mapping table 4 corresponding depth information, t is the distance from the normal plane of the end effector to the normal plane of the aperture, a, b and c are constant parameters to be solved for the first linear equation;
[0033] Substituting the designated image area into the first linear equation in which the constant parameter is solved to obtain a function value;
[0034] The depth information is corrected using the function value to obtain corrected depth information.
[0035] Optionally, the error correction method corresponding to the line offset information includes:
[0036] estimating the height error and lateral error from the end effector to the target to be operated;
[0037] According to the height error and the lateral error, the line offset information is preliminarily corrected to obtain preliminarily corrected line offset information; wherein the preliminarily corrected line offset information includes a preliminarily corrected height distance and a lateral distance;
[0038] Obtaining the offset direction of the height distance and the lateral distance represented by the line offset information after the preliminary correction;
[0039] Obtaining the offset distance of the height distance and the lateral distance represented by the line offset information after the preliminary correction as the current offset distance;
[0040] The current offset distance is offset in the opposite direction of the offset direction by a second step length; wherein the second step length is a length value related to the offset distance, and an initial value of the second step length is a preset offset distance;
[0041] Determine whether the offset direction changes after the second step length is offset;
[0042] If the offset direction has not changed, the offset distance obtained after the second step length is offset is used as the new current offset distance, and the process returns to the step of offsetting the current offset distance in the opposite direction of the offset direction by the second step length until the offset direction changes;
[0043] If the offset direction changes, the offset distance obtained after the second step length is offset is used as the new current offset distance, and the second step length is proportionally reduced to obtain a new second step length, and the step of offsetting the current offset distance by the second step length in the opposite direction of the offset direction is returned to until the second step length is reduced to less than the distance execution accuracy; wherein the distance execution accuracy represents the accuracy of the line offset information within a reasonable error range;
[0044] The line offset information of the offset distance obtained by the last offset is determined to obtain the corrected line offset information.
[0045] Optionally, the estimating the height error and lateral error from the end effector to the target to be operated includes:
[0046] After guiding the position of the multi-axis robot according to the corrected depth information, the image data of the target to be operated is collected again by using the image acquisition device to obtain auxiliary image data;
[0047] Performing data analysis on the auxiliary image data to obtain depth information;
[0048] Calculating a scaling ratio between the obtained depth information and the corrected depth information to obtain size information;
[0049] The height error and lateral error from the end effector to the target to be operated are estimated by using the size information and the line offset information.
[0050] In a second aspect, an embodiment of the present invention provides a robot guiding device, which is applied to a control device, wherein the control device is communicatively connected to a mobile multi-axis robot; the mobile multi-axis robot comprises: a multi-axis robot and a mobile platform, wherein an end effector of the multi-axis robot is equipped with an image acquisition device; the device comprises:
[0051] A control module, used to control the mobile platform to move to a predetermined reference position; wherein the reference position is a position of the target to be operated obtained after preliminary positioning of the target to be operated;
[0052] A collection module, used for collecting image data of the target to be operated by using the image collection device after the mobile platform reaches the reference position;
[0053] An analysis module, used for performing data analysis on the image data to obtain specified description information; wherein the specified description information is description information used to characterize the relative position relationship between the target to be operated and the multi-axis robot;
[0054] An error correction module, used for performing error correction on the specified description information;
[0055] The guiding module is used to guide the position of the multi-axis robot according to the corrected specified description information, so that the end effector of the guided multi-axis robot reaches the operating area for the target to be operated.
[0056] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0057] Memory, used to store computer programs;
[0058] The processor is used to implement any of the above-mentioned robot guidance method steps when executing the program stored in the memory.
[0059] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps of any robot guidance are implemented.
[0060] Beneficial effects of the embodiments of the present invention:
[0061] The robot guidance method provided by the embodiment of the present invention can control the mobile platform to move to a predetermined reference position; after the mobile platform reaches the reference position, the image data of the target to be operated is collected by an image acquisition device; the image data is analyzed to obtain the designated description information; the designated description information is corrected for errors; and the multi-axis robot is guided according to the corrected designated description information, so that the end effector of the guided multi-axis robot reaches the operation area for the target to be operated. It can be seen that compared with the prior art, the present solution additionally configures an image acquisition device for the multi-axis robot, and obtains the description information representing the positional relationship between the target to be operated and the multi-axis robot by analyzing the image data of the target to be operated collected by the image acquisition device, that is, the designated description information, and corrects the designated description information, so as to use the corrected designated description information to guide the multi-axis robot, so that when the repositioning accuracy of the mobile platform is not high, the repositioning accuracy of the multi-axis robot can be improved, thereby improving the repositioning accuracy of the mobile multi-axis robot when working.
[0062] Of course, any product or method of the present invention does not necessarily need to achieve all of the above advantages at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.
[0064] Figure 1 A schematic diagram of a flow chart of a robot guidance method provided by an embodiment of the present invention;
[0065] Figure 2 A schematic flow chart of another robot guidance method provided by an embodiment of the present invention;
[0066] Figure 3 A schematic diagram of the structure of a multi-axis robot provided by an embodiment of the present invention;
[0067] Figure 4 A schematic diagram of a coordinate system provided by an embodiment of the present invention;
[0068] FIG5( a) is a schematic diagram of the principle of pinhole imaging provided by an embodiment of the present invention;
[0069] FIG5( b ) is a schematic diagram of another principle of pinhole imaging provided by an embodiment of the present invention;
[0070] Figure 6 A schematic structural diagram of a robot guiding device provided by an embodiment of the present invention;
[0071] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0072] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field based on the present invention belong to the scope of protection of the present invention.
[0073] Below, the professional terms involved in the embodiments of the present invention are first introduced:
[0074] Multi-axis robot: also known as single-axis manipulator, industrial robot arm, etc., is an automatic control, reprogrammable, multi-degree-of-freedom, multi-purpose manipulator. Its working behavior is mainly to complete the given action by synchronous rotation of multiple axes.
[0075] Mobile platform: A platform that carries a multi-axis robot for movement. It can be an automated guided vehicle (AGV) or an autonomous mobile robot (AMR). In actual use, its specific form can be a small car.
[0076] Mobile multi-axis robot: A combination of a multi-axis robot and a mobile platform. The mobile platform can carry the multi-axis robot, so that the multi-axis robot can work in a mobile manner.
[0077] World coordinate system: The world coordinate system is the absolute coordinate system of the system. Before the user coordinate system is established, the coordinates of all points on the screen are determined by the origin of the coordinate system.
[0078] Base coordinate system: A coordinate system centered on a multi-axis robot, which can also be understood as a user coordinate system.
[0079] Coaxial camera: A camera that is rigidly connected to the end effector of a multi-axis robot, and the optical axis of the coaxial camera is parallel to the final axis of the robot arm with the end effector. In the macroscopic sense, the coaxial camera and the end effector can be regarded as a whole. The optical axis of the coaxial camera is the line through the center of the camera, that is, the line through the center of the camera; the final axis of the robot arm with the end effector is the line through the center of the end of the robot arm.
[0080] It should be noted that this solution has certain requirements for the operating environment and equipment. The following is a brief introduction:
[0081] Environmental requirements: The ground at the operation point must be flat.
[0082] Requirements for the mobile platform: The mobile platform can be repositioned roughly and accurately in the plane area, allowing the plane linear error to not exceed ±10cm and the angular error to not exceed ±20°; among which, the linear error refers to the plane height distance error and the lateral distance error, and the angular error refers to the offset angle error of the mobile platform relative to the target to be operated.
[0083] Requirements for multi-axis robots: repositioning error shall not exceed ±0.1mm.
[0084] Requirements for coaxial cameras: resolution not less than 2560*1440, no lens distortion, and if the ambient lighting is uncontrollable, a fill light device is required; lens distortion can be understood as the nonlinear mapping relationship between the spatial coordinates of an object and the image coordinates.
[0085] It should be noted that this solution can configure a control device to guide the mobile platform and the multi-axis robot. The control device is communicatively connected to the mobile platform and the multi-axis robot. It can also be understood that the control device is communicatively connected to the mobile multi-axis robot. Then, for the mobile platform part, as long as the control device can send coordinates to the mobile platform and the mobile platform can move to the position indicated by the above coordinates; the repositioning accuracy of the multi-axis robot must be more precise. Depending on the type of multi-axis robot, the coordinate protocol is also different. This solution does not specifically limit the specific type of multi-axis robot.
[0086] In addition, the image acquisition device is mounted on the mechanical arm of the multi-axis robot with the end effector, and the end effector is included in the field of view of the image acquisition device. Figure 3 The exemplary structure given is an exemplary introduction to the structure of a multi-axis robot:
[0087] like Figure 3 As shown, the multi-axis robot may be composed of a plurality of robotic arms, an image acquisition device 310 and an end effector 320 . Compared with other multi-axis robots, in this solution, the image acquisition device 310 is mounted on the robotic arm provided with the end effector 320 .
[0088] The image acquisition device 310 is used to acquire images of the target to be operated. It should be noted that the specific form of the image acquisition device can be a coaxial camera, and this solution does not specifically limit the specific form of the image acquisition device.
[0089] The end effector 320 is used to operate the target.
[0090] In addition, compared with the prior art, this embodiment mounts an image acquisition device on a robotic arm provided with an end effector, and does not require excessive deployment of the multi-axis robot, thereby ultimately achieving improved repositioning accuracy of the mobile multi-axis robot. Therefore, this solution has low cost and is easy to deploy.
[0091] After introducing the structure of the multi-axis robot, the coordinate system proposed in this embodiment is introduced below. Figure 4 As shown:
[0092] This embodiment establishes a coordinate system based on the left-hand system, in which the left thumb points to the positive direction of the x-axis and the index finger points to the positive direction of the y-axis. If the middle finger can point to the positive direction of the z-axis, this coordinate system is called a left-hand rectangular coordinate system.
[0093] x, y, and z are used to describe position, rx, ry, and rz are used to describe posture, and the combination of position and posture is called a coordinate system; among them, posture is a more common term, mostly used in the Euler coordinate system, and will not be elaborated on here.
[0094] After introducing the coordinate system, the following is a brief introduction to the preprocessing method before error correction of the specified description information:
[0095] Before guiding the robot, it is necessary to detect the appearance of the target to be operated. Detection methods such as polygon detection, Hough line detection and Hough circle detection can be used. The advantages of the above detection methods are fast speed, explainability and reliable results. For targets to be operated with complex appearance, neural network algorithms can be used for detection, and this solution does not make specific restrictions on this.
[0096] It should be noted that in actual use, by detecting the appearance of the target to be operated, it is possible to avoid the situation where the multi-axis robot reaches the operating area of the target to be operated but cannot operate correctly. For example, if the multi-axis robot is to be controlled to guide it into the operating area of switch a, the appearance shape of switch a is to be detected in advance. If the appearance shape of switch a is detected to be a circular switch, the end effector of the multi-axis robot can be aligned with the center of switch a during the error correction process.
[0097] In addition, this part is not within the scope of protection of this plan, so no further introduction will be given.
[0098] In order to improve the repositioning accuracy of a mobile multi-axis robot during operation, an embodiment of the present invention provides a robot guiding method, device, electronic device and storage medium.
[0099] First, a robot guiding method provided by an embodiment of the present invention is introduced below.
[0100] A robot guidance method provided in an embodiment of the present invention can be applied to a control device. In a specific application, the control device can be a terminal device or a server. Exemplarily, the control device can be an edge computing platform, such as a mobile phone, a personal computer PC or a server, etc. The embodiment of the present invention does not limit the specific form of the control device.
[0101] Furthermore, the control device is communicatively connected with a mobile multi-axis robot; the mobile multi-axis robot comprises: a multi-axis robot and a mobile platform, and the multi-axis robot is provided with a mechanical arm of an end effector on which an image acquisition device is mounted.
[0102] A robot guidance method may include:
[0103] Controlling the mobile platform to move to a predetermined reference position; wherein the reference position is the position of the target to be operated obtained after preliminary positioning of the target to be operated;
[0104] After the mobile platform reaches the reference position, using the image acquisition device to acquire image data of the target to be operated;
[0105] Performing data analysis on the image data to obtain specified description information; wherein the specified description information is description information used to characterize the relative position relationship between the target to be operated and the multi-axis robot;
[0106] Performing error correction on the specified description information;
[0107] According to the corrected designated description information, the multi-axis robot is positionally guided so that the end effector of the guided multi-axis robot reaches within the operation area for the target to be operated.
[0108] The robot guidance method provided by the embodiment of the present invention can control the mobile platform to move to a predetermined reference position; after the mobile platform reaches the reference position, the image data of the target to be operated is collected by an image acquisition device; the image data is analyzed to obtain the designated description information; the designated description information is corrected for errors; and the multi-axis robot is guided according to the corrected designated description information, so that the end effector of the guided multi-axis robot reaches the operation area for the target to be operated. It can be seen that compared with the prior art, the present solution additionally configures an image acquisition device for the multi-axis robot, and obtains the description information representing the positional relationship between the target to be operated and the multi-axis robot by analyzing the image data of the target to be operated collected by the image acquisition device, that is, the designated description information, and corrects the designated description information, so as to use the corrected designated description information to guide the multi-axis robot, so that when the repositioning accuracy of the mobile platform is not high, the repositioning accuracy of the multi-axis robot can be improved, thereby improving the repositioning accuracy of the mobile multi-axis robot when working.
[0109] A robot guiding method provided by an embodiment of the present invention is introduced below in conjunction with the accompanying drawings.
[0110] like Figure 1 As shown, an embodiment of the present invention provides a robot guidance method, which is applied to a control device, wherein the control device is communicatively connected with a mobile multi-axis robot; the mobile multi-axis robot comprises: a multi-axis robot and a mobile platform, wherein the multi-axis robot is provided with a mechanical arm of an end effector and an image acquisition device is mounted thereon; and the method may include steps S101-S105:
[0111] S101, controlling the mobile platform to move to a predetermined reference position;
[0112] The reference position is the position of the target to be operated obtained after preliminary positioning of the target to be operated;
[0113] It is understandable that the mobile platform can be roughly repositioned in the plane area to obtain a reference position. Since the mobile platform has errors such as laser radar errors and mechanical linkage errors, the repositioning accuracy of the mobile platform is not high. Since the reference position is obtained by preliminarily positioning the position of the target to be operated, the distance from the reference position to the target to be operated must be within a reasonable error. In addition, the target to be operated can be a switch or an object, etc., which can change according to different usage scenarios. The embodiments of the present invention do not specifically limit this. For example, in the usage scenario of an assembly line, the mobile platform is controlled to move to a predetermined reference position, and the reference position is the position of the air switch a obtained after the air switch a is preliminarily positioned.
[0114] Among them, the mobile platform is mostly a cart in actual use. For example, the automatic guided transport cart is controlled to move to a predetermined reference position.
[0115] It should be noted that when the control device controls the mobile platform to move to the reference position, it needs to send the coordinates of the reference position to the mobile platform. Different mobile platforms have different coordinate protocols. The control device sends the coordinates to the mobile platform according to the coordinate protocol. This part is not within the scope of protection of this solution, so no further elaboration will be given.
[0116] S102, after the mobile platform reaches the reference position, using the image acquisition device to acquire image data of the target to be operated;
[0117] It should be noted that after the mobile platform moves to the reference position, the image acquisition device can collect image data of the target to be operated, so the collected image data includes the target to be operated. Since the pixels of the coaxial camera have been limited as mentioned above, the clarity of the image data can be guaranteed. In addition, the image acquisition device is rigidly connected to the mechanical arm of the end effector, and from a macroscopic point of view, it can be regarded as a whole, so the positional relationship between the two is not limited.
[0118] Exemplarily, after the mobile platform A reaches the reference position, the image data of the switch b is collected using a coaxial camera.
[0119] S103, performing data analysis on the image data to obtain designated description information;
[0120] The specified description information is description information used to characterize the relative position relationship between the target to be operated and the multi-axis robot.
[0121] Exemplarily, the specified description information may include: the offset angle and offset direction from the image plane of the image acquisition device to the target to be operated in the base coordinate system, the vertical distance from the end effector to the target to be operated in the base coordinate system, and / or the height distance and lateral distance from the end effector to the target to be operated in the base coordinate system. Of course, the specified description information is not limited to the above three, and one or two of them can also be selected.
[0122] It should be noted that there are many methods for performing data analysis on image data, and most of them are existing technologies. Therefore, the process of performing data analysis on image data is not within the protection scope of the present invention, so the embodiments of the present invention do not make specific limitations on this.
[0123] S104, performing error correction on the specified description information;
[0124] It can be understood that by performing error correction on the designated description information, corrected designated description information can be obtained, and the multi-axis robot can be guided according to the description information, thereby improving the repositioning accuracy of the multi-axis robot.
[0125] It should be noted that, for ease of understanding, the process of performing error correction on the specified description information is introduced in other embodiments, so it will not be described in detail here.
[0126] S105, performing position guidance on the multi-axis robot according to the corrected designated description information, so that the guided end effector of the multi-axis robot reaches an operation area for the target to be operated;
[0127] Herein, guiding the position of the multi-axis robot according to the corrected designated description information can also be understood as controlling the multi-axis robot to reposition according to the corrected designated description information.
[0128] It should be noted that after being guided, the end effector of the multi-axis robot can reach the operating area of the target to be operated, thereby finally operating the target to be operated.
[0129] Exemplarily, in the scenario of an assembly line, the multi-axis robot A is positionally guided according to the corrected designated description information, so that the end effector of the guided multi-axis robot A reaches within the operating area for the switch b.
[0130] The robot guidance method provided by the embodiment of the present invention can control the mobile platform to move to a predetermined reference position; after the mobile platform reaches the reference position, the image data of the target to be operated is collected by an image acquisition device; the image data is analyzed to obtain the designated description information; the designated description information is corrected for errors; and the multi-axis robot is guided according to the corrected designated description information, so that the end effector of the guided multi-axis robot reaches the operation area for the target to be operated. It can be seen that compared with the prior art, the present solution additionally configures an image acquisition device for the multi-axis robot, and obtains the description information representing the positional relationship between the target to be operated and the multi-axis robot by analyzing the image data of the target to be operated collected by the image acquisition device, that is, the designated description information, and corrects the designated description information, so as to use the corrected designated description information to guide the multi-axis robot, so that when the repositioning accuracy of the mobile platform is not high, the repositioning accuracy of the multi-axis robot can be improved, thereby improving the repositioning accuracy of the mobile multi-axis robot when working.
[0131] like Figure 2 As shown, the embodiment of the present invention provides another robot guidance method, which may include steps S201-S205:
[0132] S201, controlling the mobile platform to move to a predetermined reference position;
[0133] S202, after the mobile platform reaches the reference position, using the image acquisition device to acquire image data of the target to be operated;
[0134] It should be noted that steps S201, S202 and S205 are the same as the above steps S101, S102 and S105, and thus will not be described in detail here.
[0135] S203, performing data analysis on the image data to obtain designated description information; wherein the designated description information includes: angle offset information, depth information and line offset information;
[0136] The angular offset information is the angular offset information from the image plane of the image acquisition device to the target to be operated in the base coordinate system, the depth information is the vertical distance from the end effector to the target to be operated in the base coordinate system, and the line offset information is the height distance and lateral distance from the end effector to the target to be operated in the base coordinate system. The base coordinate system is a coordinate system centered on the multi-axis robot.
[0137] It can be understood that the angular offset information is the angular offset information from the image plane of the image acquisition device to the target to be operated in the base coordinate system. After correction, the angular offset information from the image plane of the image acquisition device to the target to be operated should be 0°, that is, the image plane of the image acquisition device and the target to be operated should be parallel; the depth information is the vertical distance from the end effector to the target to be operated in the base coordinate system after correcting the angular offset information, which can also be understood as the vertical distance from the end effector to the target to be operated in a bird's-eye view; the line offset information is the height distance and lateral distance from the end effector to the target to be operated in the base coordinate system, which can also be understood as the height distance and lateral distance from the end effector to the target to be operated in the front view of the multi-axis robot after the depth information is corrected.
[0138] S204, performing error correction on each type of information among the angular offset information, the depth information, and the line offset information according to a predetermined correction order and based on an error correction method corresponding to the type of information;
[0139] The predetermined correction order is a correction order regarding the angular offset information, the depth information and the line offset information.
[0140] It should be noted that the angular offset information, depth information and line offset information all have their corresponding error correction methods, and the angular offset information, depth information and line offset information are error corrected according to a predetermined correction order and the corresponding error correction method.
[0141] Optionally, in one implementation, the predetermined correction sequence includes:
[0142] The error correction is performed on the angular offset information, the depth information and the line offset information in sequence.
[0143] It is understandable that the angular offset information is corrected first, and then the depth information is corrected after the angular offset information is corrected. After the depth information is corrected, the line offset information is corrected last. It should be noted that the predetermined correction order is certain and cannot be changed.
[0144] S205, performing position guidance on the multi-axis robot according to the corrected designated description information, so that the guided end effector of the multi-axis robot reaches an operation area for the target to be operated;
[0145] The robot guidance method provided by the embodiment of the present invention can control the mobile platform to move to a predetermined reference position; after the mobile platform reaches the reference position, the image data of the target to be operated is collected by using an image acquisition device; the image data is analyzed to obtain designated description information; according to a predetermined correction order, for each type of information in the angular offset information, depth information and line offset information, based on the error correction method corresponding to the type of information, the error correction of the type of information is performed; according to the corrected designated description information, the multi-axis robot is positionally guided so that the end effector of the guided multi-axis robot reaches the operation area for the target to be operated. Then, according to the correction order, for each type of information, based on the error correction method corresponding to the type of information, the error correction of the type of information is performed. It can be seen that the present scheme can perform error correction on each type of information in the designated description information, thereby improving the repositioning accuracy of the multi-axis robot, and thus improving the repositioning accuracy of the mobile multi-axis robot when working.
[0146] Based on the above robot guidance method, the error correction method corresponding to each specified description information will be introduced in a predetermined correction order as follows:
[0147] Optionally, in one implementation method, the error correction method corresponding to the angular offset information includes steps A1-A7:
[0148] Step A1, obtaining the offset direction of the image plane relative to the target to be operated represented by the angular offset information;
[0149] The offset direction of the image plane relative to the target to be operated can be characterized by angular offset information.
[0150] It can be understood that the offset direction can be leftward or rightward.
[0151] It is understandable that, under normal circumstances, the offset direction of the image plane relative to the target to be operated is 0°, that is, the two are parallel, but due to the low repositioning accuracy of the mobile platform introduced above, the image plane may have an offset direction relative to the target to be operated.
[0152] In addition, the offset direction of the image plane relative to the target to be operated represented by the angular offset information can also be obtained by calculation. For example, the normal plane of the end pointing direction of a certain image plane is obtained, and the plane is translated to the left or right, that is, the image acquisition device is translated to the left or right, and the imaging length of the target to be operated before and after the translation is calculated respectively, so as to obtain whether the current end is pointing to the left or right.
[0153] In order to better understand the offset direction of the image plane relative to the target to be operated, it can be introduced in conjunction with the accompanying drawings, as shown in Figure 5(a) and Figure 5(b):
[0154] 510 in the figure is the target to be operated from a top-down perspective, 520 is the aperture, and 530 is the image plane; wherein, the aperture is a photographic term, and the aperture is a device for controlling the amount of light passing through the lens and entering the photosensitive surface in the body, and it is usually inside the lens.
[0155] FIG5(a) shows the offset direction of the image plane relative to the target to be operated under normal circumstances, and the two are parallel; FIG5(b) shows the offset direction of the image plane relative to the target to be operated when an error occurs. As shown in FIG5(b), the offset direction is offset to the right.
[0156] It should be noted that, for ease of understanding, the image plane and the target to be operated can be regarded as pinhole imaging, and the aperture can be regarded as the pinhole in the pinhole imaging.
[0157] Step A2, obtaining the offset angle represented by the angle offset information as the current offset angle;
[0158] It should be noted that the angular offset information can not only represent the offset direction of the image plane relative to the target to be operated, but also represent the offset angle, and the above offset angle is used as the current offset angle.
[0159] It is understandable that the offset angle may also be referred to as an initial angle error, and the embodiment of the present invention does not specifically limit the specific naming.
[0160] For example, the angular offset information may indicate that the image plane is offset 30° to the right relative to the target to be operated.
[0161] Step A3, shifting the current offset angle in the opposite direction of the offset direction by a first step length;
[0162] Wherein, the first step length is an angle value about an offset angle, and an initial value of the first step length is a preset offset angle;
[0163] It is understandable that the initial value of the first step length is a preset offset angle, which may be changed according to the current offset angle, and the embodiment of the present invention does not specifically limit this.
[0164] For example, if the current offset angle is 20°, the offset direction is to the right, and the preset first step length is 5°, then the first step length may be offset in the opposite direction of the offset direction, that is, 5° to the left.
[0165] Step A4, determining whether the offset direction changes after the first offset length is changed;
[0166] It can be understood that after the first step of offsetting, two situations may occur: the offset direction changes or the offset direction does not change.
[0167] For example, if the current offset angle is 20°, the offset direction is to the right, and the pre-set first step length is 5°, then after the first step length is shifted to the left, the offset direction is still to the right, so the offset direction has not changed; if the current offset angle is 3°, the offset direction is to the right, and the pre-set first step length is 5°, then after the first step length is shifted to the left, the offset direction is to the left, so the offset direction has changed.
[0168] Step A5, if the offset direction has not changed, the offset angle obtained after offsetting the first step is used as the new current offset angle, and the process returns to the step of offsetting the current offset angle in the opposite direction of the offset direction by the first step, until the offset direction changes;
[0169] It can be understood that the above steps are to introduce the situation where the offset direction has not changed. If the offset direction has not changed, the offset angle obtained after the first offset step is used as the new current offset angle, and the above steps are returned until the offset direction changes.
[0170] For example, if the current offset angle is 20°, the offset direction is to the right, and the pre-set first step length is 5°, then after the first step length to the left, the offset direction is still to the right and has not changed. At this time, the offset angle is 15°. 15° is used as the new current offset angle, and 15° is further offset to the left by 5°, and 10° is used as the new current offset angle until the offset direction changes.
[0171] Step A6, if the offset direction changes, the offset angle obtained after the first offset length is used as the new current offset angle, and the first offset length is proportionally reduced to obtain a new first offset length, and the process returns to the step of offsetting the current offset angle by the first offset length in the opposite direction of the offset direction, until the first offset length is reduced to less than the angle execution accuracy;
[0172] The angle execution accuracy represents the accuracy of the angle offset information within a reasonable error range;
[0173] It can be understood that the above steps are introducing the situation where the offset direction changes. If the offset direction changes, the offset angle obtained after offsetting the first step length is used as the new current offset angle, and the first step length is reduced proportionally to obtain a new first step length, and the step of offsetting the current offset angle in the opposite direction of the offset direction by the first step length is returned until the first step length is reduced to less than the angle execution accuracy.
[0174] It should be noted that the specific ratio of the proportional reduction can be adjusted according to actual usage, and the embodiment of the present invention does not make any specific limitation to this.
[0175] It can be understood that the angle execution accuracy is mostly a pre-set accuracy within a reasonable error range. Under normal circumstances, the first step length can be proportionally reduced by 4-8 times to make the first step length reach the angle execution accuracy. The angle execution accuracy is mostly ±0.5°, and the embodiment of the present invention does not make any specific limitation on this.
[0176] For example, if the current offset angle is 3°, the offset direction is to the right, and the preset first step length is 5°, then after the first step length is offset, the offset direction is to the left, so the offset direction changes. At this time, it will be offset to the left by 2° as the new current offset angle, and the first step length will be proportionally reduced by one-half to obtain a new first step length of 2.5°. Return to the step of offsetting the current offset angle in the opposite direction of the offset direction by the first step length until the first step length is reduced to less than the angle execution accuracy of ±0.5°.
[0177] Step A7, determining the angular offset information of the offset angle obtained by the last offset, and obtaining the corrected angular offset information.
[0178] It can be understood that the angular offset information of the offset angle obtained by the last offset is used as the corrected angular offset information, so that the angular offset information within the angle execution accuracy can be obtained.
[0179] Exemplarily, the angular offset information of the offset angle obtained by the last offset is 0.1° to the left, which is used as the corrected angular offset information.
[0180] It should be noted that the error correction method of the entire angular offset information can be understood as a dichotomy method using iterative approximation, and the angular offset information is corrected multiple times by iteratively approximating the first step length, thereby obtaining the corrected angular offset information.
[0181] The error correction method corresponding to the angular offset information provided in the embodiment of the present invention can be corrected by iteratively approximating the angular offset information, thereby obtaining the corrected angular offset information. The accuracy of the corrected angular offset information is obtained through multiple iterative approximations until the angular offset information is within the angle execution accuracy, that is, the error of the angular offset information is within a reasonable range. It can be seen that the embodiment of the present invention can effectively improve the accuracy of the angular offset information of the multi-axis robot, thereby improving the repositioning accuracy of the multi-axis robot, thereby improving the repositioning accuracy of the mobile multi-axis robot when working.
[0182] Optionally, in one implementation method, the error correction method corresponding to the depth information includes steps B1-B5:
[0183] Step B1, determining two reference image areas adjacent to a designated image area and reference depth information corresponding to the two reference image areas from a pre-collected mapping table of various reference image areas corresponding to the target to be operated, to obtain a first set of data; the designated image area is an image area of the target to be operated in the image data;
[0184] It can be understood that the correction of the depth information can be understood as correction on the y-axis of the base station coordinates.
[0185] It should be noted that the reference depth information corresponding to the reference area has a corresponding relationship, and the corresponding relationship is similar to the principle of pinhole imaging. The larger the reference image area, the smaller the corresponding reference depth information, and vice versa. The smaller the reference image area, the larger the corresponding reference depth information. Among them, the reference image area is the image area corresponding to the target to be operated collected in advance, and the reference depth information is the vertical distance from the end effector to the target to be operated in the base coordinate system corresponding to the reference image area collected in advance.
[0186] Exemplarily, when the depth information is 100 mm, the image area obtained by image acquisition is 41136 pix; when the depth information is 105 mm, the image area obtained by image acquisition is 38470 pix.
[0187] Among them, the reference image area and reference depth information are the information in the mapping table of the correspondence between the two objects collected in advance. Therefore, before error correction, it is necessary to establish a mapping table of the reference image area and reference depth information corresponding to the object to be operated. The mapping table can be obtained through multiple tests and records. In addition, the reference image area is mostly pixel area, so unit conversion is required.
[0188]
[0189]
[0190] Table 1
[0191] As shown in Table 1, the reference image area is the pixel area in Table 1, and the reference depth information is the depth in Table 1. Therefore, it can be understood that two reference image areas adjacent to the specified image area and the reference depth information corresponding to the two reference image areas can be determined based on the information in Table 1. It should be noted that each target to be operated has a corresponding mapping table of reference image area and reference depth information. The data in each mapping table is pre-collected. In actual use, the mapping table of reference image area and reference depth information corresponding to the target to be operated can be directly selected for use. The above mapping table is only a mapping table corresponding to a certain target to be operated.
[0192] For example, if the specified image area is 20000pix, then the image area of 20232pix can be selected as the reference image area, and its corresponding depth information of 160mm can be selected as the reference depth information, and the image area of 19279pix can be selected as the reference image area, and its corresponding depth information of 165mm can be selected as the reference depth information, thereby obtaining the first set of data.
[0193] Step B2, determining two other reference image areas other than the reference image area in the first set of data, and reference depth information corresponding to the two other reference areas from the mapping table, to obtain a second set of data;
[0194] It is understandable that the other two selected reference image areas are not necessarily adjacent to the image area in the first set of data, and the embodiment of the present invention does not specifically limit the second set of data.
[0195] For example, if the specified image area is 20000pix, then the image area 20232pix can be selected as the reference image area, and the corresponding depth information 160mm can be selected as the reference depth information, and the image area 19279pix can be selected as the reference image area, and the corresponding depth information 165mm can be selected as the reference depth information, thereby obtaining the first set of data. Then, two other reference image areas other than the first set of data and the reference depth information corresponding to the other two reference areas can be selected, and the image area 18349pix can be selected as the reference image area, and the corresponding depth information 170mm can be selected as the reference depth information, and the image area 22410pix can be selected as the reference image area, and the corresponding depth information 150mm can be selected as the reference depth information, thereby obtaining the second set of data.
[0196] Step B3, solving the constant parameters in the first linear equation by substituting the first set of data and the second set of data into the first linear equation for solving the constant parameters; wherein the first linear equation is a quadratic equation with the image area of any target as the independent variable and the depth information of the any target as the dependent variable, and the content of the equation used when substituting the first set of data and the second set of data into the first linear equation for solving the parameters is as follows:
[0197]
[0198]
[0199]
[0200] Among them, s 1 、s 2 are the areas of the two reference images in the first set of data, s0 、s 3 are the areas of the two reference images in the second set of data, and p 1 is the reference image area s in the mapping table 1 The corresponding reference depth information, p 2 is the reference image area s in the mapping table 2 The corresponding reference depth information, p 3 is the reference image area s in the mapping table 3 The corresponding reference depth information, p 4 is the reference image area s in the mapping table 4 corresponding depth information, t is the distance from the normal plane of the end effector to the normal plane of the aperture, a, b and c are constant parameters to be solved for the first linear equation;
[0201] It can be understood that the first set of data can be substituted into the first and second equations in the first linear equation, then it can also be understood that the two parameters in a, b and c are regarded as functions of another parameter; the second set of data can be substituted into the third equation in the first linear equation to obtain the optimization of a scalar function, and directly deriving it is a linear equation, thus obtaining a, b and c; d, as an important parameter in the equation, is composed of the depth information corresponding to the image area, i.e., p, and the distance from the normal plane of the end effector to the normal plane of the aperture, i.e., t, where t is a constant. It should be noted that the results of the first and second equations in the first linear equation are both equal to 0, which means that the points composed of the two reference image areas in the first set of data and their corresponding d are all on the first linear equation, and the third equation into which the second set of data is substituted is essentially to obtain the optimization of a scalar function. The purpose of the third equation is to make the points composed of the two reference image areas in the second set of data and their corresponding d as close to the first linear equation as possible. It can also be understood that the first linear equation satisfies the points represented by the two sets of data with adjacent image areas in the image data, which are on the first linear equation and are close to the points represented by the other two sets of data.
[0202] Exemplarily, the designated image area is 20785pix, and two reference image areas adjacent to the designated image area and reference depth information corresponding to the two reference image areas are determined from the mapping table to obtain a first set of data. The reference image area 21361pix and the corresponding reference depth information 155mm in the first set of data are substituted into the first linear equation. The reference image area of 20232pix and its corresponding reference depth information of 160mm are substituted into the first linear equation From the mapping table, determine the other two reference image areas other than the reference image area in the first set of data, and the reference depth information corresponding to the other two reference areas, and obtain the second set of data. Then substitute the reference image area 22410pix and its corresponding depth information 150mm, the reference image area 19279pix and its corresponding depth information 165mm of the second set of data into the first linear equation. Thus, the constant parameters a, b and c in the first linear equation are obtained.
[0203] Step B4, substituting the designated image area into the first linear equation whose constant parameters have been solved to obtain a function value;
[0204] It can be understood that by substituting the specified image area into the first linear equation solved for the constant parameter, d can be obtained. Therefore, the function value p, that is, the depth information, can be obtained.
[0205] Step B5, using the function value to correct the depth information to obtain corrected depth information;
[0206] It can be understood that the depth information is corrected using the function value obtained above, thereby obtaining corrected depth information.
[0207] Exemplarily, the function value is 100 mm, the current depth information is 120 mm, and the current depth information is corrected to the function value, thereby obtaining the corrected depth information of 100 mm.
[0208] The error correction method corresponding to the depth information provided by the embodiment of the present invention can be directly calculated by the first linear equation to obtain the corrected depth information, which is within a reasonable error range. It can be seen that the embodiment of the present invention can effectively improve the angular offset information accuracy of the multi-axis robot, thereby improving the repositioning accuracy of the multi-axis robot, thereby improving the repositioning accuracy of the mobile multi-axis robot when working.
[0209] Optionally, in one implementation method, the error correction method corresponding to the line offset information includes steps C1-C9:
[0210] Step C1, estimating the height error and lateral error from the end effector to the target to be operated;
[0211] It is understandable that after the depth information correction is completed, the line offset information can be corrected for errors. The line offset information only occurs in the xz plane in the base station coordinate system.
[0212] In one implementation, estimating the height error and lateral error from the end effector to the target to be operated may include steps C11-C14:
[0213] Step C11, after guiding the position of the multi-axis robot according to the corrected depth information, the image data of the target to be operated is collected again by using the image acquisition device to obtain auxiliary image data;
[0214] It is understandable that after the multi-axis robot is positionally guided according to the corrected depth information, the image acquisition device can be used to acquire image data of the target to be operated again to obtain auxiliary image data, whose depth information and angular offset information are corrected.
[0215] Step C12, performing data analysis on the auxiliary image data to obtain depth information;
[0216] It is understandable that the content of data analysis on auxiliary image data has been introduced, so it will not be repeated here.
[0217] Step C13, calculating a scaling ratio between the obtained depth information and the corrected depth information to obtain size information;
[0218] The unit of the depth information obtained by analyzing the auxiliary image is pix, which means pixel, and unit conversion is required.
[0219] It can be understood that by comparing the depth information obtained by analyzing the auxiliary image with the corrected depth information, a scaling ratio can be obtained, and the scaling ratio is scale information.
[0220] Exemplarily, the depth information obtained by analyzing the auxiliary image is 10 mm, and the depth information after correction is 100 mm, so the scale information is 1:10.
[0221] Step C14: using the size information and the line offset information, estimate the height error and lateral error from the end effector to the target to be operated.
[0222] It is understandable that, based on the scale information and the line offset information obtained by analyzing the image data, the height error and lateral error from the end effector to the target to be operated can be estimated.
[0223] For example, the line offset information obtained from the image data analysis indicates that the height error from the end effector to the target to be operated is 5 mm, and the lateral error is 10 mm. Then, the line offset information obtained from the image data analysis can be divided by the scale information to obtain an estimated height error from the end effector to the target to be operated of 50 mm, and a lateral error of 100 mm.
[0224] Step C2, performing preliminary correction on the line offset information according to the height error and the lateral error to obtain preliminary corrected line offset information;
[0225] The line offset information after preliminary correction includes the height distance and the lateral distance after preliminary correction.
[0226] Exemplarily, according to the above-mentioned height error of 50 mm and lateral error of 100 mm, the line offset information is preliminarily corrected to obtain the preliminarily corrected line offset information.
[0227] Step C3, obtaining the offset direction of the height distance and the lateral distance represented by the line offset information after the preliminary correction;
[0228] The offset directions of the height distance and the lateral distance may be represented by line offset information.
[0229] It can be understood that the offset direction of the lateral distance in the line offset information may be leftward or rightward; the offset direction of the height distance in the line offset information may be upward or downward.
[0230] It should be noted that, since the line offset information is in two aspects, namely, the height distance and the lateral distance, during the error correction process, the height distance and the lateral distance can be corrected simultaneously.
[0231] Step C4, obtaining the offset distance of the height distance and the lateral distance represented by the line offset information after the preliminary correction as the current offset distance;
[0232] It should be noted that the linear information can not only represent the offset direction of the height distance and the lateral distance, but also represent the offset distance of the height distance and the lateral distance, and the above offset distance is used as the current offset distance.
[0233] Exemplarily, the line offset information may indicate that the lateral distance is offset to the right by 30 mm, and the height distance is offset to the upward by 20 mm.
[0234] Step C5, shifting the current offset distance in the opposite direction of the offset direction by a second step length;
[0235] The second step length is a length value related to the offset distance, and an initial value of the second step length is a preset offset distance.
[0236] It is understandable that the initial value of the second step length is a preset offset distance, which may be changed according to the current offset distance, and the embodiment of the present invention does not specifically limit this.
[0237] For example, if the current offset distance of the lateral distance is 20mm, the offset direction is to the right, and the preset second step length is 5mm, then the second step length can be offset in the opposite direction of the offset direction, that is, 5mm to the left; if the current offset distance of the height distance is 20mm, the offset direction is to the upward direction, and the preset second step length is 5mm, then the second step length can be offset in the opposite direction of the offset direction, that is, 5mm downward;.
[0238] Step C6, determining whether the offset direction changes after the second step length is offset;
[0239] It can be understood that after the second step length is shifted, two situations may occur: the shift direction changes or the shift direction does not change.
[0240] For example, if the current offset distance of the lateral distance is 20 mm, the offset direction is to the right, and the pre-set second step length is 5 mm, then after offsetting to the left by the second step length, the offset direction is still to the right, so the offset direction has not changed; if the current offset distance of the lateral distance is 3 mm, the offset direction is to the right, and the pre-set second step length is 5 mm, then after offsetting to the left by the second step length, the offset direction is to the left, so the offset direction has changed.
[0241] Step C7, if the offset direction has not changed, the offset distance obtained after the second step length is offset is used as the new current offset distance, and the process returns to the step of offsetting the current offset distance in the opposite direction of the offset direction by the second step length until the offset direction changes;
[0242] It can be understood that the above steps are to introduce the situation where the offset direction has not changed. If the offset direction has not changed, the offset distance obtained after the second step is offset is used as the new current offset distance, and the above steps are returned until the offset direction changes.
[0243] For example, if the current offset distance of the lateral distance is 20mm, the offset direction is to the right, and the preset second step length is 5mm, then after shifting to the left by the second step length, the offset direction is still to the right and has not changed. At this time, the offset distance is 15mm. 15mm is used as the new current offset distance, and 15mm is continuously shifted to the left by 5mm, and 10mm is used as the new current offset distance until the offset direction changes.
[0244] Step C8, if the offset direction changes, the offset distance obtained after the second step length is offset is used as the new current offset distance, and the second step length is proportionally reduced to obtain a new second step length, and the process returns to the step of offsetting the current offset distance by the second step length in the opposite direction of the offset direction, until the second step length is reduced to less than the distance execution accuracy;
[0245] The distance execution accuracy represents the accuracy of the line offset information within a reasonable error range;
[0246] It can be understood that the above steps are introducing the situation where the offset direction changes. If the offset direction changes, the offset distance obtained after the second step length is offset is used as the new current offset distance, and the second step length is proportionally reduced to obtain a new second step length, and the step of offsetting the current offset distance in the opposite direction of the offset direction by the second step length is returned until the second step length is reduced to less than the distance execution accuracy.
[0247] It should be noted that the specific ratio of the proportional reduction can be adjusted according to actual usage, and the embodiment of the present invention does not make any specific limitation to this.
[0248] It can be understood that the distance execution accuracy is mostly a pre-set accuracy within a reasonable error range. Under normal circumstances, the second step length can be proportionally reduced by 4-6 times to achieve the distance execution accuracy. The distance execution accuracy is mostly ±0.2mm, and the embodiment of the present invention does not make specific limitations on this.
[0249] For example, if the current offset distance of the lateral distance is 3mm, the offset direction is to the right, and the preset second step length is 5mm, then after the offset is the second step length, the offset direction is to the left, so the offset direction changes. At this time, the left offset is 2mm as the new current offset distance, and the second step length is proportionally reduced by one half to obtain a new second step length of 2.5mm. Return to the step of offsetting the current offset distance in the opposite direction of the offset direction by the second step length until the second step length is reduced to less than the distance execution accuracy of ±0.2mm.
[0250] Step C9, determining the line offset information of the offset distance obtained by the last offset, and obtaining the corrected line offset information.
[0251] It can be understood that the line offset information of the offset distance obtained by the last offset is used as the corrected line offset information, so that the line offset information within the distance execution accuracy can be obtained.
[0252] Exemplarily, the line offset information of the offset distance obtained by the last offset is 0.1 mm to the left and 0.2 mm upward, which is used as the corrected line offset information.
[0253] It should be noted that the error correction method of the entire line offset information can be understood as the dichotomy method using bidirectional iterative approximation, and the line offset information is corrected multiple times by iteratively approximating the second step length to obtain the corrected line offset information.
[0254] The error correction method corresponding to the line offset information provided by the embodiment of the present invention, since the line offset information includes the height distance and the lateral distance, can be corrected by performing a bidirectional iterative approximation on the line offset information, thereby obtaining the corrected line offset information. The accuracy of the corrected line offset information is obtained through multiple bidirectional iterative approximations until the line offset information is within the distance execution accuracy, that is, within the reasonable error range of the line offset information. It can be seen that the embodiment of the present invention can effectively improve the accuracy of the line offset information of the multi-axis robot, thereby improving the repositioning accuracy of the multi-axis robot, thereby improving the repositioning accuracy of the mobile multi-axis robot when working.
[0255] Based on the above method embodiment, Figure 6 As shown, an embodiment of the present invention provides a robot guiding device, which is applied to a control device, wherein the control device is communicatively connected with a mobile multi-axis robot; the mobile multi-axis robot comprises: a multi-axis robot and a mobile platform, wherein the multi-axis robot is provided with a mechanical arm of an end effector and an image acquisition device is mounted thereon; the device comprises:
[0256] The control module 610 is used to control the mobile platform to move to a predetermined reference position; wherein the reference position is the position of the target to be operated obtained after preliminary positioning of the target to be operated;
[0257] A collection module 620 is used to collect image data of the target to be operated by using the image collection device after the mobile platform reaches the reference position;
[0258] The analysis module 630 is used to perform data analysis on the image data to obtain specified description information; wherein the specified description information is description information used to characterize the relative position relationship between the target to be operated and the multi-axis robot;
[0259] An error correction module 640, configured to perform error correction on the specified description information;
[0260] The guiding module 650 is used to guide the multi-axis robot according to the corrected designated description information, so that the end effector of the multi-axis robot after being guided reaches the operating area for the target to be operated.
[0261] Optionally, the specified description information includes: angular offset information, depth information and line offset information; wherein the angular offset information is the angular offset information from the image plane of the image acquisition device to the target to be operated in the base coordinate system, the depth information is the vertical distance from the end effector to the target to be operated in the base coordinate system, and the line offset information is the height distance and lateral distance from the end effector to the target to be operated in the base coordinate system, and the base coordinate system is a coordinate system centered on the multi-axis robot;
[0262] The error correction module comprises:
[0263] an error correction submodule, configured to perform error correction on each type of information among the angular offset information, the depth information and the line offset information according to a predetermined correction order and based on an error correction method corresponding to the type of information;
[0264] The predetermined correction order is a correction order regarding the angular offset information, the depth information and the line offset information.
[0265] Optionally, the predetermined correction sequence includes:
[0266] The error correction is performed on the angular offset information, the depth information and the line offset information in sequence.
[0267] Optionally, the error correction method corresponding to the angular offset information includes:
[0268] Acquire the offset direction of the image plane relative to the target to be operated represented by the angular offset information;
[0269] Obtaining an offset angle represented by the angular offset information as a current offset angle;
[0270] The current offset angle is offset in the opposite direction of the offset direction by a first step length; wherein the first step length is an angle value with respect to the offset angle, and an initial value of the first step length is a preset offset angle;
[0271] Determining whether the offset direction changes after the first offset length;
[0272] If the offset direction has not changed, the offset angle obtained after offsetting the first step is used as the new current offset angle, and the process returns to the step of offsetting the current offset angle in the opposite direction of the offset direction by the first step until the offset direction changes;
[0273] If the offset direction changes, the offset angle obtained after the first offset length is used as the new current offset angle, and the first offset length is proportionally reduced to obtain a new first offset length, and the step of offsetting the current offset angle by the first offset length in the opposite direction of the offset direction is returned until the first offset length is reduced to less than the angle execution accuracy; wherein the angle execution accuracy represents the accuracy of the angle offset information within a reasonable error range;
[0274] The angular offset information of the offset angle obtained by the last offset is determined to obtain the corrected angular offset information.
[0275] Optionally, the error correction method corresponding to the depth information includes:
[0276] Determine two reference image areas adjacent to a designated image area and reference depth information corresponding to the two reference image areas from a pre-collected mapping table of various reference image areas corresponding to the target to be operated, and obtain a first set of data; the designated image area is an image area of the target to be operated in the image data;
[0277] Determine from the mapping table two other reference image areas other than the reference image area in the first set of data, and reference depth information corresponding to the other two reference areas, to obtain a second set of data;
[0278] The constant parameters in the first linear equation are solved by substituting the first set of data and the second set of data into the first linear equation for solving the constant parameters; wherein the first linear equation is a quadratic equation with the image area of any target as the independent variable and the depth information of the any target as the dependent variable, and the equation used when substituting the first set of data and the second set of data into the first linear equation for solving the parameters is as follows:
[0279]
[0280]
[0281]
[0282] Among them, s 1 、s 2 are the areas of the two reference images in the first set of data, s 0 、s 3 are the areas of the two reference images in the second set of data, and p 1 is the reference image area s in the mapping table 1 The corresponding reference depth information, p 2 is the reference image area s in the mapping table2 The corresponding reference depth information, p 3 is the reference image area s in the mapping table 3 The corresponding reference depth information, p 4 is the reference image area s in the mapping table 4 corresponding depth information, t is the distance from the normal plane of the end effector to the normal plane of the aperture, a, b and c are constant parameters to be solved for the first linear equation;
[0283] Substituting the designated image area into the first linear equation in which the constant parameter is solved to obtain a function value;
[0284] The depth information is corrected using the function value to obtain corrected depth information.
[0285] Optionally, the error correction method corresponding to the line offset information includes:
[0286] estimating the height error and lateral error from the end effector to the target to be operated;
[0287] According to the height error and the lateral error, the line offset information is preliminarily corrected to obtain preliminarily corrected line offset information; wherein the preliminarily corrected line offset information includes a preliminarily corrected height distance and a lateral distance;
[0288] Obtaining the offset direction of the height distance and the lateral distance represented by the line offset information after the preliminary correction;
[0289] Obtaining the offset distance of the height distance and the lateral distance represented by the line offset information after the preliminary correction as the current offset distance;
[0290] The current offset distance is offset in the opposite direction of the offset direction by a second step length; wherein the second step length is a length value related to the offset distance, and an initial value of the second step length is a preset offset distance;
[0291] Determine whether the offset direction changes after the second step length is offset;
[0292] If the offset direction has not changed, the offset distance obtained after the second step length is offset is used as the new current offset distance, and the process returns to the step of offsetting the current offset distance in the opposite direction of the offset direction by the second step length until the offset direction changes;
[0293] If the offset direction changes, the offset distance obtained after the second step length is offset is used as the new current offset distance, and the second step length is proportionally reduced to obtain a new second step length, and the step of offsetting the current offset distance by the second step length in the opposite direction of the offset direction is returned to until the second step length is reduced to less than the distance execution accuracy; wherein the distance execution accuracy represents the accuracy of the line offset information within a reasonable error range;
[0294] The line offset information of the offset distance obtained by the last offset is determined to obtain the corrected line offset information.
[0295] Optionally, the estimating the height error and lateral error from the end effector to the target to be operated includes:
[0296] After guiding the position of the multi-axis robot according to the corrected depth information, the image data of the target to be operated is collected again by using the image acquisition device to obtain auxiliary image data;
[0297] Performing data analysis on the auxiliary image data to obtain depth information;
[0298] Calculating a scaling ratio between the obtained depth information and the corrected depth information to obtain size information;
[0299] The height error and lateral error from the end effector to the target to be operated are estimated by using the size information and the line offset information.
[0300] The embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.
[0301] Memory 703, used for storing computer programs;
[0302] The processor 701 is used to implement the steps of the robot guidance method described in the embodiment of the present invention when executing the program stored in the memory 703.
[0303] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0304] The communication interface is used for communication between the above electronic device and other devices.
[0305] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0306] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0307] In another embodiment provided by the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned robot guidance methods are implemented.
[0308] In another embodiment provided by the present invention, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any robot guiding method in the above embodiments.
[0309] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.
[0310] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0311] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0312] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims
1. A robot guidance method, It is characterized in that The invention is applied to a control device, wherein the control device is connected to a mobile multi-axis robot in communication; the mobile multi-axis robot comprises: a multi-axis robot and a mobile platform, and the multi-axis robot is provided with an end effector mechanical arm on which an image acquisition device is mounted; the method comprises: Controlling the mobile platform to move to a predetermined reference position; wherein the reference position is the position of the target to be operated obtained after preliminary positioning of the target to be operated; After the mobile platform reaches the reference position, using the image acquisition device to acquire image data of the target to be operated; Performing data analysis on the image data to obtain specified description information; wherein the specified description information is description information used to characterize the relative position relationship between the target to be operated and the multi-axis robot; Performing error correction on the specified description information; wherein the specified description information includes: angular offset information, depth information and line offset information; wherein the angular offset information is the angular offset information from the image plane of the image acquisition device to the target to be operated in the base coordinate system, the depth information is the vertical distance from the end effector to the target to be operated in the base coordinate system, and the line offset information is the height distance and lateral distance from the end effector to the target to be operated in the base coordinate system, and the base coordinate system is a coordinate system centered on the multi-axis robot; According to the corrected designated description information, positionally guiding the multi-axis robot so that the end effector of the guided multi-axis robot reaches within the operation area for the target to be operated; The error correction method corresponding to the angular offset information includes: Acquire the offset direction of the image plane relative to the target to be operated represented by the angular offset information; Obtaining an offset angle represented by the angular offset information as a current offset angle; The current offset angle is offset in the opposite direction of the offset direction by a first step length; wherein the first step length is an angle value with respect to the offset angle, and an initial value of the first step length is a preset offset angle; Determining whether the offset direction changes after the first offset length; If the offset direction has not changed, the offset angle obtained after offsetting the first step is used as the new current offset angle, and the process returns to the step of offsetting the current offset angle in the opposite direction of the offset direction by the first step until the offset direction changes; If the offset direction changes, the offset angle obtained after the first offset length is used as the new current offset angle, and the first offset length is proportionally reduced to obtain a new first offset length, and the step of offsetting the current offset angle by the first offset length in the opposite direction of the offset direction is returned until the first offset length is reduced to less than the angle execution accuracy; wherein the angle execution accuracy represents the accuracy of the angle offset information within a reasonable error range; The angular offset information of the offset angle obtained by the last offset is determined to obtain the corrected angular offset information.
2. The method according to claim 1, It is characterized in that Performing error correction on the specified description information, comprising: According to a predetermined correction order, for each type of information among the angular offset information, the depth information and the line offset information, based on an error correction method corresponding to the type of information, error correction is performed on the type of information; The predetermined correction order is a correction order regarding the angular offset information, the depth information and the line offset information.
3. The method according to claim 2, It is characterized in that The predetermined correction sequence includes: The error correction is performed on the angular offset information, the depth information and the line offset information in sequence.
4. The method according to claim 2, It is characterized in that The error correction method corresponding to the depth information includes: Determine two reference image areas adjacent to a designated image area and reference depth information corresponding to the two reference image areas from a pre-collected mapping table of various reference image areas corresponding to the target to be operated, and obtain a first set of data; the designated image area is an image area of the target to be operated in the image data; Determine from the mapping table two other reference image areas other than the reference image area in the first set of data, and reference depth information corresponding to the other two reference areas, to obtain a second set of data; The constant parameters in the first linear equation are solved by substituting the first set of data and the second set of data into the first linear equation for solving the constant parameters; wherein the first linear equation is a quadratic equation with the image area of any target as the independent variable and the depth information of the any target as the dependent variable, and the equation used when substituting the first set of data and the second set of data into the first linear equation for solving the parameters is as follows: Among them, s 1 、s 2 are the areas of the two reference images in the first set of data, s 0 、s 3 are the areas of the two reference images in the second set of data, and p 1 is the reference image area s in the mapping table 1 The corresponding reference depth information, p 2 is the reference image area s in the mapping table 2 The corresponding reference depth information, p 3 is the reference image area s in the mapping table 3 The corresponding reference depth information, p 0 is the reference image area s in the mapping table 0 corresponding depth information, t is the distance from the normal plane of the end effector to the normal plane of the aperture, a, b and c are constant parameters to be solved for the first linear equation; Substituting the designated image area into the first linear equation in which the constant parameter is solved to obtain a function value; The depth information is corrected using the function value to obtain corrected depth information.
5. The method according to claim 2, It is characterized in that The error correction method corresponding to the line offset information includes: estimating the height error and lateral error from the end effector to the target to be operated; According to the height error and the lateral error, the line offset information is preliminarily corrected to obtain preliminarily corrected line offset information; wherein the preliminarily corrected line offset information includes a preliminarily corrected height distance and a lateral distance; Obtaining the offset direction of the height distance and the lateral distance represented by the line offset information after the preliminary correction; Obtaining the offset distance of the height distance and the lateral distance represented by the line offset information after the preliminary correction as the current offset distance; The current offset distance is offset in the opposite direction of the offset direction by a second step length; wherein the second step length is a length value related to the offset distance, and an initial value of the second step length is a preset offset distance; Determine whether the offset direction changes after the second step length is offset; If the offset direction has not changed, the offset distance obtained after the second step length is offset is used as the new current offset distance, and the process returns to the step of offsetting the current offset distance in the opposite direction of the offset direction by the second step length until the offset direction changes; If the offset direction changes, the offset distance obtained after the second step length is offset is used as the new current offset distance, and the second step length is proportionally reduced to obtain a new second step length, and the step of offsetting the current offset distance by the second step length in the opposite direction of the offset direction is returned to until the second step length is reduced to less than the distance execution accuracy; wherein the distance execution accuracy represents the accuracy of the line offset information within a reasonable error range; The line offset information of the offset distance obtained by the last offset is determined to obtain the corrected line offset information.
6. The method according to claim 5, It is characterized in that The estimating the height error and lateral error from the end effector to the target to be operated includes: After guiding the position of the multi-axis robot according to the corrected depth information, the image data of the target to be operated is collected again by using the image acquisition device to obtain auxiliary image data; Performing data analysis on the auxiliary image data to obtain depth information; Calculating a scaling ratio between the obtained depth information and the corrected depth information to obtain size information; The height error and lateral error from the end effector to the target to be operated are estimated by using the size information and the line offset information.
7. A robot guiding device, It is characterized in that Applied to a control device, the control device being communicatively connected to a mobile multi-axis robot; The mobile multi-axis robot comprises: a multi-axis robot and a mobile platform, wherein the end effector of the multi-axis robot is equipped with an image acquisition device; the device comprises: A control module, used to control the mobile platform to move to a predetermined reference position; wherein the reference position is a position of the target to be operated obtained after preliminary positioning of the target to be operated; A collection module, used for collecting image data of the target to be operated by using the image collection device after the mobile platform reaches the reference position; An analysis module, used for performing data analysis on the image data to obtain specified description information; wherein the specified description information is description information used to characterize the relative position relationship between the target to be operated and the multi-axis robot; An error correction module is used to perform error correction on the specified description information; wherein the specified description information includes: angular offset information, depth information and line offset information; wherein the angular offset information is the angular offset information from the image plane of the image acquisition device to the target to be operated in the base coordinate system, the depth information is the vertical distance from the end effector to the target to be operated in the base coordinate system, and the line offset information is the height distance and lateral distance from the end effector to the target to be operated in the base coordinate system, and the base coordinate system is a coordinate system centered on the multi-axis robot; A guiding module, used for guiding the position of the multi-axis robot according to the corrected designated description information, so that the end effector of the multi-axis robot after being guided reaches the operation area for the target to be operated; The error correction method corresponding to the angular offset information includes: Acquire the offset direction of the image plane relative to the target to be operated represented by the angular offset information; Obtaining an offset angle represented by the angular offset information as a current offset angle; The current offset angle is offset in the opposite direction of the offset direction by a first step length; wherein the first step length is an angle value with respect to the offset angle, and an initial value of the first step length is a preset offset angle; Determining whether the offset direction changes after the first offset length; If the offset direction has not changed, the offset angle obtained after offsetting the first step is used as the new current offset angle, and the process returns to the step of offsetting the current offset angle in the opposite direction of the offset direction by the first step until the offset direction changes; If the offset direction changes, the offset angle obtained after the first offset length is used as the new current offset angle, and the first offset length is proportionally reduced to obtain a new first offset length, and the step of offsetting the current offset angle by the first offset length in the opposite direction of the offset direction is returned until the first offset length is reduced to less than the angle execution accuracy; wherein the angle execution accuracy represents the accuracy of the angle offset information within a reasonable error range; The angular offset information of the offset angle obtained by the last offset is determined to obtain the corrected angular offset information.
8. An electronic device, It is characterized in that It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-6 when executing a program stored in a memory.
9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Mobile vision robot and measurement and control method thereof
CN106607907A
Method and device for determining repeated positioning accuracy of robot and the robot
CN110561498A
Robot correction method and device, computer equipment and storage medium
CN111438688A
High-precision control method and system of multi-axis servo system based on machine vision
CN114355953A
Method and apparatus for determining pose information of a robot, device and medium
US20220026194A1
