Grabbing mechanism control method and system based on visual positioning technology

Abstract

According to the grabbing mechanism control method and system based on the visual positioning technology, a motion control system can drive a posture adjusting mechanism to move to approach a target, and after the posture adjusting mechanism reaches a set position, a visual measurement system is started to carry out image collection measurement and posture calculation; the vision measurement system feeds back attitude errors of the mechanism and the target to the motion control system, the motion control system can drive the attitude adjusting mechanism to adjust again according to the errors, and rapid and accurate alignment and grabbing can be carried out under the condition that the target is not easy to observe manually.

Description

Technical Field

[0001] The present invention relates to the technical field of gripping mechanism control, and in particular to a gripping mechanism control method and system based on visual positioning technology. Background Art

[0002] For large-scale grabbing and connecting mechanisms that require shaft-hole matching, in some cases it is impossible to use traditional mechanical mechanisms to complete the grabbing and connecting of the target. The main shortcomings of traditional mechanisms are as follows:

[0003] The connection and locking of the docking interface requires manual confirmation. For interfaces that are difficult to observe, manual confirmation is impossible.

[0004] The posture adjustment of the mechanism requires manual operation, which is slow and inefficient;

[0005] For large targets, manual operation requires multiple people to climb onto the docking platform, which poses a safety hazard. Summary of the Invention

[0006] The present invention aims to provide a grasping mechanism control method and system based on visual positioning technology that overcomes the above problems or at least partially solves the above problems.

[0007] To achieve the above object, the technical solution of the present invention is specifically implemented as follows:

[0008] One aspect of the present invention provides a method for controlling a gripping mechanism based on visual positioning technology, comprising:

[0009] The motion control system controls the posture adjustment mechanism to reach the preset initial photographing position and then sends an in-position signal;

[0010] The visual measurement system receives the in-position signal and performs image acquisition;

[0011] The visual measurement system performs radial position measurement on the collected image and calculates the distance to the target;

[0012] The visual measurement system determines whether the distance exceeds a preset range. If so, the motion control system is notified. If not, the target features of the acquired image are extracted, and a posture calculation is performed to obtain a calculation result. A posture adjustment instruction is generated based on the calculation result and sent to the motion control system.

[0013] The motion control system controls the posture adjustment mechanism to adjust the posture according to the posture adjustment instruction;

[0014] The motion control system controls the grasping mechanism to grasp.

[0015] Optionally, the method further comprises:

[0016] Set the visual measurement system in a preset manner, wherein the visual measurement system includes four sets of monocular vision acquisition devices.

[0017] Optionally, the extraction of the target features of the acquired images includes:

[0018] Extract the preset pin hole features or the features of the artificial target preset in the target image.

[0019] Optionally, the artificial target is set as a dot array.

[0020] Optionally, the attitude calculation to obtain the calculation result includes:

[0021] Calculate the three-dimensional coordinates of the target feature in each camera coordinate system;

[0022] According to the pre-calibrated conversion relationship between the pose adjustment mechanism coordinate system and the camera coordinate system, convert the three-dimensional coordinates of the target feature in each camera coordinate system to the pose adjustment mechanism coordinate system;

[0023] According to the standard docking feature points in the pre-calibrated pose adjustment mechanism coordinate system;

[0024] Calculate the rotation matrix R and the translation matrix T from the current position to the target position;

[0025] Decompose the rotation matrix R to obtain the pitch angle, yaw angle and roll angle;

[0026] Decompose the translation matrix T to obtain the pose adjustment parameters of the displacements in three directions.

[0027] Optionally, the generation of the pose adjustment instruction according to the calculation result and sending it to the motion control system includes:

[0028] Judge whether the errors of the pitch angle, yaw angle and roll angle are less than the preset threshold;

[0029] If it is greater than or equal to the preset threshold, generate an angle pose adjustment instruction and send it to the motion control system;

[0030] Judge whether the displacements in the three directions need to be adjusted. If so, generate a displacement pose adjustment instruction and send it to the motion control system.

[0031] Optionally, the method further includes:

[0032] The visual measurement system receives the pose adjustment result feedback information, judges whether the pose adjustment is successful. If not, interrupt the pose adjustment and report an exception.

[0033] Optionally, the method further includes:

[0034] The visual measurement system determines whether the maximum pose adjustment times are exceeded. If so, the pose adjustment is interrupted and an exception is reported.

[0035] Optionally, the method further includes:

[0036] The motion control system controls the pose adjustment mechanism to advance a preset distance.

[0037] The visual measurement system determines whether the positioning feature is acquired.

[0038] If the positioning feature is not acquired, the motion control system determines whether the safety limit is reached. If the safety limit is not reached, the operation of the motion control system controlling the pose adjustment mechanism to advance the preset distance is returned until the positioning feature is acquired. If the safety limit is reached, the motion control system controls the pose adjustment mechanism to stop moving.

[0039] Another aspect of the present invention provides a grasping mechanism control system based on visual positioning technology, which uses the above-mentioned grasping mechanism control method based on visual positioning technology to control the grasping mechanism.

[0040] It can be seen that through the grasping mechanism control method and system based on visual positioning technology provided by the present invention, the motion control system can drive the pose adjustment mechanism to move close to the target. After reaching the set position, the visual measurement system is started for image acquisition measurement and pose solution. The visual measurement system feeds back the pose error between the mechanism and the target to the motion control system, and the motion control system can drive the pose adjustment mechanism to make another adjustment according to this error, so as to perform fast, accurate alignment and grasping in the case where the target is not easily observed manually. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0042] Figure 1 It is a schematic diagram of the composition of the control component architecture provided by the embodiment of the present invention;

[0043] Figure 2 It is a schematic diagram of the structure of the grasping mechanism control system based on visual positioning technology provided by the embodiment of the present invention;

[0044] Figure 3 It is a flowchart of the grasping mechanism control method based on visual positioning technology provided by the embodiment of the present invention;

[0045] Figure 4Schematic diagram of the principle of the visual grasping system provided by the embodiment of the present invention;

[0046] Figure 5 Schematic diagram of the camera installation layout provided by the embodiment of the present invention;

[0047] Figure 6 Schematic diagram of the target provided by the embodiment of the present invention;

[0048] Figure 7 Schematic diagram of the artificial target feature extraction effect provided by the embodiment of the present invention;

[0049] Figure 8 Flowchart of the monocular vision system provided by the embodiment of the present invention;

[0050] Figure 9 Schematic diagram of the pose adjustment parameter calculation provided by the embodiment of the present invention;

[0051] Figure 10 Schematic diagram of the main working process provided by the embodiment of the present invention;

[0052] Figure 11 Schematic diagram of the visual measurement guided pose adjustment process provided by the embodiment of the present invention. Detailed implementation manners

[0053] Hereinafter, exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be completely conveyed to those skilled in the art.

[0054] Specifically, before executing the grasping mechanism control method based on the visual positioning technology provided by the embodiment of the present invention, the system is built according to the Figure 1 control component architecture shown.

[0055] The control component architecture can include five levels, from top to bottom: the operation layer, the control layer, the drive layer, the execution layer, and the sensing layer.

[0056] a) The operation layer consists of a touch screen, which serves as a human-machine interface for data display and parameter setting, and has the advantages of flexible design, diverse functions, vivid and intuitive, etc., reducing the use of buttons, switches, etc.

[0057] b) The control layer consists of a graphics processor, a programmable logic controller (PLC), and the attached I / O module and communication module, which are used to generate control trajectory points.

[0058] c) The drive layer consists of multiple integrated servo drivers, which are used to convert control signals into drive voltage signals and can close the position loop, speed loop, and current loop by themselves to obtain more precise control.

[0059] d) The execution layer consists of multiple servo motors, which are used to output rotation.

[0060] e) The sensing layer consists of limit sensors, motor encoders, etc., which are used to feedback the position information of the devices in the execution layer to achieve position closed-loop control.

[0061] At the same time, based on Figure 2 , the present invention establishes a grasping mechanism control system based on visual positioning technology. The system can include: a motion control system, a visual measurement system, a posture adjustment mechanism, and a grasping target. Among them, the motion control system can drive the posture adjustment mechanism to move close to the target, and at the same time obtain the motion state of the mechanism through sensors. After the motion control system reaches the set position, it starts the visual measurement system to perform image acquisition measurement and pose calculation, and then feeds back the pose error between the mechanism and the target to the motion control system. The motion control system can drive the posture adjustment mechanism to make adjustments again according to this error.

[0062] Figure 3 The flowchart of the grasping mechanism control method based on visual positioning technology provided by the embodiment of the present invention is shown. Refer to Figure 3 , the grasping mechanism control method based on visual positioning technology provided by the embodiment of the present invention includes:

[0063] S1. After the motion control system controls the posture adjustment mechanism to reach the preset initial photographing position, it sends a in-place signal.

[0064] S2. The visual measurement system receives the in-place signal and performs image acquisition.

[0065] Specifically, after the motion control system controls the grasping mechanism to move to the preset initial photographing position, the motion control system issues an arrival signal, the visual measurement system works, acquires images, and the monitoring system works in real time to transmit image information back.

[0066] As an optional implementation manner of the embodiment of the present invention, the grasping mechanism control method based on visual positioning technology provided by the embodiment of the present invention further includes:

[0067] Set the visual measurement system according to a preset method. Among them, the visual measurement system includes four sets of monocular vision acquisition devices.

[0068] Specifically, when implemented, the visual measurement system can refer to Figure 4 and Figure 5Set in the manner shown. The present invention can adopt a large-range multi-viewpoint collaborative measurement technology. Four industrial area array cameras are installed on the front end surface of the grasping mechanism. Camera 1 and Camera 2 are installed at the positions shown in (1) above. The installation positions of the two cameras are symmetric with respect to the central axis plane, and the viewing angle included angle is about 45°; Camera 3 and Camera 4 are installed at the positions shown in (2) below. The installation positions of the two cameras are symmetric with respect to the central axis plane, and the viewing angle included angle is about 135°. When the grasping mechanism moves to the predetermined position, start the four cameras to quickly measure the pin hole features (artificially designed targets) on the target, so as to quickly position the grasping mechanism itself, and guide the posture adjustment mechanism to adjust the orientation of the grasping mechanism (6) to complete the alignment of the mechanism and the target object (7).

[0069] S3, the vision measurement system performs radial position measurement on the collected images and calculates the distance from the target;

[0070] S4, the vision measurement system determines whether the distance exceeds the preset range. If it exceeds the preset range, it notifies the motion control system. If it does not exceed the preset range, it extracts the target features of the collected images, performs pose calculation to obtain the calculation result, and generates a pose adjustment instruction according to the calculation result and sends it to the motion control system.

[0071] As an optional implementation manner of the embodiment of the present invention, extracting the target features of the collected images includes: extracting the preset pin hole features or the features of the artificial target preset in the target image. Among them, the artificial target is set as a dot array.

[0072] Specifically, the present invention uses vision measurement technology to solve the pose of the target. First, it is necessary to select appropriate extraction features. By obtaining the position of the features in the image and combining the calibration information, the current pose of the target object can be obtained. The image feature extraction accuracy determines the measurement accuracy. For this reason, as Figure 6 shown, the present invention designs an artificial target of a dot array. The circular marking points have the advantages of rotational invariance, affine invariance, convenient extraction, recognition, etc. The diameter of the circle is 1.5 mm, with a total of 11 rows and 11 columns. The range of the entire target is less than 60 mm × 60 mm. The background color of the target is a black matte material, and the dots are white fluorescent materials. When a light source is provided for illumination, the white dots have a high contrast in the image under the action of the fluorescent material, and the outline of the dots can be quickly extracted (as Figure 7 shown), and the sub-pixel dot centers are fitted, which can well adapt to the outdoor environment; there are four dots with coding rings in the center of the target for the identification and sorting of the dot array. Even if some points run out of the field of view, as long as three coding points can be seen, the calculation can be completed. Therefore, applying such a target can effectively ensure the extraction accuracy of the image center feature points and provide a basis for the subsequent measurement of the target.

[0073] As an alternative implementation of the embodiment of the present invention, obtaining the solution result through attitude calculation includes:

[0074] Calculating the three-dimensional coordinates of the target feature in each camera coordinate system;

[0075] According to the pre-calibrated conversion relationship between the coordinate system of the pose adjustment mechanism and the camera coordinate system, converting the three-dimensional coordinates of the target feature in each camera coordinate system to the coordinate system of the pose adjustment mechanism;

[0076] According to the standard docking feature points in the pre-calibrated coordinate system of the pose adjustment mechanism;

[0077] Calculating the rotation matrix R and the translation matrix T from the current position to the target position;

[0078] Decomposing the rotation matrix R to obtain the pitch angle, yaw angle, and roll angle;

[0079] Decomposing the translation matrix T to obtain the pose adjustment parameters of the three direction displacements.

[0080] Specifically, the monocular vision pose measurement method calculates through the object-image projection relationship of the camera. Usually, N feature marker points are set on the target object, that is, the coordinates of these points in the object coordinate system are known; then, the target object is photographed by the camera to obtain the image coordinates of N points on the target object. The pose of the target object in the camera coordinate system can be measured through the constraint relationship between the N points. This is the so-called PNP problem, that is, assuming that the camera is a pinhole model and has been calibrated, an image of N spatial points with known coordinates in the object coordinate system is taken, and the image plane coordinates of these N points are known, and the coordinates of these N spatial points in the camera coordinate system are determined. The prerequisite for this method is that the coordinates of the spatial points in the object coordinate system must be known in advance. The three-dimensional pose solution first preprocesses the image and extracts the target features (points or lines) on the image, then determines the corresponding relationship between them and the actual features on the target, and finally solves the pose parameters according to the obtained feature correspondence relationship. The technical process of monocular vision pose solution can be summarized as Figure 8 .

[0081] Before guiding the pose adjustment, it is necessary to calculate the pose adjustment parameters. The calculation principle is as Figure 9As shown, the three-dimensional coordinates of the circular array points on the target at the known target position in the attitude adjustment coordinate system are pre-known. The purpose of attitude adjustment is to make the circular array points on the target at the current position coincide with the target position points. Therefore, only by calculating the three-dimensional coordinates of the circular array points on the target at the current pose in the attitude adjustment mechanism coordinate system can the solution be obtained. Therefore, after the camera captures the coordinates of the circular array points on the target in the camera coordinate system, the circular array points at the current position can be converted from the camera coordinate system to the attitude adjustment mechanism coordinate system according to the hand-eye calibration parameters, and then the rotation matrix R and translation matrix T from the current position to the target position can be solved according to the corresponding points. By decomposing the R matrix, the pitch angle, yaw angle, and roll angle in the attitude adjustment parameters can be solved, and the three components in the matrix T are the displacement attitude adjustment parameters.

[0082] One of the key links in calculating the attitude adjustment parameters is to pre-calibrate the conversion relationship between the camera coordinate system and the attitude adjustment mechanism coordinate system. According to the present invention, the calibration process can be carried out by using classical methods. By moving the attitude adjustment mechanism, the camera is made to capture the target from different orientations, so as to solve the hand-eye parameters of the attitude adjustment mechanism coordinate system and the camera coordinate system.

[0083] S5. The motion control system controls the attitude adjustment mechanism to perform attitude adjustment according to the attitude adjustment instruction.

[0084] As an optional implementation manner of the embodiment of the present invention, generating an attitude adjustment instruction according to the solution result and sending it to the motion control system includes:

[0085] Judging whether the errors of the pitch angle, yaw angle, and roll angle are less than a preset threshold;

[0086] If it is greater than or equal to the preset threshold, an angle attitude adjustment instruction is generated and sent to the motion control system;

[0087] Judging whether the displacements in three directions need to be adjusted. If so, a displacement attitude adjustment instruction is generated and sent to the motion control system.

[0088] Specifically, the present invention can use a vision measurement system composed of four sets of monocular cameras to measure the radial position of the collected images, calculate the distance from the target, and if it exceeds the preset range, a warning signal is transmitted to the motion control system. The vision measurement system performs image positioning on the collected target features, solves the roll and axial pose information, and at the same time transmits the attitude adjustment instruction to the motion control system; the motion control system performs roll motion and axial motion according to the received attitude adjustment instruction, and transmits a in-place signal to the vision measurement system after the motion ends. The vision measurement system works and collects images; the above steps are repeated to iterate the pose data until the solved pose data is less than the preset threshold, and the attitude adjustment process ends.

[0089] S6. The motion control system controls the grasping mechanism to perform grasping.

[0090] Specifically, the present invention can use the deviation between the obtained coordinates of the adjustment mechanism and the target position coordinates as the action amount of the adjustable mechanism to adjust the mechanism, and then control the actions of mechanisms (3) and (4) to complete the grasping and fastening of the target, that is, complete the control process of the entire grasping mechanism.

[0091] When specifically implemented, reference can be made to Figure 10 and Figure 11 the specific examples shown for implementation:

[0092] As an alternative implementation manner of an embodiment of the present invention, the control method for a grasping mechanism based on visual positioning technology provided by the embodiment of the present invention further includes: the visual measurement system receives the feedback information of the posture adjustment result, determines whether the posture adjustment is successful, and if not, interrupts the posture adjustment and reports an abnormality. And the visual measurement system determines whether the maximum number of posture adjustment times is exceeded, and if so, interrupts the posture adjustment and reports an abnormality.

[0093] As an alternative implementation manner of an embodiment of the present invention, the control method for a grasping mechanism based on visual positioning technology provided by the embodiment of the present invention further includes:

[0094] The motion control system controls the posture adjustment mechanism to advance a preset distance;

[0095] The visual measurement system determines whether the positioning features are collected;

[0096] If the positioning features are not collected, the motion control system determines whether the safety limit is reached. If the safety limit is not reached, the operation of the motion control system controlling the posture adjustment mechanism to advance a preset distance is returned until the positioning features are collected; if the safety limit is reached, the motion control system controls the posture adjustment mechanism to stop moving.

[0097] It can be seen that through the control method for a grasping mechanism based on visual positioning technology provided by the embodiment of the present invention, the motion control system can drive the posture adjustment mechanism to move close to the target. After reaching the set position, the visual measurement system is started for image acquisition measurement and attitude calculation. The visual measurement system feeds back the attitude error between the mechanism and the target to the motion control system, and the motion control system can drive the posture adjustment mechanism to make adjustments again according to this error, enabling fast, accurate alignment and grasping in the case where the target is not easily observable manually.

[0098] The present invention also provides a control system for a grasping mechanism based on visual positioning technology. This control system for a grasping mechanism based on visual positioning technology applies the above method and uses the above control method for a grasping mechanism based on visual positioning technology to control the grasping mechanism.

[0099] It can be seen that through the grasping mechanism control system based on the vision positioning technology provided by the embodiments of the present invention, the motion control system can drive the posture adjustment mechanism to move close to the target. After reaching the set position, the vision measurement system is started to perform image acquisition measurement and posture solution. The vision measurement system feeds back the posture error between the mechanism and the target to the motion control system, and the motion control system can drive the posture adjustment mechanism to make adjustments again according to this error, so as to perform fast and accurate alignment and grasping in the case where the target is not easily observed manually.

[0100] The above are only the embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A control method for a grasping mechanism based on visual positioning technology, characterized in that Including: After the motion control system controls the posture adjustment mechanism to reach the preset initial photographing position, it sends a signal indicating that it has reached the position. The vision measurement system receives the signal indicating that it has reached the position and performs image acquisition. The vision measurement system measures the radial position of the acquired image and calculates the distance to the target. The vision measurement system determines whether the distance exceeds a preset range. If it exceeds the preset range, it notifies the motion control system. If it does not exceed the preset range, it extracts the target features of the acquired image, performs posture calculation to obtain a calculation result, and generates a posture adjustment instruction according to the calculation result and sends it to the motion control system. The motion control system controls the posture adjustment mechanism to perform posture adjustment according to the posture adjustment instruction. The motion control system controls the grasping mechanism to perform grasping.

2. The method according to claim 1, wherein It also includes: The vision measurement system is set in a preset manner, where the vision measurement system includes four sets of monocular vision acquisition devices.

3. The method according to claim 2, wherein The extraction of the target features of the acquired image includes: Extracting the preset pin hole features or the features of the artificial target set in the target image.

4. The method according to claim 3, wherein The artificial target is set as a dot array.

5. The method according to claim 4, wherein The performing of posture calculation to obtain a calculation result includes: Calculating the three-dimensional coordinates of the target features in each camera coordinate system. According to the pre-calibrated conversion relationship between the posture adjustment mechanism coordinate system and the camera coordinate system, converting the three-dimensional coordinates of the target features in each camera coordinate system to the posture adjustment mechanism coordinate system. According to the standard docking feature points in the pre-calibrated posture adjustment mechanism coordinate system. Calculating the rotation matrix R and the translation matrix T from the current position to the target position. Decomposing the rotation matrix R to obtain the pitch angle, yaw angle, and roll angle. Decomposing the translation matrix T to obtain the three-direction displacement adjustment parameters.

6. The method according to claim 5, characterized in that, The generating of the posture adjustment instruction according to the calculation result and sending it to the motion control system includes: Judging whether the errors of the pitch angle, yaw angle, and roll angle are less than a preset threshold. If it is greater than or equal to the preset threshold, generating an angle posture adjustment instruction and sending it to the motion control system. Judging whether the displacements in the three directions need to be adjusted. If so, generating a displacement posture adjustment instruction and sending it to the motion control system.

7. The method according to claim 6, characterized in that, It also includes: The vision measurement system receives the feedback information of the posture adjustment result, judges whether the posture adjustment is successful. If it is not successful, it interrupts the posture adjustment and reports an exception.

8. The method according to claim 7, wherein It also includes: The vision measurement system judges whether the maximum number of posture adjustments is exceeded. If so, it interrupts the posture adjustment and reports an exception.

9. The method according to claim 8, wherein It also includes: The motion control system controls the posture adjustment mechanism to advance a preset distance. The vision measurement system judges whether the positioning features are acquired. If the positioning features are not acquired, the motion control system judges whether the safety limit is reached. If the safety limit is not reached, it returns to execute the operation of the motion control system controlling the posture adjustment mechanism to advance the preset distance until the positioning features are acquired. If the safety limit is reached, the motion control system controls the posture adjustment mechanism to stop moving.

10. A control system for a grasping mechanism based on visual positioning technology, characterized in that, The grasping mechanism control method based on vision positioning technology as described in any one of claims 1 to 9 is adopted for grasping mechanism control.

Images