Novel end effector device based on visual positioning of tool changing manipulator

Abstract

The invention relates to the technical field of full-face tunnel boring machines, in particular to a novel end effector device based on tool changing manipulator visual positioning, which comprises a tool box, a tool rest, a tool changing manipulator and a base, and further comprises a device body, the device body comprises a forward vision positioning system, a backward vision positioning system, a bolt sleeve mechanism, a gear box and a clamping mechanism, the gear box is connected with the tool changing manipulator, the forward vision positioning system is installed on one side of the gear box, the backward vision positioning system is installed on one side of the gear box, the bolt sleeve mechanism is connected with the gear box, and the clamping mechanism is connected with the gear box. The clamping mechanism is connected with the gear box, a machine vision recognition positioning system algorithm based on target point cloud processing is achieved, bolt positioning is completed in combination with a designed end effector with a bidirectional bolt sleeve, the working efficiency and safety of automatic operation of a mechanical arm in shield cutter changing operation are improved, shield construction automation is promoted, and the labor intensity of workers is lowered. And the shield construction is realized by fewer people.

Description

Technical Field

[0001] The present invention relates to the technical field of full-face tunnel boring machines, and particularly to a novel end effector device based on vision positioning of a tool-changing manipulator. Background Art

[0002] Full-face rock tunnel boring machines are widely used in various tunnel construction projects due to their high tunneling efficiency, good safety performance, and low construction costs. During the tunneling process of a TBM, the disc cutters on the cutterhead are mainly used to cut soil and break rocks. Therefore, the disc cutters are under high stress, alternating loads, corrosion, and other working conditions for a long time. When the disc cutters serve under such harsh working conditions for a long time, they are prone to failure. Common failure modes include bearing damage, cutter ring fracture, cutter ring spalling, and cutter ring deformation, etc. During the tunneling process of a TBM, rock breaking mainly relies on the disc cutters on the cutterhead. Under the strong interaction with rock and soil, the disc cutters are extremely prone to wear and failure. As an important component of a TBM, once the disc cutters fail, the TBM must stop to replace the disc cutters, which will have a serious adverse impact on the tunneling efficiency, construction period, and cost of the tunnel. At present, most of the TBM cutter replacements are carried out manually. The cutter-changing environment is extremely harsh. When the cutter-changing workers carry out cutter-changing work underground, they have to face dangers such as mud and water, rock bursts, high temperature, high pressure, and harmful gases, and safety accidents are extremely likely to occur. Safety accidents caused by cutter-changing operations have become the main source of safety accidents during TBM construction. The failed disc cutters need to be replaced in time to ensure the normal tunneling of the TBM. However, at present, the replacement of failed disc cutters of TBMs almost entirely relies on manual labor, and manual replacement will bring a series of negative problems, including the safety of construction workers, low efficiency, and high cost, etc. Therefore, using robots for safe and efficient replacement of failed disc cutters is a current research hotspot. However, during the normal operation of the existing TBM tool-changing manipulator through the provided robotic arm, the error of the control mechanism of the entire robotic arm during end positioning is very large. Manual assistance is also required for the identification and positioning of the disc cutters, and it is difficult to be applied in large-scale projects. Therefore, the research on the end positioning of TBM tool-changing manipulators is of great importance. Summary of the Invention

[0003] The purpose of the present invention is to provide a novel end effector device based on vision positioning of a tool-changing manipulator, which can complete bolt positioning by combining a machine vision recognition and positioning system algorithm based on target point cloud processing with the designed end effector having a two-way bolt sleeve. At the same time, a method for replacing disc cutters of a tool-changing manipulator based on end vision positioning is proposed to improve the efficiency and safety of shield cutter-changing operations, promote the automation of shield construction, and realize less manpower in shield construction.

[0004] To achieve the above object, the present invention provides a novel end effector device based on the visual positioning of a tool-changing manipulator, including a tool magazine, a tool holder, a tool-changing manipulator, and a base. The tool magazine is installed on the tool holder, and the tool-changing manipulator is installed on the base. The device also includes a device body; The device body includes a forward visual positioning system, a backward visual positioning system, a bolt sleeve mechanism, a gearbox, and a clamping mechanism. The gearbox is connected to the tool-changing manipulator and is located on one side of the tool-changing manipulator. The backward visual positioning system is used to complete the working positioning during the process of removing bolts, the bolt sleeve mechanism is used to remove the bolts to be removed, so as to facilitate the subsequent removal of the hob, the forward visual positioning system is used to complete the working positioning during the process of grasping the hob, and the clamping mechanism is used to complete the clamping and removal of the hob.

[0005] Among them, the forward visual positioning system includes a front protective cover housing, a front bottom plate, a front system sealing mechanism, and a front image acquisition and processing mechanism. The front protective cover housing is connected to the gearbox and is located on one side of the gearbox; the front bottom plate is connected to the front protective cover housing and is located at the bottom of the front protective cover housing; the front system sealing mechanism is connected to the front protective cover housing and is used to achieve the sealing protection of the corresponding installation space; the front image acquisition and processing mechanism is connected to the front bottom plate and is used to complete the acquisition and processing of the corresponding positioning image information.

[0006] Among them, the front system sealing mechanism includes a front sealing joint, a front cover plate, and a front glass window. The front sealing joint is connected to the front protective cover housing and is located on one side of the front protective cover housing; the front cover plate is connected to the front protective cover housing and is located on one side of the front protective cover housing; the front glass window is connected to the front protective cover housing and is located on one side of the front protective cover housing.

[0007] Among them, the front image acquisition and processing mechanism includes a front 3D camera, a front 2D camera, a front temperature control module, and a front image processing module. The front 3D camera is connected to the front bottom plate and is located inside the front protective cover housing; the front 2D camera is connected to the front bottom plate and is located inside the front protective cover housing; the front temperature control module is connected to the front protective cover housing and is located inside the front protective cover housing. The front image processing module is electrically connected to both the front 3D camera and the front 2D camera and is electrically connected to the tool-changing manipulator. The main processing process of the front image processing module includes: obtaining the point cloud of the TBM tool magazine, segmenting the components of the point cloud, extracting the point cloud of the feature surface, fitting the point cloud plane, and determining the position coordinates of the tool magazine positioning reference point.

[0008] Among them, the rear vision positioning system includes a rear protective cover housing, a rear base plate, a rear system sealing mechanism, and a rear image acquisition and processing mechanism. The rear protective cover housing is connected to the gearbox and is located on one side of the gearbox; the rear base plate is connected to the rear protective cover housing and is located at the bottom of the rear protective cover housing; the rear system sealing mechanism is connected to the rear protective cover housing and is used to achieve the sealing protection of the corresponding installation space; the rear image acquisition and processing mechanism is connected to the rear base plate and is used to complete the acquisition and processing of the corresponding positioning image information.

[0009] Among them, the rear system sealing mechanism includes a rear sealing joint, a rear cover plate, and a rear glass window. The rear sealing joint is connected to the rear protective cover housing and is located on one side of the rear protective cover housing; the rear cover plate is connected to the rear protective cover housing and is located on one side of the rear protective cover housing; the rear glass window is connected to the rear protective cover housing and is located on one side of the rear protective cover housing.

[0010] Among them, the rear image acquisition and processing mechanism includes a rear 3D camera, a rear 2D camera, a rear temperature control module, and a rear image processing module. The rear 3D camera is connected to the rear base plate and is located inside the rear protective cover housing; the rear 2D camera is connected to the rear base plate and is located inside the rear protective cover housing; the rear temperature control module is connected to the rear protective cover housing and is located inside the rear protective cover housing. The rear image processing module is electrically connected to both the rear 3D camera and the rear 2D camera and is electrically connected to the tool change robot.

[0011] Among them, the bolt sleeve mechanism includes a sleeve wrench, a driven gear, a bearing, and a driving component. The sleeve wrench is connected to the gearbox through the bearing and is located on one side of the gearbox; the driven gear is connected to the sleeve wrench and is sleeved on the sleeve wrench; the bearing is connected to the sleeve wrench and is installed inside the gearbox; the driving component is connected to the gearbox.

[0012] Among them, the driving component includes a driving gear and a hydraulic motor. The driving gear meshes with the driven gear and is rotatably installed inside the gearbox; the output shaft of the hydraulic motor is connected to the driving gear, and the hydraulic motor is installed inside the gearbox.

[0013] Among them, the sleeve wrench includes a sleeve front cover and a sleeve core shaft. The sleeve wrench is composed of the sleeve front cover and the sleeve core shaft, and the structures at both ends of the sleeve core shaft are internal hexagon wrenches.

[0014] Among them, the clamping mechanism includes a secondary telescopic arm, a jaw connecting block, and jaws. The secondary telescopic arm is connected to the gearbox and is located on one side of the gearbox; the jaw connecting block is connected to the secondary telescopic arm and is located on one side of the secondary telescopic arm; the jaws are connected to the jaw connecting block and are located on one side of the jaw connecting block.

[0015] A novel end effector device based on vision positioning of a tool-changing manipulator according to the present invention can, according to the integrated tool system of different shield machines and the end positioning of the tool-changing manipulator, when disassembling bolts, first drive the device body to the area of the hob to be replaced by the tool-changing manipulator, so that the rear vision positioning system is directly opposite the center of the tool box of the hob, complete the end positioning through the proposed TBM tool box positioning algorithm based on point cloud processing, and then control the device body to move up and down to align with the bolts, and then complete the disassembly of the bolts through the provided bolt sleeve mechanism. When grasping the hob, after completing the disassembly of the bolts of the integrated tool system, the tool-changing manipulator controls the device body to turn, so that the front vision positioning system is directly opposite the center of the tool box of the hob. After obtaining the relative position between the end and the tool box at this moment by using the TBM tool box positioning algorithm based on point cloud processing, adjust the height of the device body so that the clamping mechanism aligns with the position of the hob, and then complete the clamping of the hob, realizing a machine vision recognition and positioning system algorithm based on target point cloud processing, combining the designed end effector with a two-way bolt sleeve to complete bolt positioning, and at the same time proposing a method for replacing the hob of the tool-changing manipulator based on end vision positioning to improve the efficiency and safety of shield tool changing operations, promote shield construction automation, and realize less manpower in shield construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art.

[0017] Figure 1 It is a schematic structural diagram of the overall novel end effector device based on vision positioning of a tool-changing manipulator according to the first embodiment of the present invention.

[0018] Figure 2 It is a schematic structural diagram of the device body according to the first embodiment of the present invention.

[0019] Figure 3 It is a schematic structural diagram of the front vision positioning system according to the first embodiment of the present invention.

[0020] Figure 4 It is a schematic internal structural diagram of the front vision positioning system according to the first embodiment of the present invention.

[0021] Figure 5It is a schematic structural diagram of the backward vision positioning system according to the first embodiment of the present invention.

[0022] Figure 6 It is a schematic internal structural diagram of the backward vision positioning system according to the first embodiment of the present invention.

[0023] Figure 7 It is a schematic structural diagram of the overall new end effector device based on the vision positioning of the tool changing manipulator according to the second embodiment of the present invention.

[0024] Figure 8 It is a schematic structural diagram of the driving component according to the second embodiment of the present invention.

[0025] Figure 9 It is a schematic structural diagram of the overall new end effector device based on the vision positioning of the tool changing manipulator according to the third embodiment of the present invention.

[0026] Figure 10 It is a schematic installation structure diagram of the jaw according to the third embodiment of the present invention.

[0027] In the figure: 1 - tool magazine, 2 - tool holder, 3 - tool changing manipulator, 4 - base, 5 - forward vision positioning system, 6 - backward vision positioning system, 7 - bolt sleeve mechanism, 8 - gearbox, 9 - gripping mechanism, 51 - front sealing joint, 52 - front protective cover housing, 53 - front bottom plate, 54 - front glass window, 55 - front cover plate, 56 - front 3D camera, 57 - front 2D camera, 58 - front temperature control module, 61 - rear sealing joint, 62 - rear protective cover housing, 63 - rear bottom plate, 64 - rear glass window, 65 - rear cover plate, 66 - rear 3D camera, 67 - rear 2D camera, 68 - rear temperature control module, 71 - socket wrench, 72 - driven gear, 73 - bearing, 74 - driving gear, 75 - hydraulic motor, 711 - socket front cover, 712 - socket mandrel, 91 - secondary telescopic arm, 92 - jaw connecting block, 93 - jaw. Detailed implementation manners

[0028] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the drawings. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0029] The first embodiment of the present application is as follows: Please refer to Figures 1 to 6 , where Figure 1 is a schematic structural diagram of the overall new end effector device based on the vision positioning of the tool changing manipulator, Figure 2 is a schematic structural diagram of the device body, Figure 3 is a schematic structural diagram of the forward vision positioning system 5, Figure 4 is a schematic internal structural diagram of the forward vision positioning system 5,Figure 5 is a schematic structural diagram of the rear vision positioning system 6, Figure 6 is a schematic internal structure diagram of the rear vision positioning system 6.

[0030] The present invention provides a novel end effector device based on the vision positioning of a tool-changing manipulator, which includes a tool magazine 1, a tool holder 2, a tool-changing manipulator 3, a base 4 and a device body. The device body includes a forward vision positioning system 5, a rear vision positioning system 6, a bolt sleeve mechanism 7, a gearbox 8 and a clamping mechanism 9. The forward vision positioning system 5 includes a front sealing joint 51, a front protective cover housing 52, a front bottom plate 53, a front glass window 54, a front cover plate 55, a front 3D camera 56, a front 2D camera 57 and a front temperature control module 58. The rear vision positioning system 6 includes a rear sealing joint 61, a rear protective cover housing 62, a rear bottom plate 63, a rear glass window 64, a rear cover plate 65, a rear 3D camera 66, a rear 2D camera 67 and a rear temperature control module 68. Through the foregoing solution, it is solved that at present, most of the tool changing of TBM is carried out manually, the tool-changing environment is extremely harsh. When the tool-changing workers carry out tool-changing work underground, they have to face dangers such as mud and water, rock burst, high temperature, high pressure and harmful gases, and safety accidents are extremely likely to occur. Safety accidents caused by tool-changing operations have become the main source of safety accidents in the process of TBM construction. The failed cutterhead needs to be replaced in time to ensure the normal tunneling of the TBM. However, at present, the replacement of the failed cutterhead of the TBM almost entirely relies on manual labor, and manual replacement will bring a series of negative problems, including the safety of construction personnel, low efficiency and high cost, etc. Therefore, using a robot to safely and efficiently replace the failed cutterhead is a current research hotspot, but the end positioning error of the existing TBM tool-changing manipulator 3 is very large, and manual assistance is still required in the identification and positioning of the cutterhead, and it is difficult to carry out large-scale engineering applications.

[0031] In this embodiment, the tool magazine 1 is installed on the tool holder 2, the tool-changing manipulator 3 is installed on the base 4, and the device body is applied to a platform built by the tool magazine 1, the tool holder 2, the tool-changing manipulator 3 and the base 4.

[0032] Among them, the gearbox 8 is connected to the tool changing manipulator 3 and is located on one side of the tool changing manipulator 3. The forward vision positioning system 5 is installed on one side of the gearbox 8, and the rearward vision positioning system 6 is installed on one side of the gearbox 8. The bolt sleeve mechanism 7 is connected to the gearbox 8, and the clamping mechanism 9 is connected to the gearbox 8. The gearbox 8 is fixed to the adjustment output end of the tool changing manipulator 3 so that the tool changing manipulator 3 can drive the gearbox 8 and the mechanisms provided on the gearbox 8 to move. The forward vision positioning system 5 and the rearward vision positioning system 6 form a two-way vision positioning system. The forward vision positioning system 5 is used for positioning when grasping the hob, and the rearward vision positioning system 6 is used for positioning when disassembling the bolt. The specific positioning coordinates are indirectly located by identifying the position of the feature surface 12 of the tool magazine 1 and the coordinate of the bolt 13.

[0033] Secondly, the front protective cover housing 52 is connected to the gearbox 8 and is located on one side of the gearbox 8; the front sealing joint 51 is connected to the front protective cover housing 52 and is located on one side of the front protective cover housing 52; the front cover plate 55 is connected to the front protective cover housing 52 and is located on one side of the front protective cover housing 52; the front bottom plate 53 is connected to the front protective cover housing 52 and is located at the bottom of the front protective cover housing 52; the front glass window 54 is connected to the front protective cover housing 52 and is located on one side of the front protective cover housing 52. The front 3D camera 56 is connected to the front bottom plate 53 and is located inside the front protective cover housing 52; the front 2D camera 57 is connected to the front bottom plate 53 and is located inside the front protective cover housing 52; the front temperature control module 58 is connected to the front protective cover housing 52 and is located inside the front protective cover housing 52. The forward vision positioning system 5 includes the front 3D camera 56, two front 2D cameras 57, the front temperature control module 58, the front sealing joint 51, the front protective cover housing 52, the front bottom plate 53, the front glass window 54 and the front cover plate 55. The forward vision positioning system 5 is fixed to the gearbox 8 through two threaded holes. The front temperature control module 58 is fixed on the inner side wall of the front protective cover housing 52 to adjust the temperature of the heating strip, achieving the dehumidification effect while preventing the equipment from being damaged due to excessive temperature; by setting the front sealing joint 51, the internal device can work normally in the tunnel environment with high humidity and high pressure; the front bottom plate 53 is fixed to the bottom of the front protective cover, and the upper surface of the front bottom plate 53 is fixed to the front 3D camera 56 to enhance the heat dissipation effect of the front 3D camera 56; the two front 2D cameras 57 are respectively fixed to the front bottom plate 53 through connecting plates to obtain the front and rear video streams.

[0034] Meanwhile, the rear protective cover housing 62 is connected to the gearbox 8 and is located on one side of the gearbox 8; the rear sealing joint 61 is connected to the rear protective cover housing 62 and is located on one side of the rear protective cover housing 62; the rear cover plate 65 is connected to the rear protective cover housing 62 and is located on one side of the rear protective cover housing 62; the rear bottom plate 63 is connected to the rear protective cover housing 62 and is located at the bottom of the rear protective cover housing 62; the rear glass window 64 is connected to the rear protective cover housing 62 and is located on one side of the rear protective cover housing 62, the rear 3D camera 66 is connected to the rear bottom plate 63 and is located inside the rear protective cover housing 62; the rear 2D camera 67 is connected to the rear bottom plate 63 and is located inside the rear protective cover housing 62; the rear temperature control module 68 is connected to the rear protective cover housing 62 and is located inside the rear protective cover housing 62. The rear vision positioning system 6 is similar in structure to the front vision positioning system 5. The rear vision positioning system 6 includes the rear 3D camera 66, the rear 2D camera 67, the rear temperature control module 68, the rear sealing joint 61, the rear protective cover housing 62, the rear bottom plate 63, the rear glass window 64 and the rear cover plate 65; the rear vision positioning system 6 is fixed to the bottom of the gearbox 8 through four threaded holes at the top, the rear temperature control module 68 is fixed to the inner side wall of the rear protective cover housing 62; the rear bottom plate 63 is fixed to the bottom of the rear protective cover, and the upper surface of the rear bottom plate 63 is fixed to the rear 3D camera 66; the rear 2D camera 67 is fixed to the rear bottom plate 63 through a connecting plate for acquiring a video stream.

[0035] The key to the operation of the tool-changing manipulator 3 for replacing the hob lies in accurately positioning the tightening bolts for the fixed hob. The corresponding 3D camera can only obtain the point cloud information within the camera's field of view through photographing. To finally obtain the three-dimensional coordinate information of the bolt position, a series of point cloud and image processing algorithms need to be used to process the point cloud data and image data. Finally, the coordinate information of the bolt is transmitted to the robot controller. Since the end area of the bolt is too small, there is less point cloud information available for direct positioning and the positioning is inaccurate. Considering comprehensively, an indirect positioning method is adopted. The point cloud information of the rectangular area at the feature surface 12 with a larger area is segmented from the point cloud information, and then the center point coordinates of this area of the point cloud are fitted. Combining the fixed geometric relationship between the bolt position and the center point coordinates of the feature surface, the three-dimensional coordinate information of the bolt is indirectly obtained through calculation and transformation. This series of processing processes includes using advanced image processing and deep learning methods for instance segmentation of the two-dimensional image feature surface area, alignment of the two-dimensional image and the point cloud, denoising the point cloud using a denoising algorithm, filtering out interference points and retaining high-quality point cloud data, extracting the point cloud data of the target area, separating the point cloud of the feature surface area from the complete point cloud data, point cloud fitting and other processes. Based on the transformation matrix of the fixed geometric relationship, the three-dimensional coordinates of the center of the target bolt are finally indirectly obtained. In addition, the 3D camera used in the above two systems uses stripe structured light coding to achieve high-precision three-dimensional reconstruction and is suitable for visual measurement of weak texture targets such as the tool magazine 1; the 2D camera used above is an ordinary color industrial camera for real-time monitoring and observation. At the same time, the pre-image processing module and the post-image processing module provided by the present invention have the same processing method, which is the specific process of the TBM tool magazine 1 positioning algorithm based on point cloud processing, mainly including: obtaining the point cloud of the TBM tool magazine 1; performing component segmentation on the point cloud and extracting the point cloud of the feature surface; fitting the point cloud plane to determine the position coordinates of the positioning reference point of the tool magazine 1. Taking the replacement of the positive hob of the cutter head as an example, the positioning algorithm of the tool-changing manipulator 3 can be divided into the following steps: 1. Disparity map generation and point cloud acquisition: In terms of disparity map generation, the RAFT-Stereo model, which has a relatively lightweight overall model, does not require a complex loss function, and is suitable for high-resolution images, is used for disparity map generation. It mainly consists of three parts: a feature extractor, a correlation pyramid, and a multi-level disparity update module; according to the obtained disparity map, using the principle of triangulation and the reprojection matrix, an initial point cloud map is generated, and point cloud denoising and point cloud registration are performed on it. The point cloud sample of the TBM tool magazine 1 and the scaled hob model obtained by the above method; 2. Target point cloud segmentation: The original input point cloud is marked as N, which consists of the coordinates of the points and the features of the corresponding points Composition. The farthest point sampling (FPS) algorithm can select a certain number of sampling centroid points from the point cloud, and the sampling points will be evenly discrete, covering the entire point cloud as much as possible. Therefore, FPS sampling is performed on the initial input point cloud to obtain point cloud samples with N1, N2, and N3 points, denoted as S1, S2, and S3 respectively, and the resolutions of these three samples are 2048, 1024, and 512 respectively. Then, using the ball query method, the neighboring points around each sampling centroid point are determined within a spherical space of a certain radius.

[0036] After obtaining the point cloud samples with different resolutions, it is necessary to construct a point embedding module to aggregate the local neighborhood features of the input points of the point cloud. Max pooling and average pooling both show good performance in aggregating local features and do not incur a large computational cost. After comprehensive comparison, we choose to use the max pooling method for point embedding to enhance the local feature extraction ability of the overall model architecture.

[0037] After performing point embedding, apply the point cloud Transformer to encode the feature information of the point clouds on the three paths respectively by combining linear layers and pooling layers, etc. In the Transformer, the attention mechanism adopts the offset attention mechanism to improve the segmentation performance of the Transformer for point clouds and obtain better segmentation results.

[0038] In offset attention, first, let Q, K, and V represent the query, key, and value matrices respectively, and perform a linear transformation on the input point cloud features to obtain Q, K, and V. The specific formulas are as follows:

[0039] where , , are weight matrices that can be updated through learning, is the dimension of the Q and K vector matrices. It should be noted that and the input feature generally have unequal dimensions D. After calculating the Q, K, and V matrices, calculate the attention feature matrix through matrix dot product. The formula is as follows: where represents the transpose matrix of the matrix , represents performing softmax and and processing on the matrix obtained by the dot product of , and then multiplying it with the matrix Perform dot multiplication to finally obtain the attention feature matrix.

[0040] Laplacian matrix Exhibits excellent processing performance in the graph convolutional network structure, where is the diagonal matrix of the processed image, is the adjacency matrix of the image, is the number of given vertices in the image. Apply the Laplacian matrix algorithm mechanism to the self-attention mechanism to form the offset attention mechanism. The offset attention mechanism calculates the offset between the input feature and the self-attention feature using matrix subtraction, then performs operation on the offset matrix, and finally sums it with the input feature to obtain the output feature , and the specific formula is as follows: where is to perform the offset attention mechanism operation on the input feature, is equivalent to the Laplacian offset operator.

[0041] In addition, in Transformer, it is necessary to perform position encoding on the points in the point cloud sample, which contains the spatial position information of the point cloud. The position encoding function is as follows: where , respectively represent the and spatial position coordinates of the th and th points, and

[0042] After performing point cloud Transformer on three point cloud samples with different resolutions respectively, three different-scale feature information containing global context features is obtained. In order to aggregate features of different scales, it is necessary to perform cross-scale fusion on the features of the three paths to obtain the final feature output. Combining with the self-attention mechanism model, the cross-scale fusion algorithm is as follows: where represents the final weighted point cloud feature, , respectively represent and operations, , , respectively represent the point cloud Transformer output feature matrices obtained from each path, while represents the transposed matrix of the point cloud Transformer output feature. After obtaining the final feature matrix the point cloud global feature is obtained through max pooling.

[0043] To perform the point cloud segmentation task, the number dimension of the global feature is repeated to be consistent with the cross-scale feature, and then it is concatenated with the cross-scale feature matrix to obtain the concatenated matrix , and then the following operations are performed: where represents the final point cloud segmentation score, is the set feature score dimension, represents appending an operation after the

[0044] 3. Feature Fitting and Spatial Positioning To perform the final positioning operation, an inherent feature of the object to be recognized needs to be set in advance, and the position of the object is indirectly determined by determining the spatial position of this feature.

[0045] After segmenting the input point cloud of the entire object, the point cloud of the set inherent feature is extracted, the plane is fitted using the least squares method, and then the plane features such as corner coordinates and edge lines are calculated. Then the position of the feature plane is determined, and according to the relative position relationship between the feature plane and the object, the spatial position of the object is indirectly determined to complete the positioning work.

[0046] Integrating the above steps, the machine vision recognition and positioning system algorithm based on target point cloud processing cooperates with the TBM tool-changing manipulator 3 to complete the automatic tool-changing operation. First, the target image is obtained through the binocular camera, and the disparity map is calculated using the RAFT-Stereo algorithm to generate the target point cloud. Secondly, the point cloud is encoded based on the point cloud Transformer, and the offset attention mechanism is used therein. Then the point clouds with different resolutions are fused across scales to aggregate features of different scales, and finally a point cloud with good segmentation effect is obtained. The feature point cloud is extracted and fitted to obtain the coordinates of the positioning reference point, thus completing the positioning work. Finally, the positioning error is calculated. Experiments prove that the positioning algorithm used in this paper has a good positioning effect, and the positioning errors in the X and Y directions are both less than 0.4 mm, and the positioning error in the depth Z direction is less than 1.7 mm, meeting the requirements of engineering applications.

[0047] When a new end effector device based on the vision positioning of a tool-changing manipulator in this embodiment is in use, it can be positioned at the end of the integrated tool system of different shield machines and the tool-changing manipulator 3. When disassembling bolts, first, the tool-changing manipulator 3 drives the device body to the area where the hob needs to be replaced, so that the rear vision positioning system 6 is directly opposite the center of the tool box 1 of the hob. The end positioning is completed through the proposed TBM tool box 1 positioning algorithm based on point cloud processing. Then, the device body is controlled to move up and down to align with the bolts, and then the bolt sleeve mechanism 7 is used to complete the disassembly of the bolts. When grasping the hob, after the disassembly of the bolts of the integrated tool system is completed, the tool-changing manipulator 3 controls the device body to turn, so that the front vision positioning system is directly opposite the center of the tool box 1 of the hob. After obtaining the relative position between the end and the tool box 1 at this moment by using the TBM tool box 1 positioning algorithm based on point cloud processing, the height of the device body is adjusted so that the clamping mechanism 9 is aligned with the position of the hob, and then the hob is clamped. The machine vision recognition and positioning system algorithm based on target point cloud processing is realized. Combined with the designed end effector with a two-way bolt sleeve, the bolt positioning is completed. At the same time, a method for replacing the hob of the tool-changing manipulator 3 based on end vision positioning is proposed to improve the efficiency and safety of the shield tool-changing operation, promote the automation of shield construction, and realize the reduction of the number of personnel in shield construction.

[0048] Second Embodiment: Please refer to Figure 7 and Figure 8 , Figure 7 which is a schematic structural diagram of the overall new end effector device based on the vision positioning of the tool-changing manipulator for the second embodiment. Figure 8 which is a schematic structural diagram of the driving component. The bolt sleeve mechanism 7 provided by the present invention includes a socket wrench 71, a driven gear 72, a bearing 73 and a driving component. The driving component includes a driving gear 74 and a hydraulic motor 75. The socket wrench 71 includes a socket front cover 711 and a socket mandrel 712.

[0049] Among them, the socket wrench 71 is connected to the gearbox 8 through the bearing 73 and is located on one side of the gearbox 8; the driven gear 72 is connected to the socket wrench 71 and sleeved on the socket wrench 71; the bearing 73 is connected to the socket wrench 71 and installed in the gearbox 8; the driving component is connected to the gearbox 8, the driving gear 74 meshes with the driven gear 72 and is rotatably installed in the gearbox 8; the output shaft of the hydraulic motor 75 is connected to the driving gear 74, the hydraulic motor 75 is installed in the gearbox 8, the socket wrench 71 is composed of a socket front cover 711 and a socket mandrel 712, both ends of the socket mandrel 712 are structured as internal hexagon wrenches, the end of the socket front cover 711 is designed with an inclination angle, which can accommodate radial and circumferential errors. There are two groups of the socket wrenches 71 in total, and the driven gears 72 are arranged on both groups of the socket wrenches 71. The two driven gears 72 are driven by the provided driving gear 74, and the driving gear 74 is driven by the hydraulic motor 75, thereby completing the synchronous driving of the two groups of the socket wrenches 71. Both sides of the gearbox 8 are sealed through the fixed bearing 73 end covers and connected to the socket wrench 71. The hydraulic motor 75 drives the driving gear 74 to drive the two driven gears 72 and the two socket wrenches 71 to rotate to remove bolts. The socket wrench 71 and the driven gear 72 are driven by splines. The hydraulic motor 75 is connected to the driving gear 74 through a spline shaft. At the same time, the socket wrench 71 contacts the gearbox 8 through the bearing 73 to realize the free rotation of the socket wrench 71.

[0050] When using the novel end effector device based on the visual positioning of the tool-changing manipulator of this embodiment, the driving gear 74 can be driven by the hydraulic motor 75, and then the two driven gears 72 and the two socket wrenches 71 are driven by the driving gear 74 to rotate, and then the corresponding bolts are removed by driving the two socket wrenches 71, greatly enhancing the practicability of the entire device.

[0051] Third Embodiment: Please refer to Figure 9 and Figure 10 , Figure 9 which is the overall structural schematic diagram of the novel end effector device based on the visual positioning of the tool-changing manipulator of the third embodiment, Figure 10 which is the installation structural schematic diagram of the jaw 93. The clamping mechanism 9 provided by the present invention includes a secondary telescopic arm 91, a jaw connecting block 92 and a jaw 93.

[0052] Among them, the secondary telescopic arm 91 is connected to the gearbox 8 and is located on one side of the gearbox 8; the jaw connecting block 92 is connected to the secondary telescopic arm 91 and is located on one side of the secondary telescopic arm 91; the jaw 93 is connected to the jaw connecting block 92 and is located on one side of the jaw connecting block 92. The jaw 93 is composed of two symmetrical structures. The jaw connecting block 92 is hinged to the jaw 93, and the jaw 93 is pushed to clamp and release by a micro oil cylinder connecting the two. The jaw 93 is connected to one end of the secondary telescopic arm 91 through the jaw connecting block 92. The secondary telescopic arm 91 is driven by a hydraulic cylinder, and both ends of the hydraulic cylinder are connected to the telescopic arm through flanges, so that the telescopic arm extends or retracts when grasping the hob.

[0053] When using the novel end effector device based on the vision positioning of the tool changing manipulator in this embodiment, the secondary telescopic arm 91 provided can drive the jaw connecting block 92 and the jaw 93 to extend and retract, and then the jaw 93 provided cooperates with the jaw connecting block 92 and its corresponding driving mechanism to complete the grasping of the hob, greatly enhancing the practicability of the entire device.

[0054] What is disclosed above is only one or more preferred embodiments of the present application, and the scope of rights of the present application cannot be limited thereby. Those of ordinary skill in the art can understand all or part of the processes of implementing the above embodiments, and the equivalent changes made according to the claims of the present application still fall within the scope covered by the present application.

Claims

1. A novel end effector device based on the visual positioning of a tool-changing manipulator, comprising a tool magazine, a tool holder, a tool-changing manipulator, and a base. The tool magazine is installed on the tool holder, and the tool-changing manipulator is installed on the base. It is characterized in that, it further comprises a device body; The device body includes a forward visual positioning system, a backward visual positioning system, a bolt sleeve mechanism, a gearbox, and a clamping mechanism. The gearbox is connected to the tool-changing manipulator and is located on one side of the tool-changing manipulator. The backward visual positioning system is used to complete the working positioning during the process of removing bolts. The bolt sleeve mechanism is used to remove the bolts to be removed, facilitating the subsequent removal of the hob. The forward visual positioning system is used to complete the working positioning during the process of grasping the hob. The clamping mechanism is used to clamp and remove the hob.

2. The novel end effector device based on the visual positioning of a tool-changing manipulator according to claim 1, characterized in that, The forward visual positioning system includes a front protective cover housing, a front bottom plate, a front system sealing mechanism, and a front image acquisition and processing mechanism. The front protective cover housing is connected to the gearbox and is located on one side of the gearbox; the front bottom plate is connected to the front protective cover housing and is located at the bottom of the front protective cover housing; the front system sealing mechanism is connected to the front protective cover housing and is used to achieve the sealing protection of the corresponding installation space; the front image acquisition and processing mechanism is connected to the front bottom plate and is used to complete the acquisition and processing of the corresponding positioning image information.

3. The novel end effector device based on the visual positioning of a tool-changing manipulator according to claim 2, characterized in that, The front system sealing mechanism includes a front sealing joint, a front cover plate, and a front glass window. The front sealing joint is connected to the front protective cover housing and is located on one side of the front protective cover housing; the front cover plate is connected to the front protective cover housing and is located on one side of the front protective cover housing; the front glass window is connected to the front protective cover housing and is located on one side of the front protective cover housing.

4. The novel end effector device based on the visual positioning of a tool-changing manipulator according to claim 2, characterized in that, The front image acquisition and processing mechanism includes a front 3D camera, a front 2D camera, a front temperature control module, and a front image processing module. The front 3D camera is connected to the front bottom plate and is located inside the front protective cover housing; the front 2D camera is connected to the front bottom plate and is located inside the front protective cover housing; the front temperature control module is connected to the front protective cover housing and is located inside the front protective cover housing. The front image processing module is electrically connected to both the front 3D camera and the front 2D camera and is electrically connected to the tool-changing manipulator. The front image processing module is used to obtain the point cloud of the TBM tool magazine, segment the components of the point cloud, extract the feature point cloud, fit the point cloud plane based on the feature point cloud, and determine the position coordinates of the tool magazine positioning reference point.

5. The novel end effector device based on the visual positioning of a tool-changing manipulator according to claim 1, characterized in that, The rear vision positioning system includes a rear protective cover housing, a rear bottom plate, a rear system sealing mechanism, and a rear image acquisition and processing mechanism. The rear protective cover housing is connected to the gearbox and is located on one side of the gearbox; the rear bottom plate is connected to the rear protective cover housing and is located at the bottom of the rear protective cover housing; the rear system sealing mechanism is connected to the rear protective cover housing and is used to achieve the sealing protection of the corresponding installation space; the rear image acquisition and processing mechanism is connected to the rear bottom plate and is used to complete the acquisition and processing of the corresponding positioning image information.

6. The novel end effector device based on the vision positioning of the tool changing manipulator according to claim 5, wherein The rear system sealing mechanism includes a rear sealing joint, a rear cover plate, and a rear glass window. The rear sealing joint is connected to the rear protective cover housing and is located on one side of the rear protective cover housing; the rear cover plate is connected to the rear protective cover housing and is located on one side of the rear protective cover housing; the rear glass window is connected to the rear protective cover housing and is located on one side of the rear protective cover housing.

7. The novel end effector device based on the vision positioning of the tool changing manipulator according to claim 5, wherein The rear image acquisition and processing mechanism includes a rear 3D camera, a rear 2D camera, a rear temperature control module, and a rear image processing module. The rear 3D camera is connected to the rear bottom plate and is located inside the rear protective cover housing; the rear 2D camera is connected to the rear bottom plate and is located inside the rear protective cover housing; the rear temperature control module is connected to the rear protective cover housing and is located inside the rear protective cover housing. The rear image processing module is electrically connected to both the rear 3D camera and the rear 2D camera and is electrically connected to the tool changing manipulator.

8. The novel end effector device based on the vision positioning of the tool changing manipulator according to claim 1, wherein The bolt sleeve mechanism includes a sleeve wrench, a driven gear, a bearing, and a driving component. The sleeve wrench is connected to the gearbox through the bearing and is located on one side of the gearbox; the driven gear is connected to the sleeve wrench and is sleeved on the sleeve wrench; the bearing is connected to the sleeve wrench and is installed inside the gearbox; the driving component is connected to the gearbox.

9. The novel end effector device based on the vision positioning of the tool changing manipulator according to claim 8, wherein The sleeve wrench includes a sleeve front cover and a sleeve core shaft. The sleeve wrench is composed of the sleeve front cover and the sleeve core shaft, and the two ends of the sleeve core shaft are in the structure of an internal hexagonal wrench.

10. The novel end effector device based on the vision positioning of the tool changing manipulator according to claim 1, wherein The clamping mechanism includes a secondary telescopic arm, a jaw connecting block, and a jaw. The secondary telescopic arm is connected to the gearbox and is located on one side of the gearbox; the jaw connecting block is connected to the secondary telescopic arm and is located on one side of the secondary telescopic arm; the jaw is connected to the jaw connecting block and is located on one side of the jaw connecting block.

Images