A new end effector device based on visual positioning of tool changing manipulator

Abstract

The present invention relates to the technical field of full-face tunnel boring machines, and specifically to a novel end effector device based on visual positioning of a tool changing manipulator, comprising a tool box, a tool holder, a tool changing manipulator and a base, and also comprising a device body; the device body comprises a forward visual positioning system, a backward visual positioning system, a bolt sleeve mechanism, a gear box and a clamping mechanism, the gear box is connected to the tool changing manipulator, the forward visual positioning system is installed on one side of the gear box, the backward visual positioning system is installed on one side of the gear box, the bolt sleeve mechanism is connected to the gear box, and the clamping mechanism is connected to the gear box, thereby realizing a machine vision recognition and positioning system algorithm based on target point cloud processing, and combining the designed end effector with a bidirectional bolt sleeve to complete bolt positioning to improve the working efficiency and safety of the manipulator performing automatic operation in shield tool changing operations, promote the automation of shield construction, and realize shield construction with fewer people.

Description

Technical Field

[0001] The present invention relates to the technical field of full-face tunnel boring machines, and in particular to a novel end effector device based on visual positioning of a tool changing manipulator. Background Art

[0002] Full-face rock tunnel boring machines (TBMs) are widely used in various tunnel construction applications due to their high tunneling efficiency, excellent safety performance, and low construction costs. During tunnel construction, TBMs primarily rely on the cutterheads to cut soil and crush rock. Consequently, the cutters are exposed to high stress, alternating loads, and corrosion for extended periods. These harsh conditions can easily lead to cutter failure, with common failure modes including bearing damage, cutter ring fracture, spalling, and deformation.

[0003] During the tunneling process, TBMs rely primarily on the cutters on the cutterhead to break rock. Under the intense interaction with the rock and soil, the cutters are extremely susceptible to wear and failure. As a key component of the TBM, once a cutter fails, the TBM must be shut down for cutter replacement, which will have a serious adverse impact on the efficiency, construction cycle, and cost of tunnel construction. Currently, most TBM cutter replacements are performed manually in an extremely harsh environment. Cutter changers face dangers such as mud and water, rock bursts, high temperatures, high pressure, and harmful gases when performing cutter replacement underground, making safety accidents very likely to occur. Safety accidents caused by cutter replacement operations have become the main source of safety accidents during TBM construction. Failed cutters need to be replaced in a timely manner to ensure normal TBM excavation. However, currently, the replacement of failed TBM cutters is almost entirely manual, which brings a series of negative issues, including worker safety, low efficiency, and high costs. Therefore, the use of robots to safely and efficiently replace failed cutters is a current research hotspot.

[0004] However, when the existing TBM tool changing robot is performing normal operations through the provided robotic arm, the control mechanism of the entire robotic arm has very large errors when performing end positioning. Manual assistance is also required in the identification and positioning of the hob, making it difficult to carry out large-scale engineering applications. Therefore, it is crucial to study the end positioning of the TBM tool changing robot. Summary of the Invention

[0005] The purpose of the present invention is to provide a new end effector device based on the visual positioning of a tool changing robot, which can complete the bolt positioning based on the machine vision recognition and positioning system algorithm of the target point cloud processing, combined with the designed end effector with a two-way bolt sleeve. At the same time, a tool changing robot roller cutter replacement method based on end visual positioning is proposed to improve the efficiency and safety of shield tool changing operations, promote the automation of shield construction, and realize the reduction of manpower in shield construction.

[0006] To achieve the above-mentioned object, the present invention provides a novel end effector device based on visual positioning of a tool changing robot, comprising a tool box, a tool holder, a tool changing robot and a base, wherein the tool box is mounted on the tool holder, the tool changing robot is mounted on the base, and the device body is also provided;

[0007] The device body includes a forward visual positioning system, a rearward visual positioning system, a bolt sleeve mechanism, a gear box and a clamping mechanism. The gear box is connected to the tool changing robot and is located on one side of the tool changing robot. The work positioning during the bolt removal process is completed by the rearward visual positioning system, the removal of the bolt to be removed is completed by the bolt sleeve mechanism, and the subsequent removal of the hob is facilitated by the forward visual positioning system. The work positioning during the gripping of the hob is completed by the clamping mechanism.

[0008] Among them, the forward visual positioning system includes a front protective cover shell, a front base plate, a front system sealing mechanism and a front image acquisition and processing mechanism. The front protective cover shell is connected to the gear box and is located on one side of the gear box; the front base plate is connected to the front protective cover shell and is located at the bottom of the front protective cover shell; the front system sealing mechanism is connected to the front protective cover shell to achieve sealing protection of the corresponding installation space; the front image acquisition and processing mechanism is connected to the front base plate to complete the acquisition and processing of corresponding positioning image information.

[0009] Among them, the front system sealing mechanism includes a front sealing joint, a front cover plate and a front glass window, the front sealing joint is connected to the front protective cover shell and is located on one side of the front protective cover shell; the front cover plate is connected to the front protective cover shell and is located on one side of the front protective cover shell; the front glass window is connected to the front protective cover shell and is located on one side of the front protective cover shell.

[0010] Among them, the front image acquisition and processing mechanism includes a front 3D camera, a front 2D camera, a front temperature control module and a front image processing module. The front 3D camera is connected to the front bottom plate and is located in the front protective cover shell; the front 2D camera is connected to the front bottom plate and is located in the front protective cover shell; the front temperature control module is connected to the front protective cover shell and is located on the inner side of the front protective cover shell. The front image processing module is electrically connected to the front 3D camera and the front 2D camera at the same time, and is electrically connected to the tool changing robot. The processing flow of the front image processing module mainly includes: obtaining the TBM tool box point cloud, performing component segmentation on the point cloud, extracting the point cloud of the feature surface, fitting the point cloud plane, and determining the position coordinates of the tool box positioning reference point.

[0011] Among them, the rear visual positioning system includes a rear protective cover shell, a rear bottom plate, a rear system sealing mechanism and a rear image acquisition and processing mechanism. The rear protective cover shell is connected to the gear box and is located on one side of the gear box; the rear bottom plate is connected to the rear protective cover shell and is located at the bottom of the rear protective cover shell; the rear system sealing mechanism is connected to the rear protective cover shell to achieve sealing protection of the corresponding installation space; the rear image acquisition and processing mechanism is connected to the rear bottom plate to complete the acquisition and processing of corresponding positioning image information.

[0012] Among them, the rear system sealing mechanism includes a rear sealing joint, a rear cover plate and a rear glass window, the rear sealing joint is connected to the rear protective cover shell and is located on one side of the rear protective cover shell; the rear cover plate is connected to the rear protective cover shell and is located on one side of the rear protective cover shell; the rear glass window is connected to the rear protective cover shell and is located on one side of the rear protective cover shell.

[0013] Among them, the rear image acquisition and processing mechanism includes a rear 3D camera, a rear 2D camera, a rear temperature control module and a rear image processing module. The rear 3D camera is connected to the rear bottom plate and is located in the rear protective cover shell; the rear 2D camera is connected to the rear bottom plate and is located in the rear protective cover shell; the rear temperature control module is connected to the rear protective cover shell and is located on the inner side of the rear protective cover shell; the rear image processing module is electrically connected to the rear 3D camera and the rear 2D camera at the same time, and is electrically connected to the tool changing robot.

[0014] In which, the bolt sleeve mechanism includes a socket wrench, a driven gear, a bearing and a driving component. The socket wrench is connected to the gear box through the bearing and is located on one side of the gear box; the driven gear is connected to the socket wrench and is sleeved on the socket wrench; the bearing is connected to the socket wrench and is installed in the gear box; the driving component is connected to the gear box.

[0015] Wherein, the driving component includes a driving gear and a hydraulic motor. The driving gear is engaged with the driven gear and is rotatably installed in the gear box. The output shaft of the hydraulic motor is connected to the driving gear, and the hydraulic motor is installed in the gear box.

[0016] Wherein, the socket wrench includes a socket front cover and a socket core shaft, and the socket wrench is composed of the socket front cover and the socket core shaft, and the structures at both ends of the socket core shaft are hexagonal wrenches.

[0017] In which, the clamping mechanism includes a secondary telescopic arm, a clamping claw connecting block and a clamping claw, the secondary telescopic arm is connected to the gear box and is located on one side of the gear box; the clamping claw connecting block is connected to the secondary telescopic arm and is located on one side of the secondary telescopic arm; the clamping claw is connected to the clamping claw connecting block and is located on one side of the clamping claw connecting block.

[0018] The present invention provides a new end effector device based on the visual positioning of the tool changing manipulator, which can be positioned according to the integrated tool system of different shield machines and the end of the tool changing manipulator. When removing the bolts, the tool changing manipulator first drives the device body to the area with the replacement hob, so that the rear visual positioning system is facing the center of the tool box of the hob, and the end positioning is completed by the proposed TBM tool box positioning algorithm based on point cloud processing, and then the device body is controlled to move up and down to align with the bolt, and then the bolt is removed by the provided bolt sleeve mechanism. When grabbing the hob, after completing the removal of the bolts of the integrated tool system, the tool changing manipulator controls the The device body is turned so that its front-facing visual positioning system is facing the center of the tool box of the hob. After obtaining the relative position of the end and the tool box at that moment using the TBM tool box positioning algorithm based on point cloud processing, the height of the device body is adjusted so that the clamping mechanism is aligned with the hob position, thereby completing the clamping of the hob. This realizes the machine vision recognition and positioning system algorithm based on target point cloud processing, combined with the designed end effector with a two-way bolt sleeve to complete bolt positioning. At the same time, a hob replacement method of a tool changing robot based on end-end visual positioning is proposed to improve the efficiency and safety of shield tool changing operations, promote the automation of shield construction, and realize shield construction with fewer people. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art.

[0020] Figure 1 It is a schematic structural diagram of the entirety of a novel end effector device based on visual positioning of a tool changing robot according to the first embodiment of the present invention.

[0021] Figure 2 It is a structural diagram of the device body of the first embodiment of the present invention.

[0022] Figure 3 2 is a schematic structural diagram of a forward vision positioning system according to a first embodiment of the present invention.

[0023] Figure 4 2 is a schematic diagram of the internal structure of the forward vision positioning system of the first embodiment of the present invention.

[0024] Figure 52 is a schematic structural diagram of a rearward vision positioning system according to a first embodiment of the present invention.

[0025] Figure 6 2 is a schematic diagram of the internal structure of the rear-facing visual positioning system according to the first embodiment of the present invention.

[0026] Figure 7 It is a schematic structural diagram of the entire new end effector device based on visual positioning of a tool changing robot according to the second embodiment of the present invention.

[0027] Figure 8 2 is a schematic structural diagram of a driving component according to a second embodiment of the present invention.

[0028] Figure 9 It is a schematic structural diagram of the entirety of a novel end effector device based on visual positioning of a tool changing robot according to the third embodiment of the present invention.

[0029] Figure 10 2 is a schematic diagram of the installation structure of the clamping jaws according to the third embodiment of the present invention.

[0030] In the figure: 1- tool box, 2- tool holder, 3- tool changing manipulator, 4- base, 5- forward visual positioning system, 6- rear visual positioning system, 7- bolt sleeve mechanism, 8- gear box, 9- clamping mechanism, 51- front sealing joint, 52- front protective cover shell, 53- front bottom plate, 54- front glass window, 55- front cover, 56- front 3D camera, 57- front 2D camera, 58- front temperature control module, 61- rear sealing Sealing head, 62-rear protective cover shell, 63-rear bottom plate, 64-rear glass window, 65-rear cover plate, 66-rear 3D camera, 67-rear 2D camera, 68-rear temperature control module, 71-socket wrench, 72-driven gear, 73-bearing, 74-driving gear, 75-hydraulic motor, 711-sleeve front cover, 712-sleeve core shaft, 91-secondary telescopic arm, 92-grip connecting block, 93-grip. DETAILED DESCRIPTION

[0031] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, but should not be understood as limiting the present invention.

[0032] The first embodiment of this application is:

[0033] See also Figures 1 to 6 ,in Figure 1 This is a schematic diagram of the overall structure of the new end effector device based on the visual positioning of the tool changing robot. Figure 2 This is a schematic diagram of the structure of the device. Figure 3 is a schematic diagram of the structure of the forward visual positioning system 5, Figure 4is a schematic diagram of the internal structure of the forward visual positioning system 5, Figure 5 is a schematic diagram of the structure of the backward visual positioning system 6, Figure 6 Schematic diagram of the internal structure of the rear vision positioning system 6.

[0034] The present invention provides a novel end effector device based on the visual positioning of a tool changing manipulator: comprising a tool box 1, a tool holder 2, a tool changing manipulator 3, a base 4 and a device body, wherein the device body comprises a forward visual positioning system 5, a backward visual positioning system 6, a bolt sleeve mechanism 7, a gear box 8 and a clamping mechanism 9, wherein the forward visual positioning system 5 comprises a front sealing joint 51, a front protective cover shell 52, a front bottom plate 53, a front glass window 54, a front cover plate 55, a front 3D camera 56, a front 2D camera 57 and a front temperature control module 58, and the backward visual positioning system 6 comprises a rear sealing joint 61, a rear protective cover shell 62, a rear bottom plate 63, a rear glass window 64, a rear cover plate 65, a rear 3D camera 66, a rear 2D camera 67 and a rear temperature control module 68. The above-mentioned solution solves most of the current TBM tool changing problems. The cutters are all replaced manually, and the environment for cutting cutters is extremely harsh. Cutters changing workers face dangers such as mud and water, rock bursts, high temperatures, high pressures, and harmful gases when changing cutters underground, making safety accidents very likely to occur. Safety accidents caused by cutter changing operations have become the main source of safety accidents in TBM construction. Failed cutters need to be replaced in time to ensure the normal excavation of TBM. However, the replacement of failed TBM cutters currently relies almost entirely on manual labor, which will bring a series of negative problems, including construction worker safety, low efficiency, and high costs. Therefore, the use of robots for safe and efficient replacement of failed cutters is a current research hotspot. However, the existing TBM cutter changing manipulator 3 end positioning error is very large, and manual assistance is still required in the identification and positioning of the cutters, making it difficult to carry out large-scale engineering applications.

[0035] In this embodiment, the tool box 1 is installed on the tool holder 2, the tool changing robot 3 is installed on the base 4, and the device body is applied to a platform constructed by the tool box 1, the tool holder 2, the tool changing robot 3 and the base 4.

[0036] Among them, the gear box 8 is connected to the tool changing robot 3 and is located on one side of the tool changing robot 3, the forward visual positioning system 5 is installed on one side of the gear box 8, the backward visual positioning system 6 is installed on one side of the gear box 8, the bolt sleeve mechanism 7 is connected to the gear box 8, the clamping mechanism 9 is connected to the gear box 8, and the gear box 8 is fixed at the adjustment output end of the tool changing robot 3 so that the gear box 8 and the mechanism provided on the gear box 8 can be driven by the tool changing robot 3 to move. The forward visual positioning system 5 and the backward visual positioning system 6 constitute a two-way visual positioning system. When grabbing the hob, the forward visual positioning system 5 is used for positioning, and when removing the bolt, the backward visual positioning system 6 is used for positioning. The specific positioning coordinates are indirectly positioned by identifying the position of the characteristic surface 12 of the tool box 1.

[0037] Secondly, the front protective cover shell 52 is connected to the gear box 8 and is located on one side of the gear box 8; the front sealing joint 51 is connected to the front protective cover shell 52 and is located on one side of the front protective cover shell 52; the front cover plate 55 is connected to the front protective cover shell 52 and is located on one side of the front protective cover shell 52; the front bottom plate 53 is connected to the front protective cover shell 52 and is located at the bottom of the front protective cover shell 52; the front glass window 54 is connected to the front protective cover shell 52 and is located on one side of the front protective cover shell 52, the front 3D camera 56 is connected to the front bottom plate 53 and is located inside the front protective cover shell 52; the front 2D camera 57 is connected to the front bottom plate 53 and is located inside the front protective cover shell 52; the front temperature control module 58 is connected to the front protective cover shell 52 and is located inside the front protective cover shell 52, the front The visual positioning system 5 includes the front 3D camera 56, the two front 2D cameras 57, the front temperature control module 58, the front sealing joint 51, the front protective cover shell 52, the front bottom plate 53, the front glass window 54 and the front cover plate 55. The forward visual positioning system 5 is fixed to the gear box 8 through two threaded holes. The front temperature control module 58 is fixed to the inner wall of the front protective cover shell 52 to adjust the temperature of the heating strip to achieve the dehumidification effect while preventing the equipment from being damaged by excessive temperature. By setting the front sealing joint 51, the internal device can work normally in a high-humidity and high-pressure tunnel environment. The front bottom plate 53 is fixed to the bottom of the front protective cover, and the top of the front bottom plate 53 is fixed to the front 3D camera 56 to enhance the heat dissipation effect of the front 3D camera 56. The two front 2D cameras 57 are respectively fixed to the front bottom plate 53 through connecting plates to obtain forward and backward video streams.

[0038] At the same time, the rear protective cover shell 62 is connected to the gear box 8 and is located on one side of the gear box 8; the rear sealing joint 61 is connected to the rear protective cover shell 62 and is located on one side of the rear protective cover shell 62; the rear cover plate 65 is connected to the rear protective cover shell 62 and is located on one side of the rear protective cover shell 62; the rear bottom plate 63 is connected to the rear protective cover shell 62 and is located at the bottom of the rear protective cover shell 62; the rear glass window 64 is connected to the rear protective cover shell 62 and is located on one side of the rear protective cover shell 62, the rear 3D camera 66 is connected to the rear bottom plate 63 and is located in the rear protective cover shell 62; the rear 2D camera 67 is connected to the rear bottom plate 63 and is located in the rear protective cover shell 62; the rear temperature control module 68 is connected to the rear protective cover The protective cover shell 62 is connected and is located on the inner side of the rear protective cover shell 62. The rear visual positioning system 6 is similar in structure to the forward visual positioning system 5. The rear visual positioning system 6 includes the rear 3D camera 66, the rear 2D camera 67, the rear temperature control module 68, the rear sealing joint 61, the rear protective cover shell 62, the rear bottom plate 63, the rear glass window 64 and the rear cover plate 65; the rear visual positioning system 6 is fixed to the bottom of the gear box 8 through four threaded holes on the top, and the rear temperature control module 68 is fixed to the inner wall of the rear protective cover shell 62; the rear bottom plate 63 is fixed to the bottom of the rear protective cover, and the top of the rear bottom plate 63 is fixed to the rear 3D camera 66; the rear 2D camera 67 is fixed to the rear bottom plate 63 through a connecting plate for obtaining a video stream.

[0039] The key to the tool changing robot 3 performing the hob replacement operation is to accurately locate the loosening bolt that fixes the hob, and the corresponding 3D camera can only obtain the point cloud information within the camera's field of view by taking pictures. To finally obtain the three-dimensional coordinate information of the bolt position, it is necessary to process the point cloud data and image data through a series of point cloud and image processing algorithms, and finally obtain the coordinate information of the bolt and transmit it to the robot controller. Since the end area of the bolt is too small, direct positioning can use less point cloud information and the positioning is inaccurate. Considering all factors, an indirect positioning method is adopted to segment the rectangular area point cloud information of the larger feature surface 12 from the point cloud information, and then fit the center point coordinates of the area point cloud. Combined with the fixed geometric relationship between the bolt position and the feature surface center point coordinates, the three-dimensional coordinate information of the bolt is indirectly obtained through calculation and transformation. This series of processing procedures includes using advanced image processing and deep learning methods to perform two-dimensional image feature surface area instance segmentation, two-dimensional image and point cloud alignment, and use denoising algorithm 3D reconstruction of the target object using 3D 3D cameras.

[0040] 1. Disparity Map Generation and Point Cloud Acquisition: For disparity map generation, we use the RAFT-Stereo model, which is lightweight, does not require complex loss functions, and is suitable for high-resolution images. This model consists of three main components: a feature extractor, a correlation pyramid, and a multi-level disparity update module. Based on the obtained disparity map, we use triangulation principles and a reprojection matrix to generate an initial point cloud, which is then denoised and registered. Examples of point clouds obtained using this method include a TBM tool box 1 and a scaled hob cutter model.

[0041] 2. Target point cloud segmentation: The original input point cloud is marked as N, which is composed of the coordinates of the point and the features of the corresponding points The Farthest Point Sampling (FPS) algorithm selects a certain number of sample centroids from a point cloud, and the sampled points are evenly discretized to cover the entire point cloud as much as possible. Therefore, FPS sampling is performed on the initial input point cloud, resulting in point cloud samples with N1, N2, and N3 points, denoted as S1, S2, and S3, respectively. The resolutions of these three samples are 2048, 1024, and 512, respectively. A spherical query method is then used to determine the neighboring points around each sample centroid within a spherical space of a certain radius.

[0042] After obtaining point cloud samples of varying resolutions, we need to construct a point embedding module to aggregate local neighborhood features of the input point cloud. Both maximum pooling and uniform pooling demonstrate good performance in aggregating local features without incurring significant computational cost. After comprehensive comparison, we chose maximum pooling for point embedding, enhancing the overall model architecture's local feature extraction capabilities.

[0043] After point embedding, the point cloud transformer is applied, combining linear and pooling layers to encode feature information on the point clouds along the three paths. The Transformer uses an offset-attention mechanism to improve the Transformer's point cloud segmentation performance, achieving better segmentation results.

[0044] In the offset attention, first, let Q, K, V represent the query, key, and value matrices respectively, and the input point cloud features Perform linear transformation to obtain Q, K, and V. The specific formulas are as follows:

[0045]

[0046] in, , , is a weight matrix that can be updated through learning, is the dimension of the Q and K vector matrices. It should be noted that and input features The dimensions D are generally not equal. After calculating the Q, K, and V matrices, the attention feature matrix is calculated by matrix dot multiplication. , the formula is as follows:

[0047]

[0048] in, Representation matrix The transposed matrix of Express and The matrix obtained by point multiplication is softmax and Processing, and then with the matrix Perform dot multiplication and finally get the attention feature matrix.

[0049] Laplacian matrix It has shown excellent processing performance in the graph convolutional network structure, is the diagonal matrix of the processed image, is the adjacency matrix of the image, is the number of vertices in the image. The Laplacian matrix algorithm mechanism is applied to the self-attention mechanism to form the offset attention mechanism. The offset attention mechanism uses matrix subtraction to calculate the input features With self-attention feature The offset between the two, and then the offset matrix Operation, and finally sum it with the input features to get the output features , the specific formula is as follows:

[0050]

[0051] in, It is to perform the offset attention mechanism operation on the input features. Equivalent to the Laplace shift operator.

[0052] In addition, in Transformer, it is necessary to encode the positions of the points in the point cloud sample, including the spatial position information of the point cloud, and the position encoding function used is as follows:

[0053]

[0054] in, , Respectively represent Hedi The spatial coordinates of the points, express

[0055] operation, thereby embedding the contextual information including the position into the Transformer.

[0056] After running the point cloud transformer on three point cloud samples of different resolutions, we obtain feature information containing global context features at three different scales. To aggregate features at different scales, we need to perform cross-scale fusion on the features of the three paths to obtain the final feature output. Combined with the self-attention mechanism model, the cross-scale fusion algorithm is as follows:

[0057]

[0058] in, represents the final weighted point cloud feature, , Respectively and operate, , , Represents the point cloud Transformer output feature matrix obtained by each path, and Represents the transposed matrix of the point cloud Transformer output features. Get the final feature matrix After that, the global features of the point cloud are obtained through maximum pooling.

[0059] In order to perform the point cloud segmentation task, the number dimension of the global feature is repeated to be consistent with the cross-scale feature, and then spliced with the cross-scale feature matrix to obtain the splicing matrix , then Do the following:

[0060]

[0061] in, represents the final point cloud segmentation score, is the feature score dimension set, Indicates Append one after the operation Operation. This completes the point cloud segmentation operation of the input point cloud.

[0062] 3. Feature fitting and spatial positioning

[0063] In order to perform the final positioning operation, it is necessary to set an inherent feature of the object to be identified in advance, and indirectly determine the position of the object by determining the spatial position of the feature.

[0064] After segmenting the input point cloud of the entire object, the point cloud with the set inherent features is extracted, and the least squares method is used to fit the plane, and then the plane features, such as corner coordinates and edge lines, are calculated. Then, the position of the feature surface is determined. Based on the relative position relationship between the feature surface and the object, the spatial position of the object is indirectly determined to complete the positioning work.

[0065] Combining the above steps, a machine vision recognition and positioning system algorithm based on target point cloud processing is used in conjunction with the TBM tool changer 3 to complete automatic tool changing. First, a binocular camera acquires a target image, and the RAFT-Stereo algorithm calculates a disparity map to generate a target point cloud. Second, the point cloud is encoded using a point cloud transformer, employing an offset attention mechanism. Point clouds of varying resolutions are then fused across scales to aggregate features at different scales, ultimately yielding a well-segmented point cloud. Feature surface point clouds are extracted and fitted to determine the coordinates of the positioning reference points, completing the positioning process. Finally, the positioning error is calculated. Experimental results demonstrate that the positioning algorithm used in this paper achieves excellent positioning results, with positioning errors of less than 0.4 mm in both the X and Y directions and less than 1.7 mm in the depth Z direction, meeting engineering application requirements.

[0066] The new end effector device based on the visual positioning of the tool changing manipulator of this embodiment can be used according to the integrated tool system of different shield machines and the end positioning of the tool changing manipulator 3. When removing the bolts, the tool changing manipulator 3 is first used to drive the device body to the area with the replacement hob, so that the rear visual positioning system 6 is facing the center of the tool box 1 of the hob, and the end positioning is completed by the proposed TBM tool box 1 positioning algorithm based on point cloud processing, and then the device body is controlled to move up and down to align with the bolt, and then the bolt sleeve mechanism 7 is used to complete the removal of the bolt. When grabbing the hob, after completing the removal of the bolt of the integrated tool system, the tool changing machine The manipulator 3 controls the steering of the device body so that the front-facing visual positioning system is facing the center of the tool box 1 of the hob. After obtaining the relative position of the end and the tool box 1 at this moment by using the TBM tool box 1 positioning algorithm based on point cloud processing, the height of the device body is adjusted so that the clamping mechanism 9 is aligned with the hob position, thereby completing the clamping of the hob. This realizes the machine vision recognition and positioning system algorithm based on target point cloud processing, combined with the designed end effector with a two-way bolt sleeve to complete the bolt positioning. At the same time, a hob replacement method of the tool changing manipulator 3 based on end visual positioning is proposed to improve the efficiency and safety of shield tool changing operations, promote the automation of shield construction, and realize the less-manned shield construction.

[0067] Second embodiment:

[0068] See also Figure 7 and Figure 8 , Figure 7 This is a schematic diagram of the overall structure of a new end effector device based on visual positioning of a tool changing robot according to the second embodiment. Figure 8This is a structural diagram of the driving component. The bolt sleeve mechanism 7 provided by the present invention includes a socket wrench 71, a driven gear 72, a bearing 73 and a driving component. The driving component includes a driving gear 74 and a hydraulic motor 75. The socket wrench 71 includes a socket front cover 711 and a socket core shaft 712.

[0069] Among them, the socket wrench 71 is connected to the gear box 8 through the bearing 73 and is located on one side of the gear box 8; the driven gear 72 is connected to the socket wrench 71 and is sleeved on the socket wrench 71; the bearing 73 is connected to the socket wrench 71 and is installed in the gear box 8; the driving component is connected to the gear box 8, the driving gear 74 is meshed with the driven gear 72, and is rotatably installed in the gear box 8; the output shaft of the hydraulic motor 75 is connected to the driving gear 74, and the hydraulic motor 75 is installed in the gear box 8. The socket wrench 71 is composed of the sleeve front cover 711 and the sleeve core shaft 712. The two ends of the sleeve core shaft 712 are structured as hexagonal wrenches. The end of the sleeve front cover 711 is designed with an inclination angle to accommodate radial and circumferential errors. There are two groups in total. Both groups of socket wrenches 71 are provided with driven gears 72. The two driven gears 72 are driven by the driving gear 74 provided. The driving gear 74 is driven by the hydraulic motor 75, thereby completing the synchronous drive of the two groups of socket wrenches 71. The two sides of the gear box 8 are sealed and connected to the socket wrenches 71 by fixed bearing 73 end covers. The hydraulic motor 75 drives the driving gear 74 and then drives the two driven gears 72 and the two socket wrenches 71 to rotate to remove the bolts. The socket wrenches 71 and the driven gears 72 are driven by a spline. The hydraulic motor 75 is connected to the driving gear 74 through a spline shaft. At the same time, the socket wrenches 71 are in contact with the gear box 8 through the bearings 73 to achieve free rotation of the socket wrenches 71.

[0070] When using a new end effector device based on visual positioning of a tool changing robot in this embodiment, the driving gear 74 can be driven by the hydraulic motor 75, and then the two driven gears 72 and the two socket wrenches 71 are driven to rotate by the driving gear 74, and then the corresponding bolts are removed by driving the two socket wrenches 71, which greatly enhances the practicality of the entire device.

[0071] Third embodiment:

[0072] See also Figure 9 and Figure 10 , Figure 9This is a schematic diagram of the overall structure of a new end effector device based on visual positioning of a tool changing robot according to the third embodiment. Figure 10 Schematic diagram of the installation structure of the clamping jaw 93 . The clamping mechanism 9 provided by the present invention includes a secondary telescopic arm 91 , a clamping jaw connecting block 92 and a clamping jaw 93 .

[0073] Among them, the secondary telescopic arm 91 is connected to the gear box 8 and is located on one side of the gear box 8; the clamp connecting block 92 is connected to the secondary telescopic arm 91 and is located on one side of the secondary telescopic arm 91; the clamp 93 is connected to the clamp connecting block 92 and is located on one side of the clamp connecting block 92. The clamp 93 is composed of two symmetrical structures. The clamp connecting block 92 is hinged to the clamp 93, and the clamp 93 is pushed to clamp and release by a micro-cylinder connecting the two. The clamp 93 is connected to one end of the secondary telescopic arm 91 through the clamp connecting block 92. The secondary telescopic arm 91 is driven by a hydraulic cylinder, and both ends of the hydraulic cylinder are connected to the telescopic arm through flanges, so that the telescopic arm can extend or retract when grabbing the hob.

[0074] When using a new end effector device based on visual positioning of a tool changing robot in this embodiment, the secondary telescopic arm 91 can be used to drive the clamping claw connecting block 92 and the clamping claw 93 to extend and retract, and then the clamping claw 93 is used to cooperate with the clamping claw connecting block 92 and its corresponding driving mechanism to complete the grasping of the hob, which greatly enhances the practicality of the entire device.

[0075] The above disclosure is merely one or more preferred embodiments of the present application and is not intended to limit the scope of the present application. A person skilled in the art will understand that all or part of the processes of the above embodiments and equivalent changes made in accordance with the claims of the present application are still within the scope of the present application.

Claims

1. A novel end effector device based on visual positioning of a tool changing robot, comprising a tool box, a tool holder, a tool changing robot and a base, wherein the tool box is mounted on the tool holder, and the tool changing robot is mounted on the base, characterized in that: Also included is a device body; The device body includes a forward visual positioning system, a rearward visual positioning system, a bolt sleeve mechanism, a gear box and a clamping mechanism. The gear box is connected to the tool changing manipulator and is located on one side of the tool changing manipulator. The rearward visual positioning system is used to complete the working positioning during the bolt removal process. The bolt sleeve mechanism is used to complete the removal of the disassembly bolts, thereby facilitating the subsequent removal of the hob. The forward visual positioning system is used to complete the working positioning during the gripping of the hob. The clamping mechanism is used to complete the clamping and disassembly of the hob. The forward visual positioning system includes a front protective cover shell, a front bottom plate, a front system sealing mechanism and a front image acquisition and processing mechanism; The front system sealing mechanism includes a front sealing joint, a front cover plate and a front glass window; The front image acquisition and processing mechanism includes a front 3D camera, a front 2D camera, a front temperature control module and a front image processing module. The front 3D camera is connected to the front bottom plate and is located in the front protective cover shell; the front 2D camera is connected to the front bottom plate and is located in the front protective cover shell; the front temperature control module is connected to the front protective cover shell and is located on the inner side of the front protective cover shell. The front image processing module is electrically connected to the front 3D camera and the front 2D camera at the same time, and is electrically connected to the tool changing robot. The front image processing module is used to obtain the TBM tool box point cloud, perform component segmentation on the point cloud, extract the feature surface point cloud, fit the point cloud plane based on the feature surface point cloud, and determine the position coordinates of the tool box positioning reference point; The rear vision positioning system includes a rear protective cover shell, a rear bottom plate, a rear system sealing mechanism and a rear image acquisition and processing mechanism; The rear system sealing mechanism includes a rear sealing joint, a rear cover plate and a rear glass window; The rear image acquisition and processing mechanism includes a rear 3D camera, a rear 2D camera, a rear temperature control module and a rear image processing module; The bolt sleeve mechanism includes a socket wrench, a driven gear, a bearing and a driving component. The socket wrench is connected to the gear box through the bearing and is located on one side of the gear box; the driven gear is connected to the socket wrench and sleeved on the socket wrench; the bearing is connected to the socket wrench and installed in the gear box; the driving component is connected to the gear box; The clamping mechanism includes a secondary telescopic arm, a clamping claw connecting block and a clamping claw. The secondary telescopic arm is connected to the gear box and is located on one side of the gear box; the clamping claw connecting block is connected to the secondary telescopic arm and is located on one side of the secondary telescopic arm; the clamping claw is connected to the clamping claw connecting block and is located on one side of the clamping claw connecting block.

2. The novel end effector device based on visual positioning of the tool changing manipulator according to claim 1 is characterized in that: The front protective cover shell is connected to the gear box and is located on one side of the gear box; the front base plate is connected to the front protective cover shell and is located at the bottom of the front protective cover shell; the front system sealing mechanism is connected to the front protective cover shell to achieve sealing protection of the corresponding installation space; the front image acquisition and processing mechanism is connected to the front base plate to complete the acquisition and processing of corresponding positioning image information.

3. The novel end effector device based on visual positioning of the tool changing manipulator according to claim 2 is characterized in that: The front sealing joint is connected to the front protective cover shell and is located on one side of the front protective cover shell; the front cover plate is connected to the front protective cover shell and is located on the top of the front protective cover shell; the front glass window is connected to the front protective cover shell and is located on one side of the front protective cover shell.

4. The novel end effector device based on visual positioning of a tool changing manipulator according to claim 1 is characterized in that: The rear protective cover shell is connected to the gear box and is located on one side of the gear box; the rear bottom plate is connected to the rear protective cover shell and is located at the bottom of the rear protective cover shell; the rear system sealing mechanism is connected to the rear protective cover shell to achieve sealing protection of the corresponding installation space; the rear image acquisition and processing mechanism is connected to the rear bottom plate to complete the acquisition and processing of corresponding positioning image information.

5. The novel end effector device based on visual positioning of the tool changing manipulator according to claim 4 is characterized in that: The rear sealing joint is connected to the rear protective cover shell and is located on one side of the rear protective cover shell; the rear cover plate is connected to the rear protective cover shell and is located on the top of the rear protective cover shell; the rear glass window is connected to the rear protective cover shell and is located on one side of the rear protective cover shell.

6. The novel end effector device based on visual positioning of the tool changing manipulator according to claim 4 is characterized in that: The rear 3D camera is connected to the rear bottom plate and is located inside the rear protective cover shell; the rear 2D camera is connected to the rear bottom plate and is located inside the rear protective cover shell; the rear temperature control module is connected to the rear protective cover shell and is located inside the rear protective cover shell; the rear image processing module is electrically connected to the rear 3D camera and the rear 2D camera at the same time, and is electrically connected to the tool changing robot.

7. The novel end effector device based on visual positioning of a tool changing manipulator according to claim 1 is characterized in that: The socket wrench includes a socket front cover and a socket core shaft. The socket wrench is composed of the socket front cover and the socket core shaft. The two end structures of the socket core shaft are hexagonal wrenches. The end of the socket front cover is designed with an inclination angle, which can accommodate radial and circumferential errors.

Images