A roller adaptive gripping manipulator claw system

Abstract

The present invention relates to a self-adaptive gripping mechanical gripper system for rollers, comprising a gripper assembly, a gripper motion support assembly, an angle adjustment drive mechanism, a visual assembly, and an image recognition device; the gripper assembly is composed of three grippers, which grip and support rollers from three positions; the first and second grippers are connected to the gripper motion support assembly for self-adaptive and active posture adjustment, respectively; the third gripper cooperates with the angle adjustment drive mechanism; the visual assembly is used to capture images of the mounting grooves on both sides of the roller; the image recognition device outputs the required rotation angle of the roller to the control system based on the images captured by the visual assembly, controls the angle adjustment drive mechanism to operate, and causes the third gripper to drive the roller to rotate to the correct installation angle. The present invention can achieve accurate gripping, recognition, and angle calibration of rollers, improve the safety and reliability of the roller installation process, reduce labor intensity and operational risks, and promote efficient production.

Description

Technical Field

[0001] The present invention relates to the technical field of manipulators, in particular to a roller self-adaptive grabbing manipulator claw system. Background Art

[0002] Belt conveyors are a critical piece of continuous conveying equipment in underground coal mining. Rollers are crucial components that support and ensure the proper operation of belt conveyors. These rollers are numerous, especially during long-distance transport. However, the harsh underground environment, coupled with factors such as the length of belt conveyor operation and the variability of the materials being transported, significantly shorten the lifespan of these rollers. To prevent roller failure, which can lead to a buildup of material on the conveyor, this can trigger a chain reaction that can cause mine shutdowns and even serious accidents such as casualties. When a belt conveyor roller fails, the damaged roller must be promptly replaced. Existing methods for replacing belt conveyor rollers include manual replacement, assisted replacement tools, replaceable roller supports, and roller replacement vehicles. Manual replacement requires the coordinated efforts of multiple workers, is labor-intensive, and carries significant safety risks. Other methods also cannot avoid manual intervention in the roller replacement process. To further enhance intelligent mining, intelligent identification and adjustment during the roller replacement process are needed to reduce manual intervention and mitigate safety risks.

[0003] Existing roller replacement devices, such as the "Non-stop roller replacement robot" disclosed in the invention patent application with authorization publication number CN114473949B, have roller grabbing claws designed in this technical solution, which are composed of upper and lower claws that are hinged to each other and controlled by a hydraulic cylinder. Only the front part of the claw is arc-shaped, and the grabbing action is achieved by two hinged parts. The claw has a fixed position, and it is impossible to prevent the roll angle of the roller from changing during the transportation and installation of the roller. In other words, it is impossible to ensure that the installation groove angle of the roller to be installed is compatible with the roller bracket during the roller installation phase. Manual intervention may be required, which has a high degree of uncertainty. For example, the "A belt conveyor non-stop roller replacement vehicle" disclosed in the invention patent application with application publication number CN114986430A, the roller replacement device designed in this technical solution consists of a symmetrically distributed parallel structure. The claw performs the roller grabbing action through four mechanical claws, and the hydraulic cylinder on the grabbing side controls the grabbing action of the manipulator. The slender shape of the robotic gripper limited its contact area with the rollers, making it impossible to prevent the rollers from shifting during transport. This could result in incorrect roller installation and installation failure. Furthermore, the hydraulic cylinder and the roller being grasped are located on the same side, making it impossible to grasp larger rollers, and there is the potential for interference between the hydraulic cylinder and the roller being grasped. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention provides a roller adaptive grasping robot claw system, which can achieve accurate grasping, identification and angle calibration of rollers, improve the safety and reliability of the roller installation process, reduce labor intensity and operation risks, and promote efficient production.

[0005] The technical solution adopted in the present invention is as follows:

[0006] A roller adaptive gripping mechanical gripper system, comprising a gripper assembly, a gripper motion support assembly, an angle adjustment drive mechanism, a visual assembly, and an image recognition device;

[0007] The gripper assembly includes a first gripper, a second gripper, and a third gripper distributed along the circumferential direction;

[0008] The structure of the third gripper includes a guide seat, in which a first shoe-shaped body is provided for sliding engagement therewith, wherein the inner surface of the first shoe-shaped body is an inner concave arc surface for contacting the circumferential surface of the roller; the first shoe-shaped body is driven by the angle adjustment drive mechanism and can rotate circumferentially along the guide seat;

[0009] The first and second claws are located above the third claw and have the same structure, each comprising an outer claw and a second shoe-shaped body. The second shoe-shaped body is located inside the outer claw and slides with the outer claw. A return spring providing preload and return force is provided between the outer claw and the second shoe-shaped body. The inner surface of the second shoe-shaped body is an inner concave arc surface for contacting the circumferential surface of the roller.

[0010] The gripper motion support assembly includes a movable top plate, the outer claw of the first gripper is directly hinged to the movable top plate; the outer claw of the second gripper is hinged to the top plate via a connecting frame, and the posture is adjusted by a third driving mechanism;

[0011] The visual component is used to collect images of the mounting grooves on both sides of the roller, and includes a collection device, an illuminator and a support frame. The support frame is installed on the top of the top plate, and the collection device and the illuminator are installed on the support frame;

[0012] The image recognition device outputs the required rotation angle of the roller to the control system based on the image collected by the visual component. The control system controls the angle adjustment drive mechanism to operate so that the third gripper drives the roller to rotate to the correct installation angle.

[0013] Further technical solutions are:

[0014] The angle adjustment drive mechanism includes a second rotation drive mechanism, a guide rail, and a spur rack slidingly engaged with the guide rail. The second rotation drive mechanism is provided with at least one output end, and the at least one output end is connected to a drive gear. The drive gear is engaged with the spur rack for transmission, driving the spur rack to move linearly along the guide rail; the outer side surface of the first shoe-shaped body is provided with external teeth in the circumferential direction that are engaged with the spur rack for transmission.

[0015] The gripper motion support assembly also includes a first rotation drive mechanism and a support arm. The first rotation drive mechanism is provided with at least one output end, the at least one output end is connected to one end of the support arm, and the other end of the support arm is connected to the top plate.

[0016] The third driving mechanism is a linear driving mechanism, a fixed end of which is hinged to the top plate, and a movable end of which is hinged to the outer side surface of the outer claw of the second gripper.

[0017] The outer claw is connected to the second shoe-shaped body through a key groove and is slidably matched.

[0018] The image recognition device is equipped with an image processing model and an image recognition model. After processing the image of the roller to be predicted collected by the visual component, the image processing model outputs an enhanced roller image to the image recognition model, which performs roller position detection and angle prediction on the image, determines whether the roller angle is correct, and outputs the angle to be adjusted to the control system.

[0019] The image processing model is obtained by training the deep learning model with the original training set;

[0020] The image recognition model is obtained by training a convolutional neural network using an expanded training set;

[0021] The expanded training set is obtained using the image processing model, including the following process:

[0022] S1. Collect images of rollers with different roll angles in a gripping state during operation, and pre-process them to form the original training set with labels of roller categories and angle information;

[0023] S2. Build a generator for the generative adversarial network;

[0024] S3. Build a discriminator for the generative adversarial network.

[0025] S4. Training the generative adversarial network using the original training set, and alternately updating the discriminator and the generator until the loss function converges or the quality of the generator output image reaches a preset standard, and the training is completed to obtain the image processing model;

[0026] S5. Input the original training set into the image processing model to obtain an enhanced roller image corresponding to the original roller image, and obtain a corresponding label file containing the roller category and coordinates, update the label of the enhanced roller image and retain the angle information to form an expanded training set.

[0027] Build the generator of the generative adversarial network, including:

[0028] Build the basic encoder and decoder structure of U-Net;

[0029] A convolutional layer, batch normalization, and activation function are added after each encoding layer of the encoder, a maximum pooling layer is used to reduce the spatial resolution of the feature map, and a residual block is introduced after each convolutional layer. A self-attention module is introduced between the last layer of the encoder and the first layer of the decoder to calculate global dependencies. The feature weights of the roller installation groove part in the image are enhanced by calculating the attention map, so that the features of the roller installation groove part are fully expressed in the encoding stage.

[0030] Each decoding layer of the decoder is combined with the corresponding encoding layer through a skip connection. A deconvolution layer, batch normalization and activation function are added after each decoding layer, and a residual block is introduced after each deconvolution layer. A channel attention mechanism is introduced after each decoding layer, and global features are extracted through global average pooling and global maximum pooling. The attention weight is calculated through a fully connected layer. The last decoding layer outputs the enhanced roller image, and the pixel values ​​of the generated image are mapped to a reasonable range through the activation function.

[0031] The discriminator is provided with a residual block; the discriminator takes the original roller image and the enhanced roller image as input, and uses the information of the original roller image to assist in judging the enhancement quality.

[0032] The image recognition model is obtained by training the convolutional neural network with an expanded training set. The training process includes:

[0033] The expanded training set is input into the YOLOv5 network model. The head part of the YOLOv5 network model adds a branch for predicting the identified roller angle based on the roller position detection in the target image. The branch is used to predict the angle of the roller structure using the category condition judgment. The original loss function of the YOLOv5 network model is used to calculate the loss of position detection. , increase the angle prediction loss , use the mean square error to calculate the error between the predicted angle and the true angle, and combine the two parts of the loss to form the total loss:

[0034] Where, α represents the weight of target prediction loss, βRepresents the weight of the angle prediction loss;

[0035] Backpropagation is performed using the total loss function to calculate the gradient of the loss with respect to the model parameters:

[0036] Where, Represents the gradient of the position detection loss with respect to the model parameters; represents the gradient of the angle prediction loss with respect to the model parameters;

[0037] The optimizer is used to update the model parameters according to the calculated gradient until the predetermined number of training times is reached or the loss converges, and the training is completed.

[0038] The beneficial effects of the present invention are as follows:

[0039] The present invention achieves accurate grasping, identification, and angle calibration of rollers, improves the safety and reliability of the roller installation process, reduces labor intensity and operation risks, and promotes efficient production. Specifically, it has the following advantages:

[0040] 1. The gripper assembly of the present invention can adapt to the changes in the diameter and weight of the roller within a certain range. Two grippers and an angle adjustment device provide effective support for the roller from three directions to prevent the roller from shifting or even slipping during transportation. At the same time, the angle of the roller in the clamping state is automatically identified by the visual device and the image recognition device. The image recognition device performs image repair and enhancement through the generative adversarial network model. The YOLOv5 model is used to determine whether the roller angle is correct and output the angle to be adjusted to the control system, which controls the gripper assembly to adjust the roller roll angle, thereby realizing the identification and calibration of the roller roll angle. This solves the problem of inaccurate roller installation caused by changes in the roller roll angle due to factors such as equipment shaking during transportation. It ensures the accurate installation of the rollers and further promotes the unmanned construction of the roller replacement process.

[0041] 2. The second tile-shaped body of the hand claw structure of the present invention is a component that is in direct contact with the roller. It can move relative to the hand body through the sliding key and the sliding groove to achieve synchronous movement with the roller, ensuring the stability of the roller support during the angle adjustment process.

[0042] 3. The image recognition device of the present invention utilizes both an image processing model and an image recognition model. The image processing model's generative adversarial network incorporates residual blocks, using skip connections to mitigate the vanishing gradient problem in deep networks. Combined with a U-Net and attention mechanism, the device retains multi-scale features while highlighting globally important regions, effectively improving the stability of model training. The image recognition model incorporates a detection branch for roller angles in YOLOv5 and optimizes the loss function, further improving target detection performance and accuracy.

[0043] Other features and advantages of the present invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Schematic diagram of the overall structure of an embodiment of the present invention.

[0045] Figure 2 for Figure 1 Schematic diagram of the cross-section structure.

[0046] Figure 3 Schematic diagram of the training and use process of the image processing and recognition model carried by the image recognition device according to an embodiment of the present invention.

[0047] Figure 4 This is a structural diagram of the generator of the generative adversarial network according to an embodiment of the present invention.

[0048] Figure 5 This is a structural diagram of the discriminator of the generative adversarial network according to an embodiment of the present invention.

[0049] In the figure: 1. First gripper; 2. Second gripper; 3. Third gripper; 4. Outer gripper; 5. Second tile-shaped body; 6. Return spring; 7. Base; 8. Outer tooth; 9. First rotary drive mechanism; 10. Support arm; 11. Top plate; 12. Second rotary drive mechanism; 13. Drive gear; 14. Straight rack; 15. Guide rail; 16. Stop block; 17. Collection device; 18. Illuminator; 19. Support frame; 20. Connecting frame; 21. Third drive mechanism; 22. Visual component; 23. Guide seat; 24. First tile-shaped body. DETAILED DESCRIPTION

[0050] The specific embodiments of the present invention are described below with reference to the accompanying drawings.

[0051] See also Figures 1 to 2 , a roller adaptive gripping manipulator gripper system of this embodiment includes a gripper assembly, a gripper motion support assembly, an angle adjustment drive mechanism, a visual assembly 22 and an image recognition device;

[0052] The gripper assembly includes a first gripper 1, a second gripper 2 and a third gripper 3 distributed along the circumferential direction;

[0053] The structure of the third gripper 3 includes a guide seat 23, in which a first shoe-shaped body 24 is provided for sliding engagement therewith. The inner surface of the first shoe-shaped body 24 is a concave arc surface for contacting the circumferential surface of the roller. The first shoe-shaped body 24 is driven by the angle adjustment drive mechanism and can rotate circumferentially along the guide seat 23.

[0054] The first hand claw 1 and the second hand claw 2 are located on the upper part of the third hand claw 3 and have the same structure. They both include an outer claw 4 and a second shoe-shaped body 5. The second shoe-shaped body 5 is located inside the outer claw 4 and slides with the outer claw 4. A return spring 6 that provides preload and return force is provided between the outer claw 4 and the second shoe-shaped body 5. The inner surface of the second shoe-shaped body 5 is an inner concave arc surface for contacting the circumferential surface of the roller.

[0055] The gripper motion support assembly includes a movable top plate 11. The outer jaw 4 of the first gripper 1 is directly hinged to the movable top plate 11 and can adaptively change its posture when in direct contact with rollers of different sizes. The outer jaw 4 of the second gripper 2 is hinged to the top plate 11 via a connecting frame 20 and can actively adjust its posture via a third drive mechanism 21.

[0056] There are two groups of visual components 22, each used to collect images of the mounting grooves on both sides of the roller. Each group of structures includes a collection device 17, an illuminator 18 and a support frame 19. The support frame 19 is installed on the top of the top plate 11 and extends outward. The collection device 17 is installed at the end of the support frame 19, and the illuminator 18 is installed in the slot-shaped through hole on the support frame 19.

[0057] The image recognition device outputs the required rotation angle of the roller to the control system based on the image collected by the visual component 22. The control system controls the angle adjustment drive mechanism to operate so that the third gripper 3 drives the roller to rotate to the correct installation angle.

[0058] like Figure 2 As shown, the angle adjustment drive mechanism includes a second rotary drive mechanism 12, a guide rail 15, and a straight rack 14 that slides with the guide rail 15. The second rotary drive mechanism 12 is provided with at least one output end, and the at least one output end is connected to a drive gear 13. The drive gear 13 is engaged with the straight rack 14 for transmission, driving the straight rack 14 to move linearly along the guide rail 15; the outer side surface of the first shoe-shaped body 24 is provided with external teeth 8 that are engaged with the straight rack 14 for transmission along the circumferential direction. Figure 2 The faces rendered in medium-dark colors are section planes.

[0059] As a preferred embodiment, the outer claw 4 and the second shoe 5 are connected via a key groove and are slidably fitted.

[0060] As an optional embodiment, protrusions are provided at both ends of the straight rack 14, and a stopper 16 is provided on the inner side of the guide rail 15, which is used to cooperate with the protrusions to constrain the displacement of the straight rack 14, thereby controlling the angle adjustment range of the first tile-shaped body 24 of the third hand claw 3.

[0061] The driving gear 13 , the spur rack 14 , the external teeth 8 and the guide rail 15 are preferably arranged in two sets respectively.

[0062] As an optional embodiment, the gripper motion support assembly, the angle adjustment drive mechanism, and the guide seat 23 are all installed on the base 7, wherein the guide rail 15 is semi-enclosedly arranged in the base 7.

[0063] like Figure 1 As shown, the gripper motion support assembly also includes a first rotation drive mechanism 9 and a support arm 10. The first rotation drive mechanism 9 is provided with at least one output end, and the at least one output end is connected to one end of the support arm 10, and the other end of the support arm 10 is connected to the top plate 11.

[0064] The first rotary drive mechanism 9 and the second rotary drive mechanism 12 are preferably rotary cylinders.

[0065] As an optional embodiment, the third driving mechanism 21 is a linear driving mechanism, a fixed end of which is hinged to the top plate 11 , and a movable end is hinged to the outer side surface of the outer claw 4 of the second gripper 2 .

[0066] The third driving mechanism 21 is preferably a telescopic hydraulic cylinder.

[0067] As a specific embodiment, the return spring 6 is arranged along the circumferential direction, and its two ends are respectively connected to the outer claw 4 and the second shoe-shaped body 5.

[0068] The acquisition device 17 may specifically be an industrial camera.

[0069] The working principle of the roller adaptive gripping manipulator system of this embodiment is as follows:

[0070] The first gripper 1 and the second gripper 2 are arranged opposite each other and are installed on the upper part of the third gripper 3. The three grippers work together to apply force to the roller in three directions and are the main components for grasping and fixing the roller. The first gripper 1 and the second gripper 2 have the same structure, both including an outer claw 4 and a second shoe 5. The second shoe 5 is arranged on the inner side of the outer claw 4 and is the component that directly contacts the roller. The second shoe 5 and the outer claw 4 are connected by a keyway and can slide relative to each other, and can move synchronously with the roller. The reset spring 6 is connected to the second shoe 5 and is arranged between the outer claws 4 to ensure that the position of the second shoe 5 is restored after the roller is installed.

[0071] The third gripper 3, while providing support for the roller, cooperates with the angle adjustment device to adjust the roller's roll angle. A spur rack 14 engages synchronously with the drive gear 13 and the external gears 8, achieving a two-stage transmission. A guide rail 15 guides and limits the movement of the spur rack 14. Stoppers 16, located at each end of the guide rail 15, control the travel of the spur rack 14 and, consequently, the angle adjustment range of the third gripper 3.

[0072] The motion strategy of the manipulator gripper system of the roller adaptive gripper in this embodiment is:

[0073] During the roller grabbing stage, the third gripper 3 first contacts the roller, providing a vertical upward radial support to the roller, and then the first gripper 1 and the second gripper 2 move downward under the drive of the first rotary drive mechanism 9. After the second tile-shaped body 5 of the first gripper 1 contacts the roller, it can rotate around the hinge part between the top plate, thereby adaptively adjusting its own posture according to the outer peripheral size of the roller, so that the inner wall of the second tile-shaped body 5 fits the roller; then the third drive mechanism 21 drives the second gripper 2 to actively adjust its posture and stick to the roller, thereby realizing three-point support for the roller and completing the roller grabbing action.

[0074] During the roller installation phase, the acquisition device 17 captures images of the mounting grooves on both sides of the roller in the gripping state. The image processing and recognition of the deep learning network of the image recognition model carried by the recognition device obtain the roll angle of the roller. The control system then issues an adjustment command to activate the angle adjustment drive mechanism: Specifically, the second rotary drive mechanism 12 drives the gear 13 to rotate. The drive gear 13 acts as the driving wheel and performs the first stage transmission with the spur rack 14. The spur rack 14 moves linearly along the guide rail 15. The spur rack 14 and the external teeth 8 perform the second stage transmission, causing the first shoe 24 of the third gripper 3 to rotate circumferentially along the guide seat and drive the gripped roller to adjust the roll angle. At the same time, the second shoe 5 of the first gripper 1 and the second gripper 2 rotate with the roller to ensure that the roller is always clamped during the angle adjustment process.

[0075] After the roller is installed, the control system issues an instruction and the angle adjustment drive mechanism is activated again to reset the first shoe 24 of the third gripper 3, and the second shoe 5 of the first gripper 1 and the second gripper 2 is reset with the help of the reset spring 6.

[0076] The image recognition device of this embodiment is equipped with an image processing model and an image recognition model. The image processing model performs image repair and enhancement on the roller image to be predicted collected in real time by the visual device through a generative adversarial network model, and outputs the enhanced roller image to the image recognition model. The model detects the position and angle of the roller in the image through the YOLOv5 model, determines whether the roller angle is correct and outputs the angle to be adjusted to the control system, controls the gripper assembly to adjust the roller rolling angle, and realizes the recognition and calibration of the roller rolling angle.

[0077] The image processing model is obtained by training a deep learning model with an original training set.

[0078] The image recognition model is obtained by training a convolutional neural network with an expanded training set.

[0079] See also Figure 3 The expanded training set is obtained using the image processing model, including the following process:

[0080] S1. Collect images of rollers with different roll angles in the gripping state during operation and pre-process them to form an original training set with roller category and angle labels.

[0081] Specifically, the original images obtained are images under underground lighting, dust and other environments. The preprocessing process includes operations such as resizing, normalizing, and marking angle information on the image samples;

[0082] S2. Combining the encoder-decoder structure of U-Net, introducing residual blocks and attention mechanisms, and constructing the generator of the generative adversarial network;

[0083] S3. Combine the residual block to build the discriminator of the generative adversarial network;

[0084] S4, using the original training set to train the generative adversarial network, the discriminator and the generator are alternately updated until the loss function converges or the quality of the generator output image reaches a preset standard, and the training is completed to obtain a trained generative adversarial network model, i.e., the image processing model;

[0085] S5. Input the original training set into the image processing model to obtain an enhanced roller image corresponding to the original roller image, and obtain a corresponding label file containing the roller category ID and coordinates (position information), update the label of the enhanced roller image and retain the angle information to form an expanded training set.

[0086] See also Figure 4 , the generator of the generative adversarial network is constructed, including:

[0087] Build the basic encoder and decoder structure of U-Net;

[0088] A convolutional layer, batch normalization, and activation function are added after each encoding layer of the encoder. A maximum pooling layer is used to reduce the spatial resolution of the feature map. A residual block is introduced after each convolutional layer to improve feature extraction capabilities and alleviate the gradient vanishing problem. A self-attention module is introduced between the last layer of the encoder and the first layer of the decoder to calculate global dependencies. The feature weights of the roller installation groove part in the image are enhanced by calculating the attention map, so that the features of the roller installation groove part are fully expressed in the encoding stage.

[0089] In the decoder part, the spatial resolution of the feature map is gradually restored. Each decoding layer is combined with the corresponding encoding layer through jump connections. A deconvolution layer, batch normalization and activation function are added after each decoding layer. A residual block is introduced after each deconvolution layer to enhance the feature recovery ability; a channel attention mechanism is introduced after each decoding layer to calculate the importance weight of each channel, which helps to refine the reconstructed image; global features are extracted through global average pooling and global maximum pooling, and the attention weight is calculated through the fully connected layer to adjust the channel features and enhance the contribution of the feature channels related to the installation slot, thereby improving the detail performance and overall image quality of the area; the last decoding layer outputs the enhanced roller image, and the pixel values ​​of the generated image are mapped to a reasonable range through the activation function.

[0090] See also Figure 5 The discriminator of the constructed generative adversarial network provides sufficient complexity to extract effective features by combining residual blocks, preventing excessive discriminator complexity from affecting training balance and further improving the model's operational efficiency. The discriminator takes both the original roller image and the enhanced roller image as input, and uses information from the original roller image to assist in determining the enhancement quality.

[0091] The step of training the generative adversarial network using the original training set includes:

[0092] Set and classify data model labels, use a learning rate scheduler to dynamically adjust the learning rate during training, and use label smoothing to vary the true label from 1 to 0.95 and the false label from 0 to 0.05 to prevent overfitting of the discriminator. When the loss functions of the discriminator and generator converge to a stable value or the enhanced roller image output by the generator meets the expected quality standard, training is considered nearly complete and the image processing model is obtained.

[0093] Specifically, the image processing model can not only output an enhanced roller image, but also output a feature mask that highlights the location of the roller and generates a labeling file containing the category ID and coordinate location of the roller structure in the image for the next image recognition task.

[0094] The image recognition model is obtained by training a convolutional neural network with an expanded training set. The training process includes: using the expanded training set as the input of the YOLOv5 network model; performing target detection (position detection) and roller angle prediction of the roller structure in the target image, calculating the corresponding loss and implementing backpropagation; using an optimizer to update the model parameters according to the calculated gradient until a predetermined number of training times is reached or the loss converges. The training can be considered completed, and a trained YOLOv5 network model, i.e., an image recognition model, is obtained.

[0095] Specifically, the YOLOv5 network model uses Backbone to extract deep features, and then uses Neck, which is a combination of a feature pyramid network and a PANet path aggregation network, to fuse multi-scale features, further improving the model's ability to detect targets of different scales. The final detection results are then generated through the Head network, including predicting the category, position, and confidence of each anchor box.

[0096] The head part of the YOLOv5 network model of this embodiment adds a branch for predicting the identified roller angle based on the roller position detection of the roller structure in the target image. The branch is used to predict the angle of the roller structure using the category condition judgment, and the original loss function of the YOLOv5 network model is used to calculate the loss of position detection. , increase the angle prediction loss , use the mean square error to calculate the error between the predicted angle and the true angle, and combine the two parts of the loss to form the total loss:

[0097] Where, α represents the weight of the target prediction loss, β represents the weight of the angle prediction loss;

[0098] Backpropagation is performed using the total loss function to calculate the gradient of the loss with respect to the model parameters:

[0099] Where, Represents the gradient of the position detection loss with respect to the model parameters; Represents the gradient of the angle prediction loss with respect to the model parameters.

[0100] Those skilled in the art will understand that the foregoing descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will be able to modify the technical solutions described in the foregoing embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A roller adaptive gripping robot claw system, characterized in that: It includes a gripper assembly, a gripper motion support assembly, an angle adjustment drive mechanism, a visual assembly (22) and an image recognition device; The gripper assembly comprises a first gripper (1), a second gripper (2) and a third gripper (3) distributed along the circumferential direction; The structure of the third hand gripper (3) includes a guide seat (23), wherein a first tile-shaped body (24) is provided in the guide seat (23) and is slidably matched with the first tile-shaped body (24), and the inner surface of the first tile-shaped body (24) is an inner concave arc surface for contacting the circumferential surface of the roller; the first tile-shaped body (24) is driven by the angle adjustment drive mechanism and can rotate circumferentially along the guide seat (23); The first hand claw (1) and the second hand claw (2) are located on the upper part of the third hand claw (3) and have the same structure, both comprising an outer claw (4) and a second tile-shaped body (5); the second tile-shaped body (5) is located on the inner side of the outer claw (4) and is slidably matched with the outer claw (4); a return spring (6) for providing preload and return force is provided between the outer claw (4) and the second tile-shaped body (5); the inner surface of the second tile-shaped body (5) is an inner concave arc surface for contacting the circumferential surface of the roller; The gripper motion support assembly comprises a movable top plate (11), wherein the outer claw (4) of the first gripper (1) is directly hinged to the movable top plate (11); the outer claw (4) of the second gripper (2) is hinged to the top plate (11) via a connecting frame (20), and the posture is adjusted via a third driving mechanism (21); The visual component (22) is used to collect images of the mounting grooves on both sides of the roller, and includes a collection device (17), an illuminator (18) and a support frame (19), wherein the support frame (19) is installed on the top of the top plate (11), and the collection device (17) and the illuminator (18) are installed on the support frame (19); The image recognition device outputs the required rotation angle of the roller to the control system based on the image collected by the visual component (22), and the control system controls the angle adjustment drive mechanism to operate so that the third hand claw (3) drives the roller to rotate to the correct installation angle; The image recognition device is equipped with an image processing model and an image recognition model. After the image processing model processes the image of the roller to be predicted collected by the visual component (22), it outputs an enhanced image of the roller to the image recognition model, which performs roller position detection and angle prediction on the image, determines whether the roller angle is correct, and outputs the angle to be adjusted to the control system; The image processing model is obtained by training the deep learning model with the original training set; The image recognition model is obtained by training the convolutional neural network with an expanded training set, and the training process includes: The expanded training set is input into the YOLOv5 network model. The head part of the YOLOv5 network model adds a branch for predicting the identified roller angle based on the roller position detection in the target image. The branch is used to predict the angle of the roller structure using the category condition judgment. The original loss function of the YOLOv5 network model is used to calculate the loss of position detection. , increase the angle prediction loss , use the mean square error to calculate the error between the predicted angle and the true angle, and combine the two parts of the loss to form the total loss: , Where, α represents the weight of target prediction loss, β Represents the weight of the angle prediction loss; Backpropagation is performed using the total loss function to calculate the gradient of the loss with respect to the model parameters: , Where, Represents the gradient of the position detection loss with respect to the model parameters; represents the gradient of the angle prediction loss with respect to the model parameters; The optimizer is used to update the model parameters according to the calculated gradient until the predetermined number of training times is reached or the loss converges, and the training is completed.

2. The roller adaptive gripping manipulator claw system according to claim 1 is characterized in that: The angle adjustment drive mechanism comprises a second rotary drive mechanism (12), a guide rail (15), and a spur rack (14) that is slidably engaged with the guide rail (15). The second rotary drive mechanism (12) is provided with at least one output end, and the at least one output end is connected to a driving gear (13). The driving gear (13) is meshed with the spur rack (14) for transmission, and drives the spur rack (14) to move linearly along the guide rail (15). The outer side surface of the first tile-shaped body (24) is provided with external teeth (8) that are meshed with the spur rack (14) for transmission along the circumferential direction.

3. The roller adaptive gripping manipulator system according to claim 1 is characterized in that: The gripper motion support assembly further comprises a first rotary drive mechanism (9) and a support arm (10), wherein the first rotary drive mechanism (9) is provided with at least one output end, wherein the at least one output end is connected to one end of the support arm (10), and the other end of the support arm (10) is connected to the top plate (11).

4. The roller adaptive gripping manipulator claw system according to claim 1 is characterized in that: The third driving mechanism (21) is a linear driving mechanism, a fixed end of which is hinged to the top plate (11), and a movable end of which is hinged to the outer side surface of the outer claw (4) of the second gripper (2).

5. The roller adaptive gripping manipulator claw system according to claim 1 is characterized in that: The outer claw (4) and the second shoe-shaped body (5) are connected and slidably fitted via a keyway.

6. The roller adaptive gripping manipulator claw system according to claim 1 is characterized in that: The expanded training set is obtained using the image processing model, including the following process: S1. Collect images of rollers with different roll angles in a gripping state during operation, and pre-process them to form the original training set with labels of roller categories and angle information; S2. Build a generator for the generative adversarial network; S3. Build a discriminator for the generative adversarial network. S4. Training the generative adversarial network using the original training set, and alternately updating the discriminator and the generator until the loss function converges or the quality of the generator output image reaches a preset standard, and the training is completed to obtain the image processing model; S5. Input the original training set into the image processing model to obtain an enhanced roller image corresponding to the original roller image, and obtain a corresponding label file containing the roller category and coordinates, update the label of the enhanced roller image and retain the angle information to form an expanded training set.

7. The roller adaptive gripping manipulator claw system according to claim 6 is characterized in that: Build the generator of the generative adversarial network, including: Build the basic encoder and decoder structure of U-Net; A convolutional layer, batch normalization, and activation function are added after each encoding layer of the encoder, a maximum pooling layer is used to reduce the spatial resolution of the feature map, and a residual block is introduced after each convolutional layer. A self-attention module is introduced between the last layer of the encoder and the first layer of the decoder to calculate global dependencies. The feature weights of the roller installation groove part in the image are enhanced by calculating the attention map, so that the features of the roller installation groove part are fully expressed in the encoding stage. Each decoding layer of the decoder is combined with the corresponding encoding layer through a skip connection. A deconvolution layer, batch normalization and activation function are added after each decoding layer, and a residual block is introduced after each deconvolution layer. A channel attention mechanism is introduced after each decoding layer, and global features are extracted through global average pooling and global maximum pooling. The attention weight is calculated through a fully connected layer. The last decoding layer outputs the enhanced roller image, and the pixel values ​​of the generated image are mapped to a reasonable range through the activation function.

8. The roller adaptive gripping manipulator claw system according to claim 6, characterized in that: The discriminator is provided with a residual block; the discriminator takes the original roller image and the enhanced roller image as input, and uses the information of the original roller image to assist in judging the enhancement quality.

Images