Intelligent material grabbing system and method based on improved YOLOv5

Abstract

The invention relates to the technical field of intelligent material recognition and grabbing, in particular to an intelligent material grabbing system and method based on improved YOLOv5. According to the improved YOLOv5 algorithm, a C3 module in a backbone network is replaced with a C2f module, a BRA attention mechanism is introduced into a Neck layer to improve the feature extraction capacity, a decoupling detection head is adopted to optimize the detection performance, and the detection efficiency is improved. And the detection precision is improved by using an EIoU loss function. The grabbing device is composed of a bottom plate, a sliding table, a lead screw, a guide rail sliding block, a camera support, a depth camera, a stepping motor and a mechanical grabbing device, the stepping motor drives the lead screw to rotate and drives the guide rail sliding block to move, and the mechanical grabbing device is connected with one side of the guide rail sliding block and used for grabbing materials to a designated position; the improved YOLOv5 algorithm is combined with the grabbing device, automatic grabbing of materials from an uncertain position to a designated position is achieved, manual intervention is reduced, and meanwhile the grabbing efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent material recognition and grasping, and in particular to an intelligent material grasping system and method based on improved YOLOv5. Background Art

[0002] Intelligent material grasping refers to the use of automation technology and machine vision to achieve the automatic identification and grasping of materials in different scenarios. Intelligent material grasping is very important for improving the efficiency of material grasping, and intelligent material grasping also directly affects production efficiency.

[0003] In the current intelligent material handling system, the materials to be grasped are placed in their respective target locations in advance. After these items are classified and confirmed by using the identification module, the materials are moved to the designated location with the help of a special grasping device. However, since multiple materials may have similar characteristics, this may cause the grasping device to be unable to accurately distinguish and grasp the specified materials, resulting in different types of materials being mistakenly moved to the same location. This situation not only prolongs the time required for material grasping, but also greatly reduces the overall efficiency of the system. Given that intelligent material handling systems are widely used in a variety of scenarios, in some cases, existing systems may not be able to fully adapt to the specific needs of specific application scenarios.

[0004] The current intelligent material grasping system has problems such as low recognition accuracy, low grasping efficiency, single application scenario, difficulty in meeting real-time requirements, and difficulty in handling changes in material position. How to accurately and quickly identify and grasp different materials is an important step in improving the working efficiency of the intelligent material grasping system. Summary of the invention

[0005] The purpose of the present invention is to provide an intelligent material grasping system and method based on improved YOLOv5, which can quickly and accurately identify and grasp different materials, can be applied to different application scenarios, improve the material grasping efficiency, and improve the material grasping fault tolerance. The intelligent material grasping system is unmanned throughout the process, which not only reduces the dependence on manual processing, but also significantly improves efficiency.

[0006] In order to achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions:

[0007] An intelligent material grabbing system based on improved YOLOv5 comprises: a base plate, a slide, a lead screw, a guide rail slider, a camera bracket, a depth camera, a stepper motor and a mechanical grabbing device; the base plate is provided with a slide, the guide rail slider is provided with the slide, the stepper motor is provided with the base plate, the lead screw is fixed to the inner side of the slide, and is provided between the slide and the guide rail slide, the guide rail slider is slidably connected with the lead screw through a through groove, the fixed output end of the stepper motor is fixedly connected with one end of the lead screw, the lead screw is driven to rotate by the stepper motor, the camera bracket is provided at the top of the two slides, the depth camera is provided at the center of the camera bracket, and is used to photograph and obtain material information and record it, and the mechanical grabbing device is connected to one side of the guide rail slider, and is used to grab the material to a specified position.

[0008] Furthermore, the mechanical grasping device includes gear one, gear two, a chain, a servo, stepper motor two, a mechanical shaft, a transmission shaft and an end actuator, the mechanical shaft is rotatably connected to the opposite side of the guide rail slider with a through groove, the transmission shaft is fixedly connected to a circular through hole at one end of the mechanical shaft, the gear one is fixedly connected to the center position of the transmission shaft, the servo body is fixed in a groove at one end of the opposite side of the guide rail slider with the through groove, the output rocker of the servo is fixed on one side of the mechanical shaft that is rotatably connected to the guide rail slider, the stepper motor two is embedded in the opposite side of the guide rail slider with the through groove, the end of the transmission shaft is rotatably connected to the fixed output end of the stepper motor two, the gear one is meshed with teeth on the top, and the chain is meshed with gear one and gear two respectively through its tooth grooves.

[0009] Furthermore, the steering gear, the second stepper motor and the mechanical shaft are provided in two groups, which are symmetrically installed on both sides of the transmission shaft at the rear side of the mechanical grasping device.

[0010] Furthermore, the transmission shaft is arranged into two groups, which are respectively installed on the front and rear sides of the mechanical grasping device. The ends on both sides of the transmission shaft installed on the rear side of the mechanical grasping device are respectively fixed to the guide rail sliders, and the two ends of the transmission shaft installed on the front side of the mechanical grasping device are slidingly connected with the through holes of the two groups of symmetrically installed mechanical shafts.

[0011] Furthermore, the end effector includes a fixed shaft, gear three, gear four, a rotating shaft, a straight cylinder and a mechanical claw, one end of the fixed shaft is fixedly mounted on the boss surface on the outside of the mechanical shaft, gear four is fixedly mounted on the boss above the other end of the fixed shaft, one side of the straight cylinder is fixedly connected to the end of the transmission shaft installed on the front side of the mechanical grasping device, gear three is fixedly connected to one side of the straight cylinder through a center hole, gear four and gear three are meshed with each other, so that the rotation of gear three drives gear four to rotate, one end of the rotating shaft is fixedly mounted on the center hole of gear four, and the other end is fixedly connected to the end of the mechanical claw.

[0012] Furthermore, two groups of the end effectors are provided and are symmetrically installed with the center of the transmission shaft as the symmetry axis, and the end effectors are fixed to the mechanical shaft through a fixed shaft and a straight cylinder.

[0013] Furthermore, the stepper motor, the lead screw, the slide table, and the guide rail slider are provided in two groups and are symmetrically installed on both sides of the base plate.

[0014] Furthermore, the depth camera captures and records material information, and the collected material information is processed using an improved YOLOv5 target detection algorithm after preprocessing to obtain material category and location information.

[0015] On the other hand, the present invention proposes an improved method for the YOLOv5 algorithm in the above-mentioned device, specifically comprising:

[0016] S1: Replace the C3 module in the backbone network with the C2f module to reconstruct the backbone network;

[0017] S2: Introducing the BRA mechanism in the Neck layer to improve feature extraction capabilities;

[0018] S3: Introducing a decoupled detection head at the Head layer to optimize detection performance;

[0019] S4: Use the EIoU loss function as the network loss function to improve detection accuracy.

[0020] Furthermore, the step S1 specifically includes:

[0021] After the first convolution operation, the C2f module divides the feature map into two parts. One part is processed by n Bottleneck modules, and the other part is directly output. Finally, the results of the two parts are spliced ​​in the channel dimension and the final feature map is generated after the second convolution operation. This branch design increases the nonlinear ability and representation ability of the network. By splicing the features of different branches in the channel dimension, feature fusion is achieved. The spliced ​​features will contain information from different branches, enriching the representation ability of the features.

[0022] The Bottleneck module contains a 1×1 convolution layer and a 3×3 convolution layer. The first 1×1 convolution layer is used for dimensionality reduction, and the second 3×3 convolution layer is used for calculation and output.

[0023] Furthermore, the step S2 specifically includes:

[0024] The BRA attention mechanism is a two-layer routing attention mechanism. The BRA attention mechanism is introduced between the Conv module and the downsampling module of the Neck layer in the YOLOv5 neural network. The BRA attention mechanism will input an image X∈RH×W×C , divided into S×S non-overlapping regions, each containing eigenvectors, we can transform X into Obtained by linear mapping The specific expressions of Q, K, and V are as follows:

[0025] Q=X r W q

[0026] K=X r W k

[0027] V=X r W v

[0028] Among them, W q ,W k ,W v ∈R C×C The projection weights belong to query, key, and value respectively; a directed graph is constructed to find the area that each given area should participate in, that is, Calculate Q r and K r The adjacency matrix of the regional correlation is:

[0029] A r =Q r (K r ) T

[0030] Then only the top k connections of each region are kept to prune the correlation graph,

[0031] I r =topkIndex(A r )

[0032] That is I r The i-th row of contains the indices of the top k most relevant regions of the i-th region.

[0033] Using the region-to-region index routing matrix I r For each query token in region i, it will look at all key-value pairs in the union of k routing regions, indexed by Aggregate the tensor of key and value:

[0034] K g =gather(K,I r ),V g =gather(V,I r )

[0035] Where K g and V g It is the tensor of the clustered key and value, and then the attention operation is applied to the clustered KV pairs:

[0036] O=Attention(Q,K g ,V g )+LCE(V)

[0037] The above function LCE(·) is parameterized using depthwise convolution, with the parameter value set to 5.

[0038] Furthermore, the step S3 specifically includes: the input of the decoupling detection head first uses a 1×1 convolution layer to reduce the number of channels to 256, and then enters two branches respectively, each branch has two 3×3 convolution layers, and the two branches are used for regression tasks and classification tasks respectively to improve the flexibility of the model; a multi-branch detection head is used for calculation to improve the detection accuracy and accelerate the network convergence, thereby improving the generalization ability and robustness of the model.

[0039] Furthermore, the step S4 specifically includes:

[0040] The loss function is the EIoU loss function, and the formula is as follows:

[0041]

[0042] Where L IoU is the intersection-over-union ratio of the predicted box and the actual box, B is the predicted box, and B gt is the real box, b is the center point of B, b gt For B gt , ρ(·) is the Euler distance between the predicted box and the center of the real box, and c is the distance between B and B gt The diagonal length of the minimum bounding box between the two, w and h are the width and height of the prediction box B, w gt and h gt For B gt The width and height of C w For B and B gt Minimum bounding box width, C h For B and B gt Minimum bounding box height.

[0043] The EIoU loss function splits the aspect ratio loss into the difference between the predicted width and height and the minimum bounding box, which includes overlap loss, center distance loss, and width and height loss. Compared with the CIoU loss function, it converges faster and has higher detection accuracy.

[0044] The backbone network of YOLOv5 consists of a structure that encapsulates three modules: the Conv module, the C3 module, and the SPPF module. The C3 modules in the backbone network are replaced by C2f modules to perform feature conversion on the input data, reducing storage overhead and computational complexity.

[0045] The BRA mechanism refers to a two-layer routing attention mechanism, which is introduced between the Conv module and the downsampling module in the Neck layer of the YOLOv5 algorithm, and integrated with the FPN and PAN structures to improve the feature extraction capability of the model.

[0046] The Head layer uses a decoupled detection head to replace the original coupled head of YOLOv5. The decoupled detection head separates the classification task from the regression task. One branch is used for the regression task to predict the location information of the bounding box, and the other branch is used for the classification task to predict the category probability of the target, thereby improving the detection effect of the network.

[0047] The network's loss function consists of three parts: object confidence loss, object classification loss, and predicted box position loss. The loss function calculates the difference between the predicted target box and the actual target box. The loss function uses the EIoU loss function, which speeds up the convergence of the model and improves the detection accuracy of the model.

[0048] Beneficial effects of the present invention:

[0049] This system significantly enhances the nonlinear ability and feature representation ability of the network by replacing the C3 module in the backbone network of the YOLOv5 algorithm with the C2f module. The C2f module divides the feature map into two parts, performs different convolutions on each part, and then achieves feature fusion through splicing, thereby improving the model's ability to express complex patterns. At the same time, it is also more lightweight in structure, reducing storage overhead and computational complexity, allowing the system to run on devices with weaker computing power. In addition, the BRA attention mechanism is introduced into the Neck layer, which filters the least relevant features in the feature map in a dynamic sparse manner and retains some important regional features, solving the problem of a large amount of redundant information in the feature map of the existing model, thereby improving the model's feature extraction ability and optimizing The performance of the model is improved while controlling the complexity of the network. The design of the decoupled detection head improves the flexibility and efficiency of detection. The classification and regression tasks are processed separately, and different tasks can be optimized independently, which accelerates the convergence process of the network. It not only improves the flexibility of detection, but also accelerates the convergence process of the network, making the system faster and more effective when facing real-time material recognition. The EIoU loss function is introduced to better reflect the difference between the predicted box and the real box by performing detailed calculations on the intersection of the predicted box and the real box, the center distance, and the width and height difference, which solves the shortcomings of the traditional IoU loss function in accuracy and improves the system's fault tolerance when dealing with positioning errors. These improvements have significantly improved the convergence and robustness of the algorithm, and also optimized the model structure.

[0050] In terms of mechanical structure, the parameters of the stepper motor and the servo are adjusted to ensure high-precision execution of the grasping action. The system symmetrically installs two sets of stepper motors, lead screws and other components on the base plate, so that the mechanical grasping device can adapt to materials of different sizes and shapes. This structure also effectively solves the problem of insufficient flexibility of traditional robotic arms when grasping, allowing multi-directional and multi-angle motion control, so that the mechanical grasping device can flexibly adjust its position and posture in three-dimensional space to grasp materials. This structural design not only achieves precise grasping, but also has good adaptability and can be applied to different scenarios.

[0051] The present invention can quickly and accurately obtain information about the material to be grasped by using the shooting function of the depth camera and the improved YOLOv5 target detection algorithm, and grasp the material through the cooperation of the bottom plate, the stepper motor 1, the slide, the screw rod, the guide rail slider, the stepper motor 2, the mechanical shaft, the steering gear, the gear 1, the gear 2, the gear 3, the gear 4, the fixed shaft, the straight cylinder, the rotating shaft and the mechanical claw. The stepper motor 2 can drive the transmission shaft and the gear 1 to rotate simultaneously, and the chain can be used to make the gear 2 rotate as the gear 1 rotates, and the gear 2 drives the transmission shaft, the straight cylinder and the gear 3 to rotate simultaneously, and the gear 3 drives the gear 4 to rotate, and the gear 4 drives the mechanical claw to rotate through the rotating shaft. Thus, the material can be grabbed and the lead screw can be driven to rotate by a stepper motor, so that the guide slider moves up and down on the slide, thereby realizing the vertical rise and fall of the material after being grabbed. The servo can be used to control the mechanical shaft to rotate slightly to realize the rotation of the grabbed material, and finally the material is grabbed from the to-be-grabbed position to the specified position, thereby realizing the automation of material grabbing, improving the material grabbing efficiency, and reducing the grabbing error rate. The entire grabbing process does not require human intervention, reducing manual handling while realizing accurate and fast material handling. These improvements improve the applicability of the overall system in different application scenarios, including but not limited to automated production lines in the manufacturing industry, logistics sorting centers, etc.

[0052] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0054] Figure 1 It is a structural schematic diagram of the present invention;

[0055] Figure 2 To improve the network structure diagram of YOLOv5;

[0056] Figure 3 It is a schematic diagram of the C2f structure;

[0057] Figure 4 This is the schematic diagram of the BRA attention mechanism;

[0058] Figure 5 This is the structural diagram of the decoupling head;

[0059] Figure 6 It is a schematic diagram of the connection between the mechanical grasping device and the guide rail slider of the present invention;

[0060] Figure 7 It is a schematic diagram of the mechanism of the end effector of the present invention;

[0061] Among them, the labels of the components in the accompanying drawings are as follows: 1. base plate, 2. stepper motor 1, 3. slide, 4. lead screw, 5. guide rail slider, 6. mechanical axis, 7. camera bracket, 8. depth camera, 9. transmission shaft, 10. servo, 11. stepper motor 2, 12. gear 1, 13. gear 2, 14. chain, 15. fixed axis, 16. gear 3, 17. gear 4, 18. rotating axis, 19. straight cylinder, 20. mechanical claw. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] Example 1

[0064] like Figure 1 , Attachment Figure 6 , Attachment Figure 7

[0065] The intelligent material grasping system based on improved YOLOv5 described in this embodiment includes a base plate 1, a slide 3, a screw 4, a guide slider 5, a camera bracket 7, a depth camera 8, a stepper motor 1 and a mechanical grasping device; the base plate 1 is installed with a slide 3, the slide 3 is installed with a guide slider 5, the stepper motor 1 is installed on the base plate 1, the screw 4 is fixed on the inner side of the slide 5, and is installed between the slide 3 and the guide slider 5, the guide slider 5 is slidably connected with the screw 4 through a through groove provided, the fixed output end of the stepper motor 2 is fixedly connected to one end of the screw 3, and the screw 3 is connected through a stepper motor 1. The stepper motor 2 drives the rotation, the camera bracket 7 is installed at the top of the two slides 3, and the depth camera 8 is installed at the center of the camera bracket 7 for photographing and obtaining material information and recording it. The mechanical grasping device is connected to one side of the guide rail slider 5 for grasping the material to the correct position; the slide 3, the screw rod 4, the guide rail slider 5 and the stepper motor 2 are arranged in two groups, which are symmetrically installed on both sides of the base plate 1; photographing and obtaining material position information is photographed and obtained by the depth camera 8, and the collected material information is calculated by using the YOLOv5 target detection algorithm to obtain the position and relative coordinates of the material.

[0066] The mechanical grasping device includes gear 1 12, gear 2 13, chain 14, stepper motor 2 11, steering gear 10, mechanical shaft 6, transmission shaft 9 and end effector, the mechanical shaft 6 is rotatably connected to the side opposite to the guide rail slider 5 with the through groove, the transmission shaft 9 is fixedly connected to the circular through hole at one end of the mechanical shaft 6, the gear 12 is fixedly connected to the center position of the transmission shaft 9, the main body of the steering gear 10 is fixed in the groove at one end of the side opposite to the guide rail slider 5 with the through groove, the output rocker of the steering gear 10 is fixed on the side of the mechanical shaft 6 rotatably connected to the guide rail slider 5, the stepper motor 2 11 is embedded in the side opposite to the guide rail slider 5 with the through groove, The end of the transmission shaft 9 is rotationally connected to the fixed output end of the stepper motor 11, the gear 12 is meshed with teeth on the top, and the chain 14 is meshed with the gear 12 and the gear 2 13 respectively through its tooth grooves; the transmission shaft 9 is arranged in two groups, which are respectively installed on the front and rear sides of the mechanical grasping device, and the ends on both sides of the transmission shaft 9 installed on the rear side of the mechanical grasping device are respectively fixed to the guide rail slider 5, and the two ends of the transmission shaft 9 installed on the front side of the mechanical grasping device are slidingly connected to the through holes of the two groups of symmetrically installed mechanical shafts 6; the servo 10, stepper motor 2 11 and mechanical shaft 6 are arranged in two groups, which are symmetrically installed on both sides of the transmission shaft 9 on the rear side of the mechanical grasping device.

[0067] The end effector comprises a fixed shaft 15, a gear three 16, a gear four 17, a rotating shaft 18, a straight cylinder 19 and a mechanical claw 20, one end of the fixed shaft 15 is fixedly mounted on the boss surface outside the mechanical shaft 6, and a gear four 17 is fixedly mounted on the boss above the other end of the fixed shaft 15, one side of the straight cylinder 19 is fixedly connected to the end of the transmission shaft 9 installed on the front side of the mechanical grasping device, the gear three 16 is fixedly connected to one side of the straight cylinder 19 through a center hole, the gear four 17 and the gear three 16 are meshed with each other, so that the rotation of the gear three 16 drives the gear four 17 to rotate, one end of the rotating shaft 18 is fixedly mounted on the center hole of the gear four 17, and the other end is fixedly connected to the end of the mechanical claw 20; the end effector is arranged in two groups, which are symmetrically mounted with the center of the transmission shaft 9 as the symmetry axis, and the end effector is fixed to the mechanical shaft 6 through the fixed shaft 15 and the straight cylinder 19, and cooperates with each other to realize the grasping of the material.

[0068] The present invention has a simple structure and is easy to use. The location and category information of the material can be obtained through the shooting function of the depth camera 8 and the improved YOLOv5 target detection algorithm. The material can be grabbed through the cooperation of the bottom plate 1, the stepper motor 1 2, the slide 3, the screw rod 4, the guide rail slider 5, the mechanical axis 6, the stepper motor 2 11, the steering gear 10, the gear 1 12, the gear 2 13, the gear 3 16, the gear 4 17, the fixed axis 15, the straight cylinder 19, the rotating axis 18 and the mechanical claw 20. The stepper motor 2 11 can drive the transmission shaft 9 and the gear 1 12 to rotate at the same time, and the chain 14 can make the gear 2 13 rotate as the gear 1 12 rotates, and the gear 2 13 drives the transmission shaft 9, the straight cylinder 19 and the gear 3 16 to rotate at the same time. The gear three 16 drives the gear four 17 to rotate, and the gear four 17 drives the mechanical claw 20 to rotate through the rotating shaft 9, so as to grab the material. The screw rod 4 can be driven to rotate by the stepping motor 2, so that the guide slider 5 can move up and down on the slide 3, so as to realize the vertical rise and fall of the material after grabbing. The mechanical shaft 6 can be controlled by the steering gear 10 to rotate slightly to realize the rotation of the grabbed material, and finally the material is grabbed from the uncertain position to the specified position, so that the present invention can intelligently grab the material with uncertain position, realize the automation of material grabbing, improve the grabbing efficiency of the material, reduce the error rate of grabbing, and the whole grabbing process does not need manual operation, reduces the dependence on manual processing, and significantly improves the efficiency.

[0069] Example 2

[0070] The improved method of the YOLOv5 algorithm described in this embodiment is: Figure 2 The improved YOLOv5 algorithm network structure includes:

[0071] S1: Replace the C3 module in the backbone network with the C2f module to reconstruct the backbone network;

[0072] S2: Introducing the BRA mechanism in the Neck layer to improve feature extraction capabilities;

[0073] S3: Introducing a decoupled detection head at the Head layer to optimize detection performance;

[0074] S4: Use the EIoU loss function as the network loss function to improve detection accuracy.

[0075] In this embodiment, the step S1 specifically includes:

[0076] like Figure 3As shown in the figure, after the first convolution operation, the C2f module divides the feature map into two parts. One part is processed by n Bottleneck modules, and the other part is directly output. Finally, the results of the two parts are spliced ​​in the channel dimension, and the final feature map is generated after the second convolution operation. This branch design increases the nonlinear ability and representation ability of the network. By splicing the features of different branches in the channel dimension, feature fusion is achieved. The spliced ​​features will contain information from different branches, enriching the representation ability of the features.

[0077] The Bottleneck module contains a 1×1 convolution layer and a 3×3 convolution layer. The first 1×1 convolution layer is used for dimensionality reduction, and the second 3×3 convolution layer is used for calculation and output.

[0078] In this embodiment, step S2 specifically includes:

[0079] like Figure 4 As shown in the figure, the BRA attention mechanism is a two-layer routing attention mechanism. The BRA attention mechanism is introduced between the Conv module and the downsampling module of the Neck layer in the YOLOv5 neural network. The BRA attention mechanism will input an image X∈R H×W×C , divided into S×S non-overlapping regions, each containing eigenvectors, we can transform X into Obtained by linear mapping The specific expressions of Q, K, and V are as follows:

[0080] Q=X r W q

[0081] K=X r W k

[0082] V=X r W v

[0083] Among them, W q ,W k ,W v ∈R C×C The projection weights belong to query, key, and value respectively; a directed graph is constructed to find the area that each given area should participate in, that is, Calculate Q r and K r The adjacency matrix of the regional correlation is:

[0084] A r =Q r (K r )T

[0085] Then only the top k connections of each region are kept to prune the correlation graph,

[0086] I r =topkIndex(A r )

[0087] That is I r The i-th row of contains the indices of the top k most relevant regions of the i-th region.

[0088] Using the region-to-region index routing matrix I r For each query token in region i, it will look at all key-value pairs in the union of k routing regions, indexed by Aggregate the tensor of key and value:

[0089] K g =gather(K,I r ),V g =gather(V,I r )

[0090] Where K g and V g It is the tensor of the clustered key and value, and then the attention operation is applied to the clustered KV pairs:

[0091] O=Attention(Q,K g ,V g )+LCE(V)

[0092] The above function LCE(·) is parameterized using depthwise convolution, with the parameter value set to 5.

[0093] In this embodiment, step S3 specifically includes: Figure 5 As shown in the figure, the input of the decoupled detection head first uses a 1×1 convolution layer to reduce the number of channels to 256, and then enters two branches respectively. Each branch has two 3×3 convolution layers. The two branches are used for regression tasks and classification tasks respectively to improve the flexibility of the model; a multi-branch detection head is used for calculation, which improves the detection accuracy and accelerates the network convergence, thereby improving the generalization ability and robustness of the model.

[0094] In this embodiment, step S4 specifically includes:

[0095] The loss function is the EIoU loss function, and the formula is as follows:

[0096]

[0097] Where L IoU is the intersection-over-union ratio of the predicted box and the actual box, B is the predicted box, and B gt is the real box, b is the center point of B, b gt For B gt , ρ(·) is the Euler distance between the predicted box and the center of the real box, and c is the distance between B and B gt The diagonal length of the minimum bounding box between the two, w and h are the width and height of the prediction box B, w gt and h gt For B gt The width and height of C w For B and B gt Minimum bounding box width, C h For B and B gt Minimum bounding box height.

[0098] The EIoU loss function splits the aspect ratio loss into the difference between the predicted width and height and the minimum bounding box, which includes overlap loss, center distance loss, and width and height loss. Compared with the CIoU loss function, it converges faster and has higher detection accuracy.

[0099] The backbone network of YOLOv5 consists of a structure that encapsulates three modules: the Conv module, the C3 module, and the SPPF module. The C3 modules in the backbone network are replaced by C2f modules to perform feature conversion on the input data, reducing storage overhead and computational complexity.

[0100] The BRA mechanism refers to a two-layer routing attention mechanism, which is introduced between the Conv module and the downsampling module in the Neck layer of the YOLOv5 algorithm, and integrated with the FPN and PAN structures to improve the feature extraction capability of the model.

[0101] The Head layer uses a decoupled detection head to replace the original coupled head of YOLOv5. The decoupled detection head separates the classification task from the regression task. One branch is used for the regression task to predict the location information of the bounding box, and the other branch is used for the classification task to predict the category probability of the target, thereby improving the detection effect of the network.

[0102] The network's loss function consists of three parts: object confidence loss, object classification loss, and predicted box position loss. The loss function calculates the difference between the predicted target box and the actual target box. The loss function uses the EIoU loss function, which speeds up the convergence of the model and improves the detection accuracy of the model.

[0103] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. An intelligent material grabbing system based on improved YOLOv5, characterized in that: include: A base plate (1), a slide (3), a screw rod (4), a guide rail slider (5), a camera bracket (7), a depth camera (8), a stepper motor (2) and a mechanical grasping device; the base plate (1) is mounted with a slide (3), the slide (3) is mounted with a guide rail slider (5), the stepper motor (2) is mounted on the base plate (1), the screw rod (4) is fixed on the inner side of the slide (3), and is mounted between the slide (3) and the guide rail slide (3), and the guide rail slider (5) is connected to the slide (3). The stepper motor (2) is slidably connected to the screw rod (4) through a through groove, the fixed output end of the stepper motor (2) is fixedly connected to one end of the screw rod (4), the screw rod (4) is driven to rotate by the stepper motor (2), the camera bracket (7) is installed at the top of the two slides (3), the depth camera (8) is installed at the center of the camera bracket (7) for photographing and obtaining material information and recording it, and the mechanical grasping device is connected to one side of the guide rail slider (5) for grasping the material to a specified position.

2. The intelligent material grasping system based on improved YOLOv5 as claimed in claim 1, characterized in that: The mechanical grasping device comprises a gear 1 (12), a gear 2 (13), a chain (14), a steering gear (10), a stepper motor 2 (11), a mechanical shaft (6), a transmission shaft (9) and an end effector, wherein the mechanical shaft (6) is rotatably connected to the side opposite to the guide rail slider (5) on which the through groove is provided, the transmission shaft (9) is fixedly connected to a circular through hole at one end of the mechanical shaft (6), the gear 1 (12) is fixedly connected to the center position of the transmission shaft (9), the steering gear (10) body is fixed in a groove at one end of the side opposite to the guide rail slider (5) on which the through groove is provided, and the transmission shaft (9) of the steering gear (10) is fixedly connected to the central ...ly connected to the central position of the transmission shaft (9), the transmission shaft (9) of the steering gear (10) The rocker is fixed on the side of the mechanical shaft (6) that is rotatably connected to the guide rail slider (5); the stepper motor 2 (11) is embedded in the opposite side of the guide rail slider (5) where a through groove is provided; the end of the transmission shaft (9) is rotatably connected to the fixed output end of the stepper motor 2 (11); the gear 1 (12) is meshed with teeth on the top; the chain (14) is meshed with the gear 1 (12) and the gear 2 (13) respectively through its tooth grooves; the steering gear (10), the stepper motor 2 (11) and the mechanical shaft (6) are arranged in two groups, which are symmetrically installed on both sides of the transmission shaft (9) at the rear side of the mechanical grasping device.

3. The intelligent material grasping system based on improved YOLOv5 as claimed in claim 2, characterized in that: The transmission shaft (9) is arranged in two groups, which are respectively mounted on the front side and the rear side of the mechanical grasping device; the ends of both sides of the transmission shaft (9) mounted on the rear side of the mechanical grasping device are respectively fixed to the guide rail slider (5); and the two ends of the transmission shaft (9) mounted on the front side of the mechanical grasping device are slidably connected to the through holes of the two groups of symmetrically mounted mechanical shafts (6).

4. The intelligent material grasping system based on improved YOLOv5 as claimed in claim 2, characterized in that: The end effector comprises a fixed shaft (15), a gear three (16), a gear four (17), a rotating shaft (18), a straight cylinder (19) and a mechanical claw (20), wherein one end of the fixed shaft (15) is fixedly mounted on the boss surface outside the mechanical shaft (6), and the gear four (17) is fixedly mounted on the boss above the other end of the fixed shaft (15), one side of the straight cylinder (19) is fixedly connected to the end of the transmission shaft (9) installed on the front side of the mechanical grasping device, and the gear three (16) is connected to the straight cylinder (19) through a center through hole. 19) is fixedly connected on one side, the teeth of gear four (17) mesh with the teeth of gear three (16) so that gear three (16) rotates to drive gear four (17) to rotate, one end of the rotating shaft (18) is fixedly installed at the center hole of gear four (17), and the other end is fixedly connected to the end of the mechanical claw (20); the end effector is arranged in two groups, symmetrically installed with the center of the transmission shaft (9) as the symmetry axis, and the end effector is fixed to the mechanical shaft (6) through a fixed shaft (15) and a straight cylinder (19).

5. The intelligent material grasping system based on improved YOLOv5 as claimed in claim 1, characterized in that: The stepper motor 1 (2), the lead screw (4), the slide table (3), and the guide rail slider (5) are provided in two groups and are symmetrically mounted on both sides of the base plate (1).

6. The method for improving YOLOv5 in the system according to any one of claims 1 to 5, characterized in that: include: S1: Replace the C3 module in the backbone network with the C2f module to reconstruct the backbone network; S2: Introducing the BRA mechanism in the Neck layer to improve feature extraction capabilities; S3: Introducing a decoupled detection head at the Head layer to optimize detection performance; S4: Use the EIoU loss function as the network loss function to improve detection accuracy.

7. The improved method according to claim 6, characterized in that: The step S1 specifically includes: After the first convolution operation, the C2f module divides the feature map into two parts. One part is processed by n Bottleneck modules, and the other part is directly output. Finally, the results of the two parts are spliced ​​in the channel dimension and the final feature map is generated after the second convolution operation. This branch design increases the nonlinear ability and representation ability of the network. By splicing the features of different branches in the channel dimension, feature fusion is achieved, so that the spliced ​​features contain information from different branches. The Bottleneck module contains a 1×1 convolution layer and a 3×3 convolution layer. The first 1×1 convolution layer is used for dimensionality reduction, and the second 3×3 convolution layer is used for calculation and output.

8. The improved method according to claim 6, characterized in that: The step S2 specifically includes: The BRA attention mechanism is a two-layer routing attention mechanism. The BRA attention mechanism is introduced between the Conv module and the downsampling module of the Neck layer in the YOLOv5 neural network. The BRA attention mechanism will input an image X∈R H×W×C , divided into S×S non-overlapping regions, each containing eigenvectors, we can transform X into Through linear mapping, we can obtain Q, K, The specific expressions of Q, K, and V are as follows: Q=X r W q K=X r W k V=X r W v Among them, W q ,W k ,W v ∈R C×C The projection weights belonging to query, key, and value respectively; By constructing a directed graph to find the region that each given region should participate in, i.e., Q r , Calculate Q r and K r The adjacency matrix of the regional correlation is: A r =Q r (K r ) T The correlation graph is then pruned by retaining only the top k connections in each region, I r =topkIndex(A r ) That is I r The i-th row of contains the indices of the top k most relevant regions of the i-th region; Using the region-to-region index routing matrix I r For each query token in region i, it will look at all key-value pairs in the union of k routing regions, indexed by Aggregate the tensor of key and value: K g =gather(K,I r ),V g =gather(V,I r ) Where K g and V g It is the tensor of the clustered key and value, and then the attention operation is applied to the clustered KV pairs: O=Attention(Q,K g ,V g )+LCE(V) The above function LCE(·) is parameterized using depthwise convolution, with the parameter value set to 5.

9. The improved method according to claim 6, characterized in that: The step S3 specifically includes: the input of the decoupling detection head first uses a 1×1 convolution layer to reduce the number of channels to 256, and then enters two branches respectively, each branch has two 3×3 convolution layers, and the two branches are used for regression tasks and classification tasks respectively to improve the flexibility of the model; a multi-branch detection head is used for calculation to improve detection accuracy and accelerate network convergence, thereby enhancing the generalization ability and robustness of the model.

10. The improved method according to claim 6, characterized in that: The step S4 specifically includes: The loss function is the EIoU loss function, and the formula is as follows: Where L IoU is the intersection-over-union ratio of the predicted box and the actual box, B is the predicted box, and B gt is the real box, b is the center point of B, b gt For B gt , ρ(·) is the Euler distance between the predicted box and the center of the real box, and c is the distance between B and B gt The diagonal length of the minimum bounding box between the two, w and h are the width and height of the prediction box B, w gt and h gt For B gt The width and height of C w For B and B gt Minimum bounding box width, C h For B and B gt Minimum bounding box height.

Images