Intelligent positioning and grabbing method for crosshead part assembly

Abstract

The application discloses an intelligent positioning and grabbing method for crosshead part assembly, which comprises the following steps: collecting images of the postures of a plurality of crosshead parts to generate training data; constructing a YOLO-Corner algorithm model for detecting the top feature points of the crosshead parts and correcting the parameters of the model; identifying the top feature points of the crosshead parts in the images and positioning the top of the crosshead parts in the images by using the feature points; calculating the pose matrix of the crosshead parts in the base coordinate system of a mechanical arm; planning the end motion of the mechanical arm according to the pose coordinate information in the pose matrix, driving the mechanical arm to grab the crosshead parts and assembling the crosshead parts. Finally, the YOLO-Corner algorithm can accurately detect the feature points of the crosshead parts and accurately calculate the spatial pose of the crosshead parts relative to the camera, so that the mechanical arm can accurately grab the crosshead parts and complete the assembly task of the crosshead parts.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent assembly of parts, in particular to an intelligent positioning and grabbing method for assembly of a crosshead part. BACKGROUND

[0002] The production and assembly of natural gas compressors currently rely on manual assembly. The crosshead, as one of the parts for manual assembly, has the characteristics of large volume and mass, and narrow assembly space. Its shape and assembly position are shown in Figure 2 When workers assemble the crosshead, they need to hold the crosshead in a certain posture and assemble it. During the assembly adjustment process, the hands will be interfered, the assembly efficiency is low, and there are safety hazards such as falling and bruising. Therefore, the assembly by using the visual mechanical arm intelligent grabbing method can solve the above problems.

[0003] Through the analysis of the cross shape and the assembly position, it is found that the space placement position of the crosshead can be calculated by detecting the top features of the crosshead through visual recognition. It is most beneficial to the intelligent assembly of the crosshead to clamp the crosshead from above. Since the traditional image processing method is greatly affected by the change of image gray value, it is mainly aimed at specific shape detection with smooth surface and less texture features. However, in actual industrial production, the crosshead is a special-shaped part, the top surface shape is an unconventional shape, and the top surface adopts a rough machining process with irregular texture features such as concave-convex, protrusions, oil stains, paint, rust, etc. It is difficult to detect the top features of the crosshead through the traditional image processing method, so as to calculate the space placement position of the crosshead. Secondly, although the mechanical arm can replace manual assembly to improve efficiency, but in the movement process of the mechanical arm, the high joint movement speed will increase the vibration impact of the joint, thereby reducing the service life of the mechanical arm. SUMMARY

[0004] In view of the above shortcomings of the prior art, the present application provides an intelligent positioning and grabbing method for assembly of a crosshead part, to solve the problem that the recognition accuracy, motion compliance and high efficiency cannot be considered in the intelligent assembly process of the crosshead.

[0005] To achieve the above application purposes, the technical solution adopted by the present application is:

[0006] An intelligent positioning and grabbing method for assembly of a crosshead part is provided, which includes the following steps:

[0007] S1: Collect images of a plurality of crosshead part placement postures, and label the top feature points on the images to obtain the feature points of the top of the crosshead part, and generate training data;

[0008] S2: A YOLO-Corner algorithm model for detecting the top feature points of the crosshead part is constructed, the YOLO-Corner algorithm model comprises a feature extraction layer, a feature fusion layer and an output layer, the YOLO-Corner algorithm model is trained by using training data, the feature points labeled on the image are taken as the output, the parameters of the model are corrected, and the trained YOLO-Corner algorithm model is output;

[0009] S3: An image in which the crosshead part is placed is collected, and is input into the trained YOLO-Corner algorithm model, the feature points of the top of the crosshead part in the image are recognized, and the feature points are used to position the top of the crosshead part in the image;

[0010] S4: The pixel coordinates of the feature points are obtained, a crosshead coordinate system is established on the image, 3D space coordinates of the feature points in the crosshead coordinate system are obtained, the spatial pose of the crosshead relative to the camera is calculated, the spatial pose of the crosshead relative to the camera is converted into a pose in the base coordinate system of the mechanical arm, and a pose matrix of the crosshead coordinate system in the base coordinate system of the mechanical arm is calculated;

[0011] S5: The pose coordinates of the grasping points on the crosshead part in the base coordinate system of the mechanical arm are obtained according to the pose matrix, the pose coordinates are used to plan the movement of the end of the mechanical arm, the mechanical arm is driven to grasp the crosshead part and assemble the crosshead part.

[0012] Further, the feature extraction layer comprises a C3k2 module and a receptive field attention convolution module RFAConv, a feature complementary mapping module FCM is introduced into the C3k2 module to form a C3k2_FCM module;

[0013] The feature fusion layer comprises an up-sampling module EUCB, a triple attention mechanism and a receptive field attention convolution module RFAConv; the triple attention mechanism comprises three branches, and the receptive field attention convolution module RFAConv generates independent attention weights for each receptive field by focusing on the spatial features in the texture features.

[0014] Further, the method for processing the image by the C3k2_FCM module is as follows:

[0015] A1: The input features of the image are divided into two parts, i.e., a channel and a channel. X input ( X input ∈ R C×H×W ) and αC ;

[0016] ;

[0017] ​ ;

[0018] in, α This refers to the channel splitting ratio. R C×H×W Image features, including the number of image channels. C Vertical latitude H Width of horizontal dimension W ; They are respectively αC aisle, Image features of the channel, This is the channel segmentation function;

[0019] A2: The segmented image features The data is passed to a standard 3x3 convolutional branch to extract channel feature information. Image features Passing the data to the pointwise convolution PWConv branch preserves shallow spatial location information. ;

[0020] ;

[0021] in, This is a mapping function between the spatial and semantic information of image features;

[0022] A3: Channel feature information Global average pooling is introduced through channel interaction to obtain global information for each channel. Then, a sigmoid activation function is used to assign channel information weights to the global information of each channel. ;

[0023] ;

[0024] ;

[0025] in, i Number the convolution kernel. k i For the first i One convolutional kernel, This is the mapping function for each feature layer channel in the channel interaction. For input number i Channel feature information of each convolutional kernel For the first i The output channel feature information of each convolutional kernel It is the sigmoid activation function. These are the pixel numbers for the vertical dimension and the pixel numbers for the horizontal dimension, respectively. The output function of the convolution kernel. For channel weight set;

[0026] A4: Shallow spatial location information Spatial information weights are assigned to important information at each location through spatial interaction. ;

[0027] , ;

[0028] in, For spatial clustering convolution mapping functions, For spatial weight set;

[0029] A5: Weighting spatial information Mapping to channel feature information In the middle, channel information weight Mapping to shallow spatial location information In the middle, channel feature information is realized. Information on the characteristics of Tao The complementary fusion yields feature maps of spatial and semantic relationships. ;

[0030] ;

[0031] The upsampling module EUCB processes images in the following way:

[0032] B1: Use an upsampling module with a scaling factor of 2 to upsample the feature map. The scale is magnified by 2 times, and then the magnified feature map is enhanced using 3×3 depthwise convolution (DWC), batch normalization (BN), and ReLU activation function. ;

[0033] B2: Then, use 1×1 convolution. C 1×1 To reduce feature maps The channel outputs enhanced feature maps. ;

[0034] ;

[0035] in, This is a 1×1 convolution operation. It is the ReLU activation function. For batch normalization function, This is a 3×3 depthwise convolution operation. For upsampling operation, This is the output function of the upsampling module EUCB.

[0036] Furthermore, the triple attention mechanism processes images as follows:

[0037] Enhance feature maps In the triple attention mechanism, three branches are used to enhance the feature map. Perform convolution operations to capture enhanced feature maps Based on the features corresponding to the feature points at the top of the crosshead part, we obtain the feature map corresponding to the feature points.

[0038] Furthermore, the method of processing images using the triple attention mechanism specifically includes the following steps:

[0039] C1: The input enhanced feature map Pass it to each of the three branches respectively;

[0040] C2: In the first branch, the input augmented feature map Rotate 90° counterclockwise along the vertical axis to obtain the rotation tensor. Rotation tensor The shape is W × H × C ;

[0041] The rotation tensor is obtained through the Z-Pool layer. Simplified to a shape of 2× H × C rotation tensor Then, it goes through a standard convolutional layer with a kernel size of k×k to reach a batch normalized (BN) layer;

[0042] Batch normalized BN layers calculate the rotation tensor upper feature data x mean ,variance Feature data x Standardize, and then scale and translate the standardized rotation tensor. The shape is simplified to 1× H × C Output Finally, the Sigmoid activation function is used to generate information about the output. Attention weights ;

[0043] ;

[0044] ;

[0045] ;

[0046] ;

[0047] ;

[0048] ;

[0049] in, The feature function is max pooling. The characteristic function is the average pooling function. u The identifier for the feature data. x u For the first u Each feature data, m The number of feature data, The mean of the feature data, The variance of the feature data, For correction factor, These are the coefficients and deviations of the output function, respectively. For the first u Output of each feature data;

[0050] Attention weight Applied to the rotation tensor Then rotate it 90° clockwise along the vertical axis to obtain the enhanced feature map. Output feature map of the original shape ;

[0051] C3: In the second branch, the feature map will be enhanced. Rotate 90° counterclockwise around the horizontal axis to obtain the rotation tensor. Rotation tensor The shape is H × C × W The rotation tensor will be passed through the Z-Pool layer. Simplified to shape 2× C × W rotation tensor ;

[0052] Rotation Tensor The process proceeds through a standard convolutional layer with a kernel size of k×k to a batch normalized (BN) layer, resulting in a simplified output shape of 1×k. C × W Output Finally, the Sigmoid activation function is used to generate information about the output. Attention weights ;

[0053] Attention weight Applied to the rotation tensor Then rotate it 90° clockwise along the horizontal axis to obtain the preserved and enhanced feature map. Output feature map of the original shape ;

[0054] C4: In the third branch, enhance the feature map. Simplify directly to a shape of 2× using the Z-Pool layer. H × W tensor It passes through a standard convolutional layer with a kernel size of k×k to reach a batch normalized (BN) layer, with an output shape of 1× H × W Output Finally, the Sigmoid activation function is used to generate information about the output. Attention weights Pay attention weights Apply to tensor The output feature map is obtained from the above. ;

[0055] C5: Average aggregation of the outputs from the three branches to calculate the enhanced feature map. Corresponding output features We obtain the feature map corresponding to the feature points at the top of the crosshead part;

[0056] .

[0057] Furthermore, the receptive field attention convolutional module RFAConv processes images in the following way:

[0058] The feature map corresponding to the feature point is input into the receptive field attention convolution module RFAConv. Through the spatial features of the receptive field, the attention weight of each receptive field is generated, and through the weighting operation, the recognized feature point is output. The feature point is used to locate the top of the crosshead part in the image.

[0059] Furthermore, the method by which the receptive field attention convolutional module RFAConv processes images specifically includes the following steps:

[0060] D1: The feature maps corresponding to the feature points are converted into receptive field spatial feature maps through group convolution (Group-Conv). During the conversion, each receptive field corresponds to an independent sliding window. The size of the receptive field spatial feature map is then enlarged to the size of the feature map corresponding to the feature points. v Multiply by 10 to generate a magnified feature map;

[0061] D2: Perform global average pooling on the feature information of each receptive field in the magnified feature map, compressing the feature information of each receptive field into a single value; then, use 1x1 grouped convolution to perform information interaction on the average pooled feature information; finally, perform classification operation on the feature information after 1x1 grouped convolution to generate the attention weight of each receptive field and output the receptive field attention map.

[0062] D3: The receptive field attention map and the receptive field spatial feature map are weighted and re-weighted, and the identified feature points are output through the convolutional layer. The feature points are used to locate the top of the crosshead part in the image.

[0063] ;

[0064] ;

[0065] ;

[0066] in, The classification function for the output layer. A global function for the feature information of the receptive field. X To enlarge the feature map, For 1x1 grouped convolution, for Grouped convolution, For normalization function, z i It is the first in the input vector i K is the length of the input vector; ReLU(.) is the corrected linear unit function;

[0067] For feature data at feature points, R F These are the feature points output.

[0068] Further, step S4 includes:

[0069] S41: Based on the identified feature points in the image, obtain the pixel coordinates of the feature points, calibrate the camera's intrinsic parameters, establish a crosshair coordinate system on the image, take the 0-degree posture as the reference, set the origin of the crosshair coordinate system at the center of the surface where the feature points are located, and use the origin of the crosshair coordinate system as the grab point.

[0070] S42: Based on the dimensions of the crosshead, obtain the 3D spatial coordinates of the feature points in the crosshead coordinate system. Assign the 3D spatial coordinates of the feature points to the pixel coordinates in the correct order, and calculate the spatial pose matrix of the crosshead component relative to the camera using the camera's intrinsic parameters. From the pose matrix Extract the pose information of the crosshead component relative to the camera;

[0071] S43: Fix the checkerboard calibration plate to the end of the robotic arm that grips the crosshead part. Fix the camera at a set distance directly above the checkerboard calibration plate via a bracket, so that the camera is parallel to the horizontal plane, and take M images of the calibration plate in different postures.

[0072] S44: Based on the pose transformation matrix of the robotic arm end effector relative to the robotic arm base coordinate system. The position matrix of the checkerboard calibration plate at the end of the robotic arm and the pose transformation matrix of the camera relative to the checkerboard calibration board. Construct the pose relationship matrix corresponding to each calibration board image;

[0073] ;

[0074] in, Let be the pose transformation matrix of the camera relative to the base coordinate system of the robotic arm;

[0075] S45: For two different calibration board images, construct the pose transformation matrix using the pose relationship matrix. The equation to be solved;

[0076] ;

[0077] in, Let be the pose transformation matrix of the camera relative to the checkerboard calibration board corresponding to two different calibration board images. This represents the pose transformation matrix of the robotic arm end effector relative to the robotic arm base coordinate system for two different calibration plate images.

[0078] S46: Construct a system of equations to be solved corresponding to M calibration plate images;

[0079] ;

[0080] in, Let be the pose transformation matrix of the camera relative to the checkerboard calibration board corresponding to two adjacent different calibration board images. This is the pose transformation matrix of the robotic arm end effector relative to the robotic arm base coordinate system corresponding to two adjacent different calibration plate images;

[0081] S47: Solving the pose transformation matrix using the system of equations to be solved Calculate the pose matrix of the crosshead coordinate system in the robot arm base coordinate system. ;

[0082] .

[0083] The beneficial effects of this invention are as follows:

[0084] 1. Compared with traditional image processing techniques for identifying feature points of crosshead parts, the YOLO-Corner algorithm proposed in this invention is based on deep learning and can avoid the influence of messy texture features and uneven lighting on grayscale value changes caused by crosshead.

[0085] 2. By introducing an attention mechanism module into the neck network to integrate information from different dimensions, the inherent features in the messy texture can be better captured, enhancing the network's ability to understand and process the feature map of the crosshead part, and making the algorithm more accurate in locating feature points.

[0086] 3. This invention combines spatial attention mechanism with convolution operation, thereby enabling the convolutional neural network to more effectively understand and process local regions in the image, and improving the accuracy of the algorithm in extracting feature points on the crosshead part.

[0087] 4. This invention enhances the information interaction between different levels and stages of the network by introducing EUCB upsampling operation into the neck network, increases the algorithm's ability to understand the contextual information of the feature map of the crosshead part, and efficiently enhances image features without significantly increasing the model's computational parameters, making it easier for the algorithm to capture feature points.

[0088] 5. By introducing the FCM module to improve the backbone network, the spatial location information of the crosshead feature points is more deeply integrated into the network, making it better aligned with the deep semantic information, thereby improving the algorithm's ability to locate feature points.

[0089] 6. This invention collects a large number of calibration images of different postures to simultaneously calculate the pose coordinates of multiple crosshead parts in the robot arm's base coordinate system, which is used for subsequent robot arm grasping path planning, thus improving calculation efficiency and positioning accuracy.

[0090] This invention ultimately enables the YOLO-Corner algorithm to accurately detect the feature points of the crosshead and accurately calculate the spatial pose of the crosshead relative to the camera, so that the robotic arm can accurately grasp the crosshead part and complete the crosshead assembly task. Attached Figure Description

[0091] Figure 1 A flowchart of an intelligent positioning and gripping method for assembling crosshead parts.

[0092] Figure 2 This is a schematic diagram showing the assembly posture of the crosshead part.

[0093] Figure 3 This is a schematic diagram of the feature points of the crosshead part.

[0094] Figure 4 A schematic diagram showing the sequence of feature point annotations for different orientations of the crosshead component.

[0095] Figure 5 Feature point diagram of the identified crosshead part.

[0096] Figure 6This is a diagram showing the pose information of the detected crosshead component.

[0097] Figure 7 This is a schematic diagram of the joint coordinate system of the robotic arm.

[0098] Figure 8 This is a schematic diagram of the intelligent assembly of the crosshead parts. Detailed Implementation

[0099] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0100] like Figure 1 As shown, an intelligent positioning and gripping method for assembling crosshead parts includes the following steps:

[0101] S1: Collect images of the placement postures of several crosshead parts, and label the top feature points on the images to obtain the feature points at the top of the crosshead parts and generate training data.

[0102] In this embodiment, since there is currently no publicly available dataset for the top feature points of the natural gas compressor crosshead, this embodiment establishes a dataset of different perspectives with different top texture features composed of 30 crosshead parts, collecting a total of 1561 samples. Insufficient image sample diversity can lead to overfitting during network model training. To prevent overfitting, this invention uses a hybrid data augmentation method, including random occlusion, different backgrounds, non-uniform scaling, mirroring, perspective transformation, and color temperature transformation, to increase the number of samples from 1561 to 3122. The Labelme annotation tool is then used to annotate the top features (grasping direction plane) with feature points and detection boxes, obtaining the image dataset of the top feature points of the natural gas compressor crosshead. The top feature points of the crosshead parts are shown below. Figure 3 As shown.

[0103] The order of feature point annotations for the crosshead part under different orientations is as follows: Figure 4 As shown, the definition Figure 4 The crosshead part is in the standard posture (positioning angle is 0 degrees). The upper left corner feature point in this posture is defined as point 1 and marked in a clockwise direction. When the crosshead posture is close to the standard posture (the offset angle is greater than 135 degrees or less than 45 degrees), the upper left corner is also defined as point 1 and marked in a clockwise direction.

[0104] S2: Construct a YOLO-Corner algorithm model for detecting top feature points of crosshead parts. The YOLO-Corner algorithm model includes a feature extraction layer, a feature fusion layer, and an output layer. Train the YOLO-Corner algorithm model using training data, take the marked feature points on the image as the output, correct the model parameters, and output the trained YOLO-Corner algorithm model.

[0105] The feature extraction layer includes the C3k2 module and the receptive field attention convolution module RFAConv. The C3k2 module introduces the feature complement mapping module FCM to form the C3k2_FCM module.

[0106] The feature fusion layer includes an upsampling module EUCB, a triple attention mechanism, and a receptive field attention convolution module RFAConv. The triple attention mechanism has three branches. The receptive field attention convolution module RFAConv generates independent attention weights for each receptive field by focusing on spatial features in the texture features.

[0107] S3: Collect an image of the crosshead parts, input it into the trained YOLO-Corner algorithm model, identify the feature points at the top of the crosshead parts in the image, and use the feature points to locate the top of the crosshead parts in the image.

[0108] This embodiment introduces a Feature Complementary Mapping (FCM) module to improve the C3k2 module in the backbone network, and names it C3k2_FCM. The role of the FCM module is to integrate the spatial location information of the target more deeply into the network, making it better aligned with deep semantic information, thereby improving the localization capability of crosshead feature points.

[0109] The C3k2_FCM module processes images in the following way:

[0110] The C3k2_FCM module is used to perform channel segmentation on image features, extracting shallow spatial location information and channel feature information. A sigmoid activation function is then used to assign weights to channel and spatial information, followed by complementary fusion to obtain feature maps representing spatial and semantic relationships. .

[0111] Specifically, the following steps are included:

[0112] A1: Input features of the image X input ( X input ∈ R C×H×W The channel is divided into: αC Channels and The passage is divided into two parts;

[0113] ;

[0114] ;

[0115] in, α This refers to the channel splitting ratio. R C×H×W Image features, including the number of image channels. C Vertical latitude H Width of horizontal dimension W ; They are respectively αC aisle, Image features of the channel, This is the channel segmentation function;

[0116] A2: The segmented image features The data is passed to a standard 3x3 convolutional branch to extract channel feature information. Image features Passing the data to the pointwise convolution PWConv branch preserves shallow spatial location information. ;

[0117] ;

[0118] in, This is a mapping function between the spatial and semantic information of image features;

[0119] A3: Channel feature information Global average pooling is introduced through channel interaction to obtain global information for each channel. Then, a sigmoid activation function is used to assign channel information weights to the global information of each channel. ;

[0120] ;

[0121] ;

[0122] in, i Number the convolution kernel. k i For the first i One convolutional kernel, This is the mapping function for each feature layer channel in the channel interaction. For input number i Channel feature information of each convolutional kernel For the first i The output channel feature information of each convolutional kernel It is the sigmoid activation function. These are the pixel numbers for the vertical dimension and the pixel numbers for the horizontal dimension, respectively. The output function of the convolution kernel. For channel weight set;

[0123] A4: Shallow spatial location information Spatial information weights are assigned to important information at each location through spatial interaction. ;

[0124] , ;

[0125] in, For spatial clustering convolution mapping functions, For spatial weight set;

[0126] A5: Weighting spatial information Mapping to channel feature information In the middle, channel information weight Mapping to shallow spatial location information In the middle, channel feature information is realized. Information on the characteristics of Tao The complementary fusion yields feature maps of spatial and semantic relationships. ;

[0127] .

[0128] The upsampling module EUCB processes images in the following way:

[0129] The feature map is processed using the upsampling module EUCB. The feature map is then enlarged and enhanced, and then the number of channels in the feature map is reduced to output an enhanced feature map. .

[0130] Specifically, the following steps are included:

[0131] B1: Use an upsampling module with a scaling factor of 2 to upsample the feature map. The scale is magnified by 2 times, and then the magnified feature map is enhanced using 3×3 depthwise convolution (DWC), batch normalization (BN), and ReLU activation function. ;

[0132] B2: Then, use 1×1 convolution. C 1×1 To reduce feature maps The channel outputs enhanced feature maps. ;

[0133] ;

[0134] in, This is a 1×1 convolution operation. It is the ReLU activation function. For batch normalization function, This is a 3×3 depthwise convolution operation. For upsampling operation, This is the output function of the upsampling module EUCB.

[0135] The triple attention mechanism for image processing is as follows:

[0136] Enhance feature maps In the triple attention mechanism, three branches are used to enhance the feature map. Perform convolution operations to capture enhanced feature maps The feature map corresponding to the feature points at the top of the crosshead part is obtained by examining the features of those feature points. Specifically, the following steps are included:

[0137] C1: The input enhanced feature map Pass it to each of the three branches respectively;

[0138] C2: In the first branch, the input augmented feature map Rotate 90° counterclockwise along the vertical axis to obtain the rotation tensor. Rotation tensor The shape is W × H × C ;

[0139] The rotation tensor is obtained through the Z-Pool layer. Simplified to a shape of 2× H × C rotation tensor Then, it goes through a standard convolutional layer with a kernel size of k×k to reach a batch normalized (BN) layer;

[0140] Batch normalized BN layers calculate the rotation tensor upper feature data x mean ,variance Feature data x Standardize, and then scale and translate the standardized rotation tensor. The shape is simplified to 1× H × C Output ,

[0141] Finally, the Sigmoid activation function is used to generate information about the output. Attention weights ;

[0142] ;

[0143] ;

[0144] ;

[0145] ;

[0146] ;

[0147] ;

[0148] in, The feature function is max pooling. The characteristic function is the average pooling function. u The identifier for the feature data. x u For the first u Each feature data, m The number of feature data, The mean of the feature data, The variance of the feature data, For correction factor, These are the coefficients and deviations of the output function, respectively. For the first u Output of each feature data;

[0149] Attention weight Applied to the rotation tensor Then rotate it 90° clockwise along the vertical axis to obtain the enhanced feature map. Output feature map of the original shape ;

[0150] C3: In the second branch, the feature map will be enhanced. Rotate 90° counterclockwise around the horizontal axis to obtain the rotation tensor. Rotation tensor The shape is H × C × W The rotation tensor will be passed through the Z-Pool layer. Simplified to shape 2× C × W rotation tensor ;

[0151] Rotation Tensor The process proceeds through a standard convolutional layer with a kernel size of k×k to a batch normalized (BN) layer, resulting in a simplified output shape of 1×k. C × W Output Finally, the Sigmoid activation function is used to generate information about the output. Attention weights ;

[0152] Attention weight Applied to the rotation tensor Then rotate it 90° clockwise along the horizontal axis. This yields a feature map that preserves and enhances the original features. Output feature map of the original shape ;

[0153] C4: In the third branch, enhance the feature map. Simplify directly to a shape of 2× using the Z-Pool layer. H × W tensor It passes through a standard convolutional layer with a kernel size of k×k to reach a batch normalized (BN) layer, with an output shape of 1× H × W Output Finally, the Sigmoid activation function is used to generate information about the output. Attention weights Pay attention weights Apply to tensor The output feature map is obtained from the above. ;

[0154] C5: Average aggregation of the outputs from the three branches to calculate the enhanced feature map. Corresponding output features We obtain the feature map corresponding to the feature points at the top of the crosshead part;

[0155] .

[0156] The triple attention mechanism module can integrate information from different dimensions through the above methods to better capture the inherent features in messy textures, thereby enhancing the network's ability to understand and process the feature maps of crosshead parts and making the algorithm more accurate in locating feature points.

[0157] In traditional convolution operations (Conv), each receptive field uses the same kernel parameters, failing to distinguish information differences between different locations, which limits the algorithm's performance. Furthermore, due to the limitations of spatial attention and existing spatial attention mechanisms, while they can highlight key features, they cannot effectively address the parameter sharing problem of large-size convolution kernels because they only focus on spatial features. Therefore, the receptive field attention convolution module RFAConv is introduced. By focusing on the spatial features of the receptive field, it generates independent attention weights for each receptive field, thereby achieving convolution operations without parameter sharing.

[0158] The receptive field attention convolutional module RFAConv processes images in the following way:

[0159] The feature map corresponding to the feature point is input into the receptive field attention convolution module RFAConv. Through the spatial features of the receptive field, the attention weight of each receptive field is generated, and through the weighting operation, the recognized feature point is output. The feature point is used to locate the top of the crosshead part in the image.

[0160] Specifically, the following steps are included:

[0161] D1: The feature maps corresponding to the feature points are converted into receptive field spatial feature maps through group convolution (Group-Conv). During the conversion, each receptive field corresponds to an independent sliding window. The size of the receptive field spatial feature map is then enlarged to the size of the feature map corresponding to the feature points. v Multiply, generate magnified feature map X m ;

[0162] D2: Enlarge the feature map X m Global average pooling is performed on the feature information of each receptive field to compress the feature information of each receptive field into a single value; then, 1x1 grouped convolution is used to interact with the average pooled feature information; finally, classification operation is performed on the feature information after 1x1 grouped convolution to generate the attention weight of each receptive field and output the receptive field attention map.

[0163] D3: The receptive field attention map and the receptive field spatial feature map are weighted and re-weighted, and the identified feature points are output through the convolutional layer. The feature points are then used to locate the top of the crosshead part in the image.

[0164] ;

[0165] ;

[0166] ;

[0167] in, The classification function for the output layer. A global pooling function for the feature information of the receptive field. X m To enlarge the feature map, For 1x1 grouped convolution, for Grouped convolution, For normalization function, z i It is the first in the input vector iK is the length of the input vector; ReLU(.) is the corrected linear unit function; For feature data at feature points, R F These are the feature points output.

[0168] S4: Based on the identified feature points in the image, obtain the pixel coordinates of the feature points, establish a crosshair coordinate system on the image, obtain the 3D spatial coordinates of the feature points in the crosshair coordinate system, calculate the spatial pose of the crosshair relative to the camera, convert the spatial pose of the crosshair relative to the camera into the pose in the robot arm base coordinate system, and calculate the pose matrix of the crosshair coordinate system in the robot arm base coordinate system.

[0169] Step S4 specifically includes the following steps:

[0170] S41: Based on the identified feature points in the image, such as Figure 5 As shown, the pixel coordinates of the feature points are obtained, and the intrinsic parameters of the camera are calibrated. A crosshair coordinate system is established on the image. With the 0-degree posture as the reference, the origin of the crosshair coordinate system is set at the center of the surface where the feature points are located, and the origin of the crosshair coordinate system is used as the grab point.

[0171] S42: Based on the dimensions of the crosshead, obtain the 3D spatial coordinates of the feature points in the crosshead coordinate system. Assign the 3D spatial coordinates of the feature points to the pixel coordinates in the correct order. Combine this with the camera intrinsic parameters to calculate the spatial pose matrix of the crosshead coordinate system relative to the camera coordinate system. From the pose matrix Extract the pose information of the crosshead component relative to the camera, such as... Figure 6 As shown;

[0172] S43: Fix the checkerboard calibration plate to the end of the robotic arm that grips the crosshead part. Fix the camera at a set distance directly above the checkerboard calibration plate via a bracket, so that the camera is parallel to the horizontal plane, and take M images of the calibration plate in different postures.

[0173] The crosshead parts in this embodiment are characterized by their large quantity, identical postures but different angles, single-layer arrangement, and stationary position relative to the robotic arm and camera. Therefore, an "eye-to-hand" calibration method is adopted, which involves fixing a checkerboard calibration plate to the end of the robotic arm and fixing the camera to a set position directly above the calibration plate using a bracket (the distance must be such that the camera's field of view completely includes a batch of arranged crosshead parts). The camera lens is then parallel to the horizontal plane, and M images (M>>2) of the calibration plate in different postures are taken. This invention took 50 images.

[0174] S44: Based on the pose transformation matrix of the robotic arm end effector relative to the robotic arm base coordinate system. The position matrix of the checkerboard calibration plate at the end of the robotic arm and the pose transformation matrix of the camera relative to the checkerboard calibration board. Construct the pose relationship matrix corresponding to each calibration board image;

[0175] ;

[0176] in, Let be the pose transformation matrix of the camera relative to the base coordinate system of the robotic arm;

[0177] S45: For two different calibration board images, construct the pose transformation matrix using the pose relationship matrix. The equation to be solved;

[0178] ;

[0179] in, Let be the pose transformation matrix of the camera relative to the checkerboard calibration board corresponding to two different calibration board images. This represents the pose transformation matrix of the robotic arm end effector relative to the robotic arm base coordinate system for two different calibration plate images.

[0180] S46: Construct a set of equations to be solved corresponding to M calibration board images. In this embodiment, in order to reduce calibration error, it is necessary to list M-1 sets of equations to be solved to solve the matrix.

[0181] ;

[0182] in, Let be the pose transformation matrix of the camera relative to the checkerboard calibration board corresponding to two adjacent different calibration board images. This is the pose transformation matrix of the robotic arm end effector relative to the robotic arm base coordinate system corresponding to two adjacent different calibration plate images;

[0183] S47: Solving the pose transformation matrix using the system of equations to be solved Calculate the pose matrix of the crosshead coordinate system in the robot arm base coordinate system. ;

[0184] .

[0185] S5: Obtain the pose coordinates of the gripping point on the crosshead part in the robot arm's base coordinate system based on the pose matrix. Use the pose coordinates to plan the robot arm's end effector motion, drive the robot arm to grip the crosshead part, and assemble it. Figure 7 and Figure 8As shown, the robotic arm motion path is planned based on the pose coordinates to perform the gripping and assembly process of the crosshead part.

Claims

1. An intelligent positioning and gripping method for assembling crosshead parts, characterized in that, Includes the following steps: S1: Collect images of the placement postures of several crosshead parts, and label the top feature points on the images to obtain the feature points at the top of the crosshead parts and generate training data. S2: Construct a YOLO-Corner algorithm model for detecting feature points on the top of crosshead parts. The YOLO-Corner algorithm model includes a feature extraction layer, a feature fusion layer, and an output layer. Train the YOLO-Corner algorithm model using training data, take the marked feature points on the image as the output, correct the model parameters, and output the trained YOLO-Corner algorithm model. S3: Collect an image of the crosshead parts, input it into the trained YOLO-Corner algorithm model, identify the feature points on the top of the crosshead parts in the image, and use the feature points to locate the top of the crosshead parts in the image. S4: Obtain the pixel coordinates of the feature points, establish a crosshead coordinate system on the image, obtain the 3D spatial coordinates of the feature points in the crosshead coordinate system, calculate the spatial pose of the crosshead relative to the camera, convert the spatial pose of the crosshead relative to the camera into the pose in the robot arm base coordinate system, and calculate the pose matrix of the crosshead coordinate system in the robot arm base coordinate system. S5: Obtain the pose coordinates of the gripping point on the crosshead part in the base coordinate system of the robotic arm according to the pose matrix, use the pose coordinates to plan the end effector motion of the robotic arm, drive the robotic arm to grip the crosshead part and assemble it. The feature extraction layer includes a C3k2 module and a receptive field attention convolution module RFAConv. The C3k2 module introduces a feature complement mapping module FCM to form a C3k2_FCM module. The feature fusion layer includes an upsampling module EUCB, a triple attention mechanism, and a receptive field attention convolution module RFAConv. The triple attention mechanism includes three branches, and the receptive field attention convolution module RFAConv generates independent attention weights for each receptive field by focusing on spatial features in the texture features. The C3k2_FCM module processes images as follows: A1: Input features of the image X input ( X input ∈ R C×H×W The channel is divided into: αC Channels and The passage is divided into two parts; ； ； in, α This refers to the channel splitting ratio. R C×H×W Image features, including the number of image channels. C Vertical latitude H Width of horizontal dimension W ; They are respectively αC aisle, Image features of the channel, This is the channel segmentation function; A2: The segmented image features The data is passed to a standard 3x3 convolutional branch to extract channel feature information. Image features Passing the data to the pointwise convolution PWConv branch preserves shallow spatial location information. ; ； in, This is a mapping function between the spatial and semantic information of image features; A3: Channel feature information Global average pooling is introduced through channel interaction to obtain global information for each channel. Then, a sigmoid activation function is used to assign channel information weights to the global information of each channel. ; ； ； in, i Number the convolution kernel. k i For the first i One convolutional kernel, This is the mapping function for each feature layer channel in the channel interaction. For input number i Channel feature information of each convolutional kernel For the first i The output channel feature information of each convolutional kernel It is the sigmoid activation function. These are the pixel numbers for the vertical dimension and the pixel numbers for the horizontal dimension, respectively. The output function of the convolution kernel. For channel weight set; A4: Shallow spatial location information Spatial information weights are assigned to important information at each location through spatial interaction. ; ， ； in, For spatial clustering convolution mapping functions, For spatial weight set; A5: Weighting spatial information Mapping to channel feature information In the middle, channel information weight Mapping to shallow spatial location information In the middle, channel feature information is realized. With shallow spatial location information The complementary fusion yields feature maps of spatial and semantic relationships. ; ； The method by which the upsampling module EUCB processes the image is as follows: B1: Use an upsampling module with a scaling factor of 2 to upsample the feature map. The scale is magnified by 2 times, and then the magnified feature map is enhanced using 3×3 depthwise convolution (DWC), batch normalization (BN), and ReLU activation function. ; B2: Then, use 1×1 convolution. C 1×1 To reduce feature maps The channel outputs enhanced feature maps. ; ； in, This is a 1×1 convolution operation. It is the ReLU activation function. For batch normalization function, This is a 3×3 depthwise convolution operation. For upsampling operation, This is the output function of the upsampling module EUCB; The method for processing images using the triple attention mechanism is as follows: Enhance feature maps In the triple attention mechanism, three branches are used to enhance the feature map. Perform convolution operations to capture enhanced feature maps Based on the features corresponding to the feature points at the top of the crosshead part, we obtain the feature map corresponding to the feature points; The method for processing images using the triple attention mechanism specifically includes the following steps: C1: The input enhanced feature map Pass it to each of the three branches respectively; C2: In the first branch, the input augmented feature map Rotate 90° counterclockwise along the vertical axis to obtain the rotation tensor. Rotation tensor The shape is W × H × C ; The rotation tensor is obtained through the Z-Pool layer. Simplified to a shape of 2× H × C rotation tensor Then, it goes through a standard convolutional layer with a kernel size of k×k to reach a batch normalized (BN) layer; Batch normalized BN layers calculate the rotation tensor upper feature data x mean ,variance Feature data x Standardize, and then scale and translate the standardized rotation tensor. The shape is simplified to 1× H × C Output Finally, the Sigmoid activation function is used to generate information about the output. attention weights ; ； ； ； ； ； ； in, The feature function is max pooling. The characteristic function is the average pooling function. u The identifier for the feature data. x u For the first u Each feature data, m The number of feature data, The mean of the feature data, The variance of the feature data, For correction factor, These are the coefficients and deviations of the output function, respectively. For the first u Output of each feature data; Attention weight Applied to the rotation tensor Then rotate it 90° clockwise along the vertical axis to obtain the enhanced feature map. Output feature map of the original shape ; C3: In the second branch, the feature map will be enhanced. Rotate 90° counterclockwise around the horizontal axis to obtain the rotation tensor. Rotation tensor The shape is H × C × W The rotation tensor will be passed through the Z-Pool layer. Simplified to shape 2× C × W rotation tensor ; Rotation Tensor The process proceeds through a standard convolutional layer with a kernel size of k×k to a batch normalized (BN) layer, resulting in a simplified output shape of 1×k. C × W Output Finally, the Sigmoid activation function is used to generate information about the output. attention weights ; Attention weight Applied to the rotation tensor Then rotate it 90° clockwise along the horizontal axis to obtain the preserved and enhanced feature map. Output feature map of the original shape ; C4: In the third branch, enhance the feature map. Simplify directly to a shape of 2× using the Z-Pool layer. H × W tensor It passes through a standard convolutional layer with a kernel size of k×k to reach a batch normalized (BN) layer, with an output shape of 1× H × W Output Finally, the Sigmoid activation function is used to generate information about the output. attention weights Pay attention weights Apply to tensor The output feature map is obtained from the above. ; C5: Average aggregation of the outputs from the three branches to calculate the enhanced feature map. Corresponding output features We obtain the feature map corresponding to the feature points at the top of the crosshead part; ； The method by which the receptive field attention convolution module RFAConv processes images is as follows: The feature map corresponding to the feature point is input into the receptive field attention convolution module RFAConv. Through the spatial features of the receptive field, the attention weight of each receptive field is generated, and through the weighting operation, the recognized feature point is output. The feature point is used to locate the top of the crosshead part in the image.

2. The intelligent positioning and gripping method for assembling crosshead parts according to claim 1, characterized in that, The method for processing images by the receptive field attention convolution module RFAConv specifically includes the following steps: D1: The feature maps corresponding to the feature points are converted into receptive field spatial feature maps through group convolution (Group-Conv). During the conversion, each receptive field corresponds to an independent sliding window. The size of the receptive field spatial feature map is then enlarged to the size of the feature map corresponding to the feature points. v Multiply by 10 to generate a magnified feature map; D2: Perform global average pooling on the feature information of each receptive field in the magnified feature map, compressing the feature information of each receptive field into a single value; then, use 1x1 grouped convolution to perform information interaction on the average pooled feature information; finally, perform classification operation on the feature information after 1x1 grouped convolution to generate the attention weight of each receptive field and output the receptive field attention map. D3: The receptive field attention map and the receptive field spatial feature map are weighted and re-weighted, and the identified feature points are output through the convolutional layer. The feature points are used to locate the top of the crosshead part in the image. ； ； ； in, The classification function for the output layer. A global pooling function for the feature information of the receptive field. X To enlarge the feature map, For 1x1 grouped convolution, for Grouped convolution, For normalization function, z i It is the first in the input vector i K is the length of the input vector; ReLU(.) is the corrected linear unit function; For feature data at feature points, R F These are the feature points output.

3. The intelligent positioning and gripping method for assembling crosshead parts according to claim 1, characterized in that, Step S4 includes: S41: Based on the identified feature points in the image, obtain the pixel coordinates of the feature points, calibrate the camera's intrinsic parameters, establish a crosshair coordinate system on the image, take the 0-degree posture as the reference, set the origin of the crosshair coordinate system at the center of the surface where the feature points are located, and use the origin of the crosshair coordinate system as the grab point. S42: Based on the dimensions of the crosshead, obtain the 3D spatial coordinates of the feature points in the crosshead coordinate system. Assign the 3D spatial coordinates of the feature points to the pixel coordinates in the correct order, and calculate the spatial pose matrix of the crosshead component relative to the camera using the camera's intrinsic parameters. From the pose matrix Extract the pose information of the crosshead component relative to the camera; S43: Fix the checkerboard calibration plate to the end of the robotic arm that grips the crosshead part. Fix the camera at a set distance directly above the checkerboard calibration plate via a bracket, so that the camera is parallel to the horizontal plane, and take M images of the calibration plate in different postures. S44: Based on the pose transformation matrix of the robotic arm end effector relative to the robotic arm base coordinate system. The position matrix of the checkerboard calibration plate at the end of the robotic arm and the pose transformation matrix of the camera relative to the checkerboard calibration board. Construct the pose relationship matrix corresponding to each calibration board image; ； in, Let be the pose transformation matrix of the camera relative to the base coordinate system of the robotic arm; S45: For two different calibration board images, construct the pose transformation matrix using the pose relationship matrix. The equation to be solved; ； in, Let be the pose transformation matrix of the camera relative to the checkerboard calibration board corresponding to two different calibration board images. This represents the pose transformation matrix of the robotic arm end effector relative to the robotic arm base coordinate system for two different calibration plate images. S46: Construct a system of equations to be solved corresponding to M calibration plate images; ； in, Let be the pose transformation matrix of the camera relative to the checkerboard calibration board corresponding to two adjacent different calibration board images. This is the pose transformation matrix of the robotic arm end effector relative to the robotic arm base coordinate system corresponding to two adjacent different calibration plate images; S47: Solving the pose transformation matrix using the system of equations to be solved Calculate the pose matrix of the crosshead coordinate system in the robot arm base coordinate system. ; 。

Images