A gasket polishing visual positioning method for large-caliber seat ring ring gap block end face operation

Abstract

The application provides a gasket polishing visual positioning method for large-caliber seat ring ring mouth gasket end face operation, when the system is started, the model and the camera are initialized and set; after the robot polishing platform runs to a fixed position, the system sends a start camera image capturing and identification event, the two events are executed by thread 1 and thread 2 respectively; in thread 1, when the event information is received, the camera starts to capture the image, and the captured image is put into a shared memory queue; when thread 2 receives the identification event information, the image is taken from the shared memory queue, and inference is started; at the same time, thread 2 converts the left upper corner and right lower corner positions of the target frame meeting the identification condition in the inference result into offset pixel distances relative to the center position of the image, and reports the distances to the system; according to the offset pixel distance, the actual offset position is calculated by the upper system combined with the relative position relationship of the camera and the milling cutter, so that the milling cutter is moved to the corresponding starting polishing position, and automatic polishing is realized.

Description

Technical Field

[0001] This invention relates to the field of large-diameter seat ring pad processing technology, and in particular to a visual positioning method for pad grinding facing the end face of a large-diameter seat ring pad. Background Technology

[0002] In industrial manufacturing, precise grinding of targets is a crucial step in ensuring product quality. However, since robots cannot always position the milling cutter to a fixed location before grinding, accurate target positioning is necessary to determine the starting position for the milling cutter. However, the positioning of the support block faces numerous challenges in practical applications.

[0003] See Figure 1 , 2 Traditional image processing methods have limitations in locating pad blocks. Due to factors such as lighting and environment, pad blocks may exhibit shadows, varying degrees of corrosion, and surface defects in images, making it difficult for traditional methods to accurately pinpoint their location. To address this issue, this paper first attempts to detect pad blocks using a combination of traditional detection algorithms and OpenCV. In the traditional algorithm, the image is converted to grayscale, Gaussian blurred, and then the Canny operator is used for edge contour detection. Finally, the contours are filtered and fitted to obtain the target's contour information. Opening and closing operations are also incorporated to obtain a more accurate target location, removing unwanted areas. Furthermore, Hough transform is used for circular detection of the pad blocks, leveraging the shape prior of the pad blocks to attempt precise target location.

[0004] However, due to the complexity of the environment and the surface of the object being inspected, traditional algorithms struggle to accurately pinpoint the target's location. Furthermore, once parameters are fine-tuned for one sample, they need to be readjusted for other samples to correctly detect the pad's position. Therefore, traditional methods have significant limitations in practical applications and cannot meet the demands of industrial production. Summary of the Invention

[0005] This invention provides a visual positioning method for grinding pads on the end face of large-diameter seat ring pads. This system uses deep learning to overcome the shortcomings of traditional algorithms in stably identifying grinding targets. Combining classic deep learning detection and recognition structures, a deep learning algorithm, YOLO-DNet, is designed for pad grinding to achieve accurate target identification and positioning.

[0006] To achieve the above-mentioned technical features, the objective of this invention is as follows: a visual positioning method for grinding pads facing the end face of large-diameter seat ring pads, comprising the following steps: System initialization settings: When the system starts, it initializes the model and camera settings; System-triggered events: Once the robotic polishing platform reaches a fixed position, the system issues an event to start camera image capture and recognition. These two events are executed by thread 1 and thread 2, respectively. Execution of system thread 1: In thread 1, when an event message is received, the camera begins to capture images and puts the captured images into a shared memory queue; Execution of system thread 2: When thread 2 receives the recognition event information, it retrieves the image from the shared memory queue and begins inference. At the same time, thread 2 converts the upper left and lower right corners of the target boxes that meet the recognition conditions in the inference results into offset pixel distances relative to the center of the image and reports this distance to the system. Determining the system's grinding location: Based on the offset pixel distance, the upper-level system calculates the actual offset position by combining the relative positional relationship between the camera and the milling cutter, thereby moving the milling cutter to the corresponding starting grinding position to achieve automatic grinding.

[0007] The model employs the deep learning algorithm YOLO-DNet and uses the CC module to accurately identify targets in complex backgrounds. The CC module, through the combination of convolutional stream and CSAB module, enriches the gradient information flow and performs channel and spatial feature selection on the features of multi-stream convergence, thereby making the detection more accurate.

[0008] The CC module includes the Conv module and the CSAB module. The Conv module is a convolution + BatchNormalization + ReLU module, and the CSAB module is a Channel-Spatial Attention Block.

[0009] The specific process of the deep learning algorithm YOLO-DNet is as follows: First, the image is input into the network, and it is first subjected to convolution + normalization + SiLU activation. Then, the obtained features are fed into the subsequent CC Block module for feature extraction, and the features of the backbone network are output in the third CC Block module and the final SPPF module. Meanwhile, the PAN-FPN structure from general object detection in deep learning is adopted.

[0010] Simultaneously, two prediction heads are used to predict the target category, confidence score, and location at two higher scales. Finally, both prediction heads are decoupled to separate classification and detection, and do not depend on the anchor. When calculating the loss, positive and negative samples are selected based on the classification and regression metric scores. Binary cross-entropy loss is used to calculate the classification error, and DFL+CIOU loss is used to calculate the regression error.

[0011] After training to obtain the final model, it is deployed on an industrial control computer. Here, a quantization + ONNX + OpenCV solution is used for deployment.

[0012] Before deploying the model, the model is first converted and quantized. To avoid unnecessary calibration, the model is quantized using FP16 and converted to ONNX using a tool. Then, OpenCV is used to load the ONNX model and related configurations, build the network, and obtain the DNN inference entity; When the image arrives, the input image is preprocessed and inferred by the DNN model in the same way as during model training. The inference results are then NMS and post-processed and output. Finally, the identified and filtered target boxes are transformed and reported to the system.

[0013] Beneficial effects of this invention: 1. The deep learning scheme designed in this invention overcomes the shortcomings of traditional algorithms in stably identifying the polishing target. Combining the classic deep learning detection and recognition structure, a deep learning algorithm YOLO-DNet is designed for pad polishing to achieve accurate target identification and positioning.

[0014] 2. The deep learning scheme used in this invention designs a basic feature extraction module CC for feature extraction. By using a gradient splitting mechanism combined with CSAB channels and a spatial attention module, the features of targets in the image are enhanced, overcoming the problem of difficulty in identification caused by the complexity of the environment and the surface of the object being inspected.

[0015] 3. This invention designs a deployment scheme for the Yolo-DNet algorithm that combines OpenCV and quantization, and uses multi-threaded parallel technology to design a visual positioning system, thereby optimizing the execution efficiency of the target positioning and recognition system. Attached Figure Description

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] Figure 1 This is an example of the target to be detected in this invention.

[0018] Figure 2 Not a traditional algorithm.

[0019] Figure 3 This invention relates to the deep learning algorithm architecture YOLO-DNet.

[0020] Figure 4 This is the CC module of the present invention.

[0021] Figure 5 This is the training loss curve for this invention.

[0022] Figure 6 This is the test result (PR curve) of the present invention.

[0023] Figure 7 (a) and (b) are examples of the detection results of the present invention.

[0024] Figure 8 This is the vision system integration solution of the present invention. Detailed Implementation

[0025] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0026] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0027] Example 1: The present invention mainly includes two aspects: (1) the design of a target recognition and detection algorithm; and (2) the integration of a vision system. These will be described in detail below.

[0028] 1. Algorithm Design: Due to the complex environment, corrosion, and defects of the target blocks, it is difficult to accurately locate the targets when using deep learning for identification and detection. To accurately identify targets in complex backgrounds, a CC module was designed. By combining convolutional gradient splitting and CSAB modules, it not only enriches the gradient information flow but also performs channel and spatial feature selection on the features converging from multiple flows, thus making the detection more accurate. Here, Conv represents convolution + BatchNormalization + ReLU; CSAB represents Channel-Spatial Attention Block, which uses pooling + 2D convolution + Sigmoid activation function to generate feature channel weights to select feature channels. Then, it uses the mean and maximum method to calculate the feature spatial location weights. Finally, the weighted channel features are multiplied by the weights to obtain features with spatial location information. CC represents the Conv gradient splitting module + CSAB module, which first divides the features into multiple parts for processing and then uses them as input to CSAB to obtain channel and spatial weighted features to highlight the target's position in the feature space. The algorithm is described in detail below.

[0029] See Figure 3 , 4 First, the image is input into the network, where it undergoes convolution, normalization, and SiLU activation. Then, the resulting features are fed into the subsequent CC Block module for feature extraction, and the backbone network features are output in the third block and the final SPPF (Spatial Pyramid Pooling Fast) module. SPPF, representing fast spatial pyramid pooling, is used to extract multi-scale feature information to enhance the model's receptive field. Simultaneously, the PAN-FPN structure from general deep learning object detection is adopted, using upsampling and the C2f feature extraction layer commonly used in detection to enhance multi-scale feature fusion and mitigate the problem of inaccurate prediction due to the loss of semantic and positional information. Furthermore, due to the high accuracy requirements for target localization and the close proximity of the selected camera to the target, resulting in a large target area, two prediction heads are used to predict the target category, confidence score, and location at two higher scales. Finally, both prediction heads employ a mainstream decoupling method, separating classification and detection without relying on anchors. When calculating the loss, positive and negative samples are selected based on the classification and regression metric scores. The classification error is calculated using binary cross-entropy loss, and the regression error is calculated using DFL+CIOU loss.

[0030] Training and testing results: 6000 images were collected and labeled, and the test set and validation set were divided in an 8:2 ratio. The model converged after 100 epochs of training. Figure 5 The training curves represent the regression and classification losses. A test set of 1200 images was constructed to test the model, and the test results were evaluated using the area under the PR curve. P (Precision) represents the proportion of true positive samples among the predicted positive samples; R (Recall) represents the proportion of predicted positive samples among the true positive samples. Higher P and R values ​​indicate better model performance.

[0031] The recognition results are displayed as follows Figure 7 As shown in (a) and (b).

[0032] 2. Vision system integration: After training to obtain the final model, it needs to be deployed on an industrial control computer. Here, a quantization + ONNX + OpenCV solution is used for deployment, such as... Figure 8 As shown.

[0033] Before deploying the model, it needs to be converted and quantized. To avoid unnecessary calibration, an FP16 quantization model is used, and the model is converted to an ONNX model using a tool. Then, OpenCV is used to load the ONNX model and related configurations, build the network, and obtain the DNN inference entity. When the image arrives, the input image undergoes the same preprocessing and DNN model inference as during model training. The inference results are then subjected to NMS and post-processing before output. Finally, the identified and filtered target boxes are converted and reported to the system.

[0034] 3. System Operation: See Figure 8 When the system starts, it initializes the model and camera settings; Once the robotic polishing platform reaches a fixed position, the system issues an event to start camera image capture and recognition. These two events are executed by thread 1 and thread 2, respectively. In thread 1, when an event message is received, the camera begins to capture images and puts the captured images into a shared memory queue; When thread 2 receives the recognition event information, it retrieves the image from the shared memory queue and begins inference. At the same time, thread 2 converts the upper left and lower right corners of the target boxes that meet the recognition conditions in the inference results into offset pixel distances relative to the center of the image and reports this distance to the system. The upper-level system calculates the actual offset position based on the relative positional relationship between the camera and the milling cutter, thereby moving the milling cutter to the corresponding starting grinding position to achieve automatic grinding.

Claims

1. A visual positioning method for grinding pads facing the end face of large-diameter seat ring pads, comprising the following steps: System initialization settings: When the system starts, it initializes the model and camera settings; System-triggered events: Once the robotic polishing platform reaches a fixed position, the system issues an event to start camera image capture and recognition. These two events are executed by thread 1 and thread 2, respectively. Execution of system thread 1: In thread 1, when an event message is received, the camera begins to capture images and puts the captured images into a shared memory queue; Execution of system thread 2: When thread 2 receives the recognition event information, it retrieves the image from the shared memory queue and begins inference. At the same time, thread 2 converts the upper left and lower right corners of the target boxes that meet the recognition conditions in the inference results into offset pixel distances relative to the center of the image and reports this distance to the system. Determining the system's grinding location: Based on the offset pixel distance, the upper-level system calculates the actual offset position by combining the relative positional relationship between the camera and the milling cutter, thereby moving the milling cutter to the corresponding starting grinding position to achieve automatic grinding; The model employs the deep learning algorithm YOLO-DNet and uses the CC module to accurately identify targets in complex backgrounds. The CC module enriches the gradient information flow by combining the convolutional stream and the CSAB module, and performs channel and spatial feature selection on the features of multi-stream convergence, thereby making the detection more accurate. The CC module includes the Conv module and the CSAB module. The Conv module is a convolution + BatchNormalization + ReLU module, and the CSAB module is a Channel-Spatial Attention Block. The specific process of the deep learning algorithm YOLO-DNet is as follows: First, the image is input into the network, and it is first subjected to convolution + normalization + SiLU activation. Then, the obtained features are fed into the subsequent CC Block module for feature extraction, and the features of the backbone network are output in the third CC Block module and the final SPPF module. Meanwhile, the PAN-FPN structure from general object detection in deep learning is adopted.

2. The visual positioning method for grinding pad blocks facing the end face of a large-diameter seat ring pad block according to claim 1, characterized in that, Simultaneously, two prediction heads are used to predict the target category, confidence score, and location at two higher scales. Finally, both prediction heads are decoupled to separate classification and detection, and do not depend on the anchor. When calculating the loss, positive and negative samples are selected based on the classification and regression metric scores. Binary cross-entropy loss is used to calculate the classification error, and DFL+CIOU loss is used to calculate the regression error.

3. The visual positioning method for grinding pad blocks facing the end face of a large-diameter seat ring pad block according to claim 1, characterized in that, After training to obtain the final model, it is deployed on an industrial control computer. Here, a quantization + ONNX + OpenCV solution is used for deployment.

4. The visual positioning method for grinding pad blocks facing the end face of a large-diameter seat ring pad block according to claim 3, characterized in that, Before deploying the model, the model is first converted and quantized. To avoid unnecessary calibration, the model is quantized using FP16 and converted to ONNX using a tool. Then, OpenCV is used to load the ONNX model and related configurations, build the network, and obtain the DNN inference entity; When the image arrives, the input image is preprocessed and inferred by the DNN model in the same way as during model training. The inference results are then NMS and post-processed and output. Finally, the identified and filtered target boxes are transformed and reported to the system.

Images