A method for identifying large lumps of pellets in a disk pelletizing machine

Abstract

The present invention discloses a method for identifying large pellets in a disk pelletizing machine. The method for identifying large pellets in the disk pelletizing machine comprises the following steps: S1, image acquisition; S2, image processing; S3, building a target detection model; S4, training the target detection model; S5, real-time image prediction; S6, setting the starting value of the robot; S7, fishing operation; S8, information storage. The present invention solves the problems in the prior art that the manual fishing method is time-consuming, laborious and low in efficiency, cannot complete the operation in real time, and there are certain potential safety hazards in the overhaul.

Description

Technical Field

[0001] The present invention relates to the technical field of monitoring, and particularly relates to a method for identifying large lumps of pellets in a disk pelletizer. Background Art

[0002] The pellet production process includes processes such as raw material preparation, batching, mixing, pelletizing, drying and roasting, cooling, finished product and return ore treatment, etc. Pelletizing is an important link among them, and the quality of green pellets directly affects the quality of the final finished pellets. The disk pelletizer generates green pellets within a certain particle size range through operations such as feeding, adding water, and speed regulation. However, during the actual pelletizing process, large lumps of pellets with a volume larger than the preset range will be generated. And the large lumps of pellets have the following many negative impacts: 1. The large lumps of pellets will affect the movement trajectory of small pellets and hinder the growth of small pellets; 2. Qualified pellets cannot be discharged in time and continue to grow in the disk pelletizer, resulting in the generation of more large lumps of pellets; 3. Small pellets are discharged from the disk pelletizer in advance under collision, making the particle size of the green pellets not meet the standard and the strength poor. Therefore, it is necessary to fish out the large lumps in time. At present, the method of manual fishing is time-consuming, laborious and inefficient, cannot complete the operation in real time, and there are certain safety hazards. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for identifying large lumps of pellets in a disk pelletizer with high working efficiency, which can automatically fish out large lumps in time and eliminate safety hazards, aiming at the deficiencies existing in the prior art.

[0004] To achieve the above purpose, the technical solution adopted by the method for identifying large lumps of pellets in a disk pelletizer of the present invention is as follows:

[0005] A method for identifying large lumps of pellets in a disk pelletizer includes the following steps:

[0006] S1. Image acquisition: Acquire images of large lumps of pellets generated during the operation of the disk pelletizer;

[0007] S2. Image processing:

[0008] S21. Label the images acquired in S1 into two categories: one category is large lumps of pellets, marked as dakuai, and the other category is robots, marked as mechine;

[0009] S22. Establish a data set;

[0010] S3. Build an object detection model:

[0011] S31. Output feature maps of three different sizes: The main structure of the object detection model is YOLOv3, and only convolutional layers are used in the network of the object detection model. Three feature maps of different sizes are output respectively through the backbone network in YOLOv3. Among them, the small-size feature map is used to detect large objects, the medium-size feature map is used to detect medium-sized objects, and the large-size feature map is used to detect small objects. The three feature maps of different sizes form a feature pyramid structure for feature fusion. The output dimension of the feature map is N×N×[3×(4 + 1 + a)], where N×N is the grid points divided by the output feature map, 3 means three anchor boxes are used for each grid point, 4 represents the values of the prediction box, and the values of the prediction box include the abscissa t x , the ordinate t y , of the center point of the prediction box w , the width t h , and the height t

[0012] S32. Perform weighted processing on the three feature maps of different sizes;

[0013] S33. Predict the three weighted feature maps of different sizes: The object detection model uses double-layer convolution to predict the three weighted feature maps respectively. The double-layer convolution is a 3×3 convolution and a 1×1 convolution. The 3×3 convolution is used for feature integration, and the 1×1 convolution is used to adjust the number of channels;

[0014] S4. Train the object detection model:

[0015] S41. Augment the images: The input images of the model are changed by operations such as cropping, rotating, scaling, and color transformation of the images to improve the generalization ability of the model;

[0016] S42. Freeze the image feature extraction network and only train other neural networks: Since the backbone network Darknet53 has been pre-trained on other datasets and has the ability of preliminary feature extraction, the image feature extraction network, that is, Darknet53, is frozen first, and only other neural networks are trained to speed up its training speed;

[0017] S421. Calculate the model loss function:

[0018] The model loss function is Loss = Loss1 + Loss2 + Loss3 + Loss4,

[0019] Loss1 is the center coordinate error, Loss2 is the width and height coordinate error, Loss3 is the confidence error, and Loss4 is the classification error,

[0020]

[0021] Among them, λ coord 、-λ noobj are preset constants; S2 is the number of grids; indicates whether the j-th anchor box in the i-th grid corresponds to the predicted object. If it corresponds, the value is 1; otherwise, the value is 0; x represents the abscissa of the upper left corner point of the target box, y represents the ordinate of the upper left corner point of the target box, w represents the width of the target box, and h represents the height of the target box; is the true value. If the bounding box of the grid corresponds to the predicted box, the value is 1; otherwise, the value is 0; indicates that the anchor box in the i-th grid does not correspond to the predicted object;

[0022] S422. Use the gradient descent optimization algorithm for neural network training:

[0023] After obtaining the loss function Loss of the model, the model uses the Adam gradient descent optimization algorithm to train the neural network in the model, reduce the loss function, and improve the accuracy of the model;

[0024] S43. Unfreeze the image feature extraction network and train the entire model:

[0025] After completing the training of the network other than Darknet53, unfreeze it and train the entire object detection model to continuously improve the performance of the model. The training steps are the same as those in S42;

[0026] S5. Real-time image prediction:

[0027] S51. Send the real-time acquired image into the trained model;

[0028] S52. Decode the detection box according to the prior box and the output feature map. The formula is as follows:

[0029] b x = σ(t x ) + c x ;

[0030] b y = σ(t y ) + c y ;

[0031]

[0032] Among them, b x represents the abscissa of the center point of the predicted box on the feature map, b y represents the ordinate of the center point of the predicted box on the feature map, b w represents the width of the predicted box on the feature map, b hrepresents the height of the prediction box on the feature map, σ represents the sigmoid activation function, c x 、c y represents the grid number coordinates of the current position relative to the upper left grid offset in the feature map, P w 、P h are the width and height of the pre-set anchor box mapped to the feature map;

[0033] S53. Perform score sorting and non-maximum suppression screening to obtain the filtered prediction boxes that meet the conditions;

[0034] S6. Set the robot startup value;

[0035] S7. Fishing operation: When the robot startup preset value is met, send a startup command to the robot to perform a fishing operation;

[0036] S8. Information storage: Save the large pellet images and related information locally.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] 1. Automatically judge whether there are large pellets in the disc pelletizer through vision technology, and automatically control the ball fishing robot to perform fishing operations, improving work efficiency and eliminating safety hazards.

[0039] 2. During the training process, the training set is enhanced with data and then sent into the model for training until convergence, enhancing the robustness of the model.

[0040] 3. Set a startup value for the robot to avoid the robot damaging the growth of small balls due to frequent fishing operations, thus ensuring the quality of green balls, while reducing the wear of the robot's front claws and power consumption. Description of the Drawings

[0041] Figure 1 is the flowchart of the method for identifying large pellets in the disc pelletizer of the present invention;

[0042] Figure 2 are the large pellets detected using the object detection algorithm;

[0043] Figure 3 is the display interface of the large pellet identification system for the disc pelletizer.

[0044] Among them, 1 is the left ladder frame, 11 is the left handrail, 2 is the right ladder frame, 21 is the right handrail, 3 is the left bottom frame, 31 is the left roller, 32 is the left counterweight, 4 is the right bottom frame, 41 is the right roller, 42 is the right counterweight, 5 is the first connection mechanism, 51 is the first male plug, 52 is the first female plug, 6 is the left half-bridge, 61 is the left fence, 7 is the right half-bridge, 71 is the right fence, 8 is the second connection structure, 81 is the second male plug, 82 is the second female plug, 83 is the bolt, 9 is the third connection mechanism, 91 is the third male plug, and 92 is the third female plug. Detailed implementation manners

[0045] The following further clarifies the present invention in conjunction with the accompanying drawings and specific implementation manners. It should be understood that these implementation manners are only used to illustrate the present invention and not to limit the scope of the present invention. After reading the present invention, those skilled in the art's various equivalent modifications of the present invention all fall within the scope defined by the appended claims of this application.

[0046] As Figures 1 - 3 shown, a method for identifying large lumps of pellets in a disk pelletizing machine includes the following steps:

[0047] S1. Image acquisition: Install a monitoring camera on the railing near the disk pelletizing machine so that it faces the disk pelletizing machine directly, which is used to collect the video of large lumps of pellets generated during the operation of the disk pelletizing machine. Extract the images from the video in a certain number of frames in a loop, and manually filter out the images that do not contain large lumps of pellets;

[0048] S2. Image processing:

[0049] S21. Since the images may contain abnormal shapes during the grasping process of the robot arm, which affects the accuracy of recognition, the images collected in S1 are manually labeled using labelimg and divided into two categories: one category is large lumps of pellets, marked as dakuai, and the other category is the robot, marked as mechine;

[0050] S22. Establish a data set, which is divided into a training set, a validation set, and a test set for subsequent training of the deep learning object detection network. The sum of the training set and the validation set and the test set is in a ratio of 9:1, and the ratio of the training set to the validation set is 9:1;

[0051] S3. Build an object detection model:

[0052] S31. Output three feature maps of different sizes: The main structure of the target detection model is yolov3, and the network in the target detection model only uses convolutional layers. The backbone network in yolov3 outputs three feature maps of different sizes, where small-size feature maps detect large objects, medium-size feature maps detect medium objects, and large-size feature maps detect small objects. The three feature maps of different sizes form a feature pyramid structure for feature fusion. The output dimension of the feature map is N×N×[3×(4+1+a)], where N×N is the grid points of the output feature map, 3 means that each grid point uses three anchor boxes, and 4 means the value of the prediction box. The value of the prediction box includes the horizontal coordinate t of the center point of the prediction box. x , the vertical coordinate t of the center point of the prediction box y , the width of the prediction box t w and the height t of the predicted box h , 1 represents the confidence of the prediction box, and a represents the category of the target;

[0053] S32, performing weighted processing on the feature maps of three different sizes;

[0054] S33. Predict the three weighted feature maps of different sizes: The target detection model uses a double-layer convolution to predict the three weighted feature maps respectively. The double-layer convolution is a 3×3 convolution and a 1×1 convolution. The 3×3 convolution is used for feature integration, and the 1×1 convolution is used to adjust the number of channels.

[0055] S4. Train the target detection model:

[0056] S41. Enhanced image: By performing operations such as cropping, rotation, scale transformation, and color transformation on the image, the model’s input image is changed to improve the model’s generalization ability.

[0057] S42. Freeze the image feature extraction network and only train other neural networks: Since the backbone network Darknet53 has been pre-trained on other datasets and has preliminary feature extraction capabilities, the image feature extraction network, namely Darknet53, is frozen first, and only other neural networks are trained to speed up their training;

[0058] S421. Calculation of model loss function:

[0059] The model loss function is Loss = Loss1 + Loss2 + Loss3 + Loss4,

[0060] Loss1 is the center coordinate error, Loss2 is the width and height coordinate error, Loss3 is the confidence error, and Loss4 is the classification error.

[0061]

[0062]

[0063] where λ coord and -λ noobj are preset constants; S2 is the number of grids; indicates whether the j-th anchor box in the i-th grid corresponds to the predicted object. If it corresponds, the value is 1; otherwise, the value is 0. x represents the abscissa of the upper left corner point of the target box, y represents the ordinate of the upper left corner point of the target box, w represents the width of the target box, and h represents the height of the target box; is the ground truth value. If the bounding box of the grid corresponds to the predicted box, the value is 1; otherwise, the value is 0; indicates that the anchor box in the i-th grid does not correspond to the predicted object;

[0064] S422. Use the gradient descent optimization algorithm for neural network training:

[0065] After obtaining the loss function Loss of the model, the model uses the Adam gradient descent optimization algorithm to train the neural network in the model, reduce the loss function, and improve the accuracy of the model;

[0066] S43. Unfreeze the image feature extraction network and train the entire model:

[0067] After completing the training of the network other than Darknet53, unfreeze it and train the entire object detection model to continuously improve the performance of the model. The training steps are the same as those in S42;

[0068] S5. Real-time image prediction:

[0069] S51. Send the real-time acquired image into the trained model: Use the RTSP protocol to read the video of the camera in real time;

[0070] S52. Decode the detection box according to the prior box and the output feature map. The formula is as follows:

[0071] b x = σ(t x ) + c x ;

[0072] b y = σ(t y ) + c y ;

[0073]

[0074] where b x represents the abscissa of the center point of the predicted box on the feature map, and b y represents the ordinate of the center point of the predicted box on the feature map, bw Denote the width of the prediction box on the feature map, b h Denote the height of the prediction box on the feature map, σ represents the sigmoid activation function, c x 、c y Denote the grid number coordinates of the current position relative to the top - left grid offset in the feature map, P w 、P h are the width and height of the pre - set anchor box mapped to the feature map;

[0075] S53. Perform score sorting and non - maximum suppression screening to obtain the prediction boxes that meet the conditions after screening;

[0076] S6. Robot startup value setting: Since the robot's frequent scooping operations will damage the growth of the small balls, affect the quality of the green balls, and at the same time wear the front claws of the robot and consume electricity, it is necessary to preset the robot startup conditions. If the detected large - sized ball agglomerates are greater than a certain number, or there is a prediction box area greater than the pre - set large - sized ball agglomerate area threshold, then control the robot to perform a scooping operation;

[0077] S7. Scooping operation: If the number of detected large - sized ball agglomerates reaches a certain threshold or the area of the detected large - sized ball agglomerates reaches a certain threshold, send a program running instruction to the robot to perform a grasping operation. If no large - sized ball agglomerates are detected, send a program pause instruction to the robot, and at this time the robot will be in a waiting state;

[0078] S8. Information storage: Save the large - sized ball agglomerate images and related information locally.

Claims

1. A method for identifying large pellets in a disk pelletizer, characterized in that, It includes the following steps: S1. Image acquisition: Acquire images of large lumps of pellets generated during the operation of the disc pelletizer; S2. Image processing: S21. Label the images acquired in S1 into two categories: one is large lumps of pellets, labeled as "dakuai", and the other is the robot, labeled as "mechine"; S22. Establish a dataset; S3. Build an object detection model: S31. Output feature maps of three different sizes: The main structure of the object detection model is yolov3, and only convolutional layers are used in the network of the object detection model. Three feature maps of different sizes are output through the backbone network in yolov3. Among them, the small-sized feature map detects large objects, the medium-sized feature map detects medium-sized objects, and the large-sized feature map detects small objects. The three feature maps of different sizes form a feature pyramid structure for feature fusion. The output dimension of the feature map is N×N×[3×(4 + 1 + a)], where N×N is the grid points divided by the output feature map, 3 means using three anchor boxes for each grid point, 4 represents the values of the prediction box, and the values of the prediction box include the abscissa t x of the center point of the prediction box y , the ordinate t w of the center point of the prediction box h , the width t of the prediction box and the height t of the prediction box , 1 represents the confidence of the prediction box, and a represents the category of the target; S32. Perform weighted processing on three feature maps of different sizes; S33. Predict the three feature maps of different sizes after weighting: The object detection model uses double-layer convolution to predict the three weighted feature maps respectively. The double-layer convolution is a 3×3 convolution followed by a 1×1 convolution. The 3×3 convolution is used for feature integration, and the 1×1 convolution is used to adjust the number of channels; S4. Train the object detection model: S41. Augment images: Change the input images of the model by performing operations such as cropping, rotation, scale transformation, and color transformation on the images to improve the generalization ability of the model; S42. Freeze the image feature extraction network and only train other neural networks: Since the backbone network Darknet53 has been pre-trained on other datasets and has preliminary feature extraction capabilities, first freeze the image feature extraction network, that is, Darknet53, and only train other neural networks to speed up its training; S421. Calculate the model loss function: The model loss function is Loss = Loss1 + Loss2 + Loss3 + Loss4, where Loss1 is the center coordinate error, Loss2 is the width and height coordinate error, Loss3 is the confidence error, and Loss4 is the classification error; Among them, λ coord and -λ noobj are preset constants; S 2 is the number of grids; indicates whether the j-th anchor box in the i-th grid corresponds to the predicted object. If it corresponds, the value is 1; otherwise, the value is 0. x represents the abscissa of the upper left corner point of the target box, y represents the ordinate of the upper left corner point of the target box, w represents the width of the target box, and h represents the height of the target box; is the true value. If the bounding box of the grid corresponds to the predicted box, the value is 1; otherwise, the value is 0; indicates that the anchor box in the i-th grid does not correspond to the predicted object; S422. Use the gradient descent optimization algorithm to train the neural network: After obtaining the loss function Loss of the model, the model uses the Adam gradient descent optimization algorithm to train the neural network in the model, reduce the loss function, and improve the accuracy of the model; S43. Unfreeze the image feature extraction network and train the entire model: After completing the training of the network other than Darknet53, unfreeze it and train the entire object detection model to continuously improve the performance of the model. The training steps are the same as S42; S5. Real-time image prediction: S51. Send the real-time acquired images into the trained model; S52. Decode the detection boxes according to the prior boxes and the output feature maps. The formula is as follows: b x = σ(t x ) + c x ; b y = σ(t y ) + c y ; Among them, b x represents the abscissa of the center point of the prediction box on the feature map, b y represents the ordinate of the center point of the prediction box on the feature map, b w represents the width of the prediction box on the feature map, b h represents the height of the prediction box on the feature map, σ represents the sigmoid activation function, c x and c y represent the grid number coordinates of the current position relative to the upper left corner grid offset in the feature map, P w and P h are the widths and heights of the pre-set anchor box mapped to the feature map; S53. Perform score sorting and non-maximum suppression screening to obtain the predicted boxes that meet the conditions after screening; S6. Set the robot startup value; S7. Retrieval operation: When the preset value for robot startup is met, send a startup command to the robot to perform a retrieval operation; S8. Information storage: Save the large lump pellet images and related information locally.

Images