Grabbing method and device for tobacco filter stick box, electronic equipment and medium

Abstract

The embodiment of the invention relates to the field of tobacco filter stick production and logistics automation, and provides a tobacco filter stick box grabbing method and device, electronic equipment and a medium. The method comprises the steps that image data of a target tobacco filter stick box to be grabbed currently by a stacking robot is obtained; inputting the image data into a tobacco filter stick box rotation target detection model, so that the tobacco filter stick box rotation target detection model performs rotation frame positioning on the tobacco filter stick box in the image data, and a positioning result is obtained; and on the basis of the positioning result, the orientation of a gripper of the stacking robot is adjusted to grab the target tobacco filter stick box to be grabbed. Therefore, the tobacco filter stick box in any posture can be detected, the grabbing direction of the stacking robot is correspondingly adjusted, and the production efficiency is improved.

Description

Technical Field

[0001] The invention relates to the field of tobacco filter rod production and logistics automation, and in particular to a method, device, electronic equipment and medium for grabbing a tobacco filter rod box. Background Art

[0002] As a robot system used for automatic palletizing operations, the palletizing robot plays a vital role in the tobacco production industry in various production links, such as raw cigarette pack palletizing, semi-finished tobacco box palletizing, finished cigarette piece automatic palletizing, carton folding palletizing, and auxiliary material automatic palletizing.

[0003] The fully automatic production process of cigarette filter rods includes: the automated guided vehicle (AGV), referred to as the AGV cart, transports the empty tobacco filter rod box to the bracket of the designated production machine; the palletizing robot grabs the empty tobacco filter rod box from the top vertically downward and sends it to the conveyor at the entrance of the automatic cartoning machine to complete automatic loading; the palletizing robot grabs the full tobacco filter rod box and sends it to the bracket near the production machine; the AGV cart transports the full tobacco filter rod box out of the production area. In this entire production process, the direction of the tobacco filter rod box in the bracket area and the conveyor at the entrance and exit of the automatic cartoning machine is fixed, and the palletizing robot without a camera performs the grabbing work in the direction of the flush tobacco filter rod box according to a fixed procedure.

[0004] With regard to the above-mentioned fully automatic production process of cigarette filter rods, when the position and direction of the tobacco filter rod box in the bracket area and the automatic cartoning machine entrance and exit conveyor in the production area are greatly offset, the stacking robot without a camera will still work according to a fixed procedure, which will not be able to grab the tobacco filter rod box normally or may damage the tobacco filter rod box. The only way is to manually discover the abnormal situation in time, suspend the work of the stacking robot, adjust the position and direction of the tobacco filter rod box, and then give the stacking robot a command to start working again. Such abnormal situations have greatly affected production safety and greatly reduced production efficiency. Summary of the invention

[0005] The present invention provides a method, device, electronic equipment and medium for grasping a tobacco filter rod box, which are used to solve the defect in the prior art that a stacking robot without a camera cannot normally grasp the tobacco filter rod box when the position direction of the tobacco filter rod box placed on the bracket area and the entrance and exit conveyor of the automatic cartoning machine in the production area is greatly offset, and the detection of the tobacco filter rod box in any direction can be achieved. The stacking robot adjusts the grasping direction accordingly, thereby ensuring production safety and improving production efficiency.

[0006] The invention provides a method for grabbing a tobacco filter rod box, comprising the following steps.

[0007] Acquire image data of a target tobacco filter rod box to be currently grasped by the palletizing robot; The image data is input into a tobacco filter rod box rotating target detection model, so that the tobacco filter rod box rotating target detection model performs a rotating frame positioning on the tobacco filter rod box in the image data to obtain a positioning result; Based on the positioning result, the gripper of the palletizing robot is adjusted to grasp the target tobacco filter rod box to be grasped; The tobacco filter rod box rotating target detection model is trained based on the following steps: Acquire tobacco filter rod box sample images in different rotation postures; Annotating the sample images of the tobacco filter rod box with a rotation frame based on the rotation posture of the tobacco filter rod box, and dividing the annotated sample images of the tobacco filter rod box into a model training set, a model verification set, and a model test set; The YOLOv7 target detection model is trained by the model training set, so that the YOLOv7 target detection model learns the rotation posture and the corresponding rotation frame position of the tobacco filter rod box in the model training set, and the model performance of the trained YOLOv7 target detection model is evaluated by the model verification set to obtain a model evaluation result; When the model evaluation result meets the first preset condition, the evaluated YOLOv7 target detection model is tested by using the model test set to obtain a model test result; When the model test result meets the second preset condition, it is determined that the training of the YOLOv7 target detection model is completed, and the trained YOLOv7 target detection model is used as a tobacco filter rod box rotation target detection model.

[0008] According to the method provided by the present invention, the method further comprises: The rotation box is defined using the long side representation; The sample image of the tobacco filter rod box is annotated with a rotation frame based on the rotation posture and definition method of the tobacco filter rod box.

[0009] According to the method provided by the present invention, the method further comprises: The YOLOv7 target detection model is trained using the model training set, and the target rotation box regression loss function is calculated after each training; When the target rotation frame regression loss function tends to be stable, the model verification set is input into the trained YOLOv7 target detection model to obtain a first predicted rotation frame of the tobacco filter rod box output by the model; Using a circular smooth label method to compare the angular distance between the first predicted rotation box and the labeled rotation box of the model validation set; The model performance of the trained YOLOv7 target detection model is evaluated based on the angular distance to obtain a model evaluation result.

[0010] According to the method provided by the present invention, the method further comprises: When the angular distance is less than a preset threshold, determining that the model evaluation result meets a first preset condition; Inputting the model test set into the evaluated YOLOv7 target detection model to obtain a second predicted rotation frame of the tobacco filter rod box output by the model; A model test result is determined based on the second predicted rotation box and the labeled rotation box of the model test set.

[0011] According to the method provided by the present invention, the method further comprises: When the angular distance between the second predicted rotation box and the labeled rotation box of the model test set is less than the preset threshold, it is determined that the training of the YOLOv7 target detection model is completed, and the trained YOLOv7 target detection model is used as the tobacco filter rod box rotation target detection model.

[0012] According to the method provided by the present invention, the method further comprises: When the angular distance is greater than a preset threshold, the weight parameters of the YOLOv7 target detection model are adjusted, and the YOLOv7 target detection model is retrained using the model training set.

[0013] The present invention also provides a tobacco filter rod box grabbing device, comprising the following modules: An acquisition module, used to acquire image data of a target tobacco filter rod box to be currently grasped by the palletizing robot; A positioning module, used for inputting the image data into a tobacco filter plug box rotating target detection model, so that the tobacco filter plug box rotating target detection model performs a rotating frame positioning on the tobacco filter plug box in the image data to obtain a positioning result; A grasping module, used for adjusting the grasper of the palletizing robot to grasp the target tobacco filter rod box to be grasped based on the positioning result; A model training module is used to obtain sample images of tobacco filter rod boxes with different rotation postures; annotate the sample images of tobacco filter rod boxes with rotation boxes based on the rotation posture of the tobacco filter rod boxes, and divide the annotated sample images of tobacco filter rod boxes into a model training set, a model verification set and a model test set; train a YOLOv7 target detection model through the model training set so that the YOLOv7 target detection model learns the rotation posture and corresponding rotation box position of the tobacco filter rod boxes in the model training set, and evaluate the model performance of the trained YOLOv7 target detection model through the model verification set to obtain a model evaluation result; when the model evaluation result meets a first preset condition, test the evaluated YOLOv7 target detection model through the model test set to obtain a model test result; when the model test result meets a second preset condition, determine that the training of the YOLOv7 target detection model is completed, and use the trained YOLOv7 target detection model as a tobacco filter rod box rotation target detection model.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for grabbing a tobacco filter rod box as described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for grabbing a tobacco filter rod box as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for grabbing a tobacco filter rod box as described above is implemented.

[0017] The method, device, electronic device and medium for grasping a tobacco filter rod box provided by the present invention obtain image data of a target tobacco filter rod box to be grasped by a palletizing robot; input the image data into a tobacco filter rod box rotating target detection model, so that the tobacco filter rod box rotating target detection model performs rotation frame positioning on the tobacco filter rod box in the image data, and obtains a positioning result; based on the positioning result, the gripper of the palletizing robot is adjusted to grasp the target tobacco filter rod box to be grasped; wherein the tobacco filter rod box rotating target detection model is trained based on the following steps: obtaining sample images of tobacco filter rod boxes in different rotation postures; performing rotation frame annotation on the sample images of tobacco filter rod boxes based on the rotation posture of the tobacco filter rod boxes, and dividing the annotated sample images of tobacco filter rod boxes into model training images. The YOLOv7 target detection model is trained by the model training set, so that the YOLOv7 target detection model learns the rotation posture and corresponding rotation frame position of the tobacco filter rod box in the model training set, and the model performance of the trained YOLOv7 target detection model is evaluated by the model verification set to obtain the model evaluation result; when the model evaluation result meets the first preset condition, the evaluated YOLOv7 target detection model is tested by the model test set to obtain the model test result; when the model test result meets the second preset condition, it is determined that the training of the YOLOv7 target detection model is completed, and the trained YOLOv7 target detection model is used as the tobacco filter rod box rotation target detection model. Compared with the prior art, when the position direction of the tobacco filter rod box placed on the pallet area and the automatic boxing conveyor in the production area is greatly offset, the palletizing robot without a camera cannot normally grasp the tobacco filter rod box. In this way, the detection of tobacco filter rod boxes in any direction can be achieved, and the stacking robot can adjust the grasping direction accordingly to ensure production safety and improve production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a schematic flow chart of the method for grabbing a tobacco filter rod box provided by the present invention.

[0020] Figure 2 It is a flow chart of a method for constructing a rotating target detection model for a tobacco filter rod box provided by the present invention.

[0021] Figure 3The three views of the structure of the fully automatic flexible stacking system for tobacco filter rod boxes provided by the present invention are shown.

[0022] Figure 4 It is a schematic diagram of the structure of the YOLOv7 target detection network model provided by the present invention.

[0023] Figure 5 It is a structural schematic diagram of the counterclockwise rotation and clockwise rotation of the x-axis in the long side representation provided by the present invention.

[0024] Figure 6 It is a structural schematic diagram of the annular smooth label provided by the present invention.

[0025] Figure 7 It is a recognition effect diagram of the tobacco filter rod box rotating target detection model provided by the present invention.

[0026] Figure 8 It is a structural schematic diagram of a grabbing device for a tobacco filter rod box provided by the present invention.

[0027] Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] To facilitate understanding of the embodiments of the present invention, specific embodiments will be further explained below in conjunction with the accompanying drawings. The embodiments do not constitute a limitation on the embodiments of the present invention.

[0030] The present invention relates to a fully automatic flexible stacking system for cigarette filter rods in the tobacco industry. The system is composed of a rectangular coordinate stacking robot, a bracket area, and an inlet and outlet conveyor of an automatic cartoning machine. The structure is as follows: Figure 3As shown. The palletizing robot is responsible for transporting the empty cartons in the bracket area to the upper conveyor in the cartoning area, and transporting the solid cartons on the lower conveyor in the cartoning area to the bracket area; the bracket area is used to store cartons, and is divided into an empty box area and a solid box area; the entrance and exit conveyor of the automatic cartoning machine consists of an upper and lower conveyor, the upper one is responsible for transporting the empty cartons to the automatic cartoning machine, and the lower conveyor is responsible for sending out the solid cartons that the automatic cartoning machine has completed filter rod boxing. The fully automatic production process of cigarette filter rods mainly includes: the AGV trolley transports the empty tobacco filter rod box to the empty box area bracket of the designated bracket area; the palletizing robot grabs the empty tobacco filter rod box vertically downward from the top and sends it to the conveyor belt of the entrance conveyor of the automatic cartoning machine to complete automatic loading; the palletizing robot grabs the full tobacco filter rod box and sends it to the solid box area bracket of the bracket area; the AGV trolley transports the full tobacco filter rod box out of the production area. During the entire production process, the direction of the tobacco filter rod box in the bracket area and on the entrance and exit conveyor of the automatic cartoning machine is fixed, and the palletizing robot without a camera performs grasping work along the direction of the flush tobacco filter rod box according to a fixed procedure.

[0031] In view of the production characteristics of the fully automatic flexible stacking system for cigarette filter rods, when the position and direction of the tobacco filter rod box in the bracket area and the automatic cartoning machine entrance and exit conveyor in the production area are greatly offset, the stacking robot without a camera will still work according to a fixed procedure, which will not be able to grab the tobacco filter rod box normally or may damage the tobacco filter rod box. The only way is to manually discover the abnormal situation in time, suspend the work of the stacking robot, adjust the position and direction of the tobacco filter rod box, and then give the stacking robot a command to start working again. Such abnormal situations have greatly affected production safety and greatly reduced production efficiency.

[0032] To solve the above problems, the present invention provides a method for constructing a rotating target detection model for a tobacco filter rod box. According to the production characteristics of the fully-automatic flexible stacking system for cigarette filter rods, the visual function is introduced into the stacking robot, the YOLOv7 target detection network model is optimized and designed, and a rotating target detection model for a tobacco filter rod box is constructed to achieve real-time detection of information such as the direction and posture of the tobacco filter rod box, thereby providing a basis for the stacking robot to smoothly grasp the tobacco filter rod box in any direction.

[0033] Figure 1 Schematic diagram of the process of grabbing the tobacco filter rod box provided by the present invention, such as Figure 1 As shown, the method includes the following: S11, obtaining image data of the target tobacco filter rod box to be currently grasped by the palletizing robot.

[0034] The palletizing robot with visual function is introduced to obtain image data of the target tobacco filter rod box to be grasped, wherein the visual function of the palletizing robot can be realized by a camera or the like.

[0035] S12, inputting the image data into a tobacco filter rod box rotating target detection model, so that the tobacco filter rod box rotating target detection model performs a rotating frame positioning on the tobacco filter rod box in the image data to obtain a positioning result.

[0036] The acquired image data is input into the above-constructed tobacco filter box rotating target detection model, so that the tobacco filter box rotating target detection model performs rotating frame positioning on the tobacco filter box in the image data to obtain the positioning result, such as Figure 7 The rotating frame positioning identification result may also include the number, position, rotating frame size and angle information of the tobacco filter rod box.

[0037] S13, adjusting the gripper of the palletizing robot to grasp the target tobacco filter rod box based on the positioning result.

[0038] The palletizing robot adjusts the direction of the gripper based on the positioning result to smoothly grasp the tobacco filter rod box.

[0039] The method for grabbing tobacco filter rod boxes provided by the present invention obtains the image data of the target tobacco filter rod box to be grabbed by the palletizing robot; inputs the image data into the tobacco filter rod box rotating target detection model, so that the tobacco filter rod box rotating target detection model performs rotation frame positioning on the tobacco filter rod box in the image data to obtain a positioning result; and adjusts the gripper of the palletizing robot to grasp the target tobacco filter rod box to be grasped based on the positioning result. Compared with the prior art, when the position direction of the tobacco filter rod box placed on the bracket area and the entrance and exit conveyor of the automatic cartoning machine in the production area is greatly offset, the palletizing robot without a camera cannot normally grasp the tobacco filter rod box. In this way, the detection of tobacco filter rod boxes in any direction can be realized, and the palletizing robot adjusts the grasping direction accordingly to complete the grasping, thereby improving production efficiency.

[0040] Figure 2 FIG. 1 is a flow chart of a method for constructing a rotating target detection model for a tobacco filter rod box provided by the present invention. Figure 2 As shown, the method includes the following: S21. Acquire sample images of tobacco filter rod boxes in different rotation postures.

[0041] Obtain sample images of tobacco filter rod boxes. The sample images of tobacco filter rod boxes are taken at the factory where the production logistics project implemented by the right holder is located, and include tobacco filter rod boxes in various postures that actually appear. Then, pre-processing operations such as randomly changing the brightness and contrast of the image and adding Gaussian noise can be performed on the original sample image data.

[0042] S22, annotating the tobacco filter rod box sample images with rotation frames based on the rotation posture of the tobacco filter rod box, and dividing the annotated tobacco filter rod box sample images into a model training set, a model verification set, and a model test set.

[0043] In the embodiment of the present invention, the long side representation method is used to define the rotation frame, and the sample image of the tobacco filter rod box is annotated with the rotation frame based on the rotation posture and definition method of the tobacco filter rod box: the long side representation method is used when defining the representation form of the rectangular rotation frame, and the rotation rectangular frame is defined as ,in, is the coordinate of the center point of the rotating rectangle. is the long side of the rotated rectangle, is the short side of the rotated rectangle, the rotation angle is the angle between the long side and the horizontal axis counterclockwise, ranging from ,like Figure 5 shown.

[0044] Specifically, in the task of rotating target detection, the rotation box is currently the most popular representation method, which can simply and intuitively obtain the rotation angle of the detected target; the specific representation forms mainly include OpenCV representation and long edge representation, both of which use The five-parameter method differs only in the definitions of the long and short sides and the angles; the present invention adopts the long side representation method to define the rotated rectangular frame.

[0045] Furthermore, the rolabelimg rotation annotation tool is used to perform rotation frame annotation on the tobacco filter rod box in the tobacco filter rod box sample image to create a label file to form a self-made experimental data set, including a model training set, a model verification set, and a model test set.

[0046] S23. Train the YOLOv7 target detection model through the model training set so that the YOLOv7 target detection model learns the rotation posture and the corresponding rotation box position of the tobacco filter rod box in the model training set, and evaluate the model performance of the trained YOLOv7 target detection model through the model verification set to obtain a model evaluation result.

[0047] Based on the model training set in the tobacco filter rod box sample image data, the network model training conditions are set to train the established YOLOv7 target detection model. When the loss function is no longer significantly reduced, the model performance of the trained YOLOv7 target detection model is evaluated through the model verification set to obtain a model evaluation result.

[0048] Specifically, the YOLOv7 target detection model is trained using the model training set, and the target rotation box regression loss function is calculated after each training. When the target rotation box regression loss function tends to be stable, the model verification set is input into the trained YOLOv7 target detection model to obtain the first predicted rotation box of the tobacco filter rod box output by the model. The circular smooth label method is used to compare the angular distance between the first predicted rotation box and the labeled rotation box of the model verification set. Based on the angular distance, the model performance of the trained YOLOv7 target detection model is evaluated to obtain a model evaluation result.

[0049] Specifically, the YOLOv7 target detection network is mainly composed of four parts: Input, Backbone, Neck and Head Prediction; Specifically, the YOLOv7 algorithm is a new generation of YOLO series algorithms proposed on the basis of the YOLOv5 algorithm to optimize the speed and accuracy of the model. It mainly consists of four parts: input, backbone network, neck and prediction head. Its network structure is as follows: Figure 4 As shown in the figure; the input part mainly adjusts the input image to the input size standard set by the backbone network; the backbone network part includes three parts: CBS convolution module, E-ELAN (Extended-ELAN) module and MPConv module. The CBS convolution module uses the statistical information between different batch data to further improve the generalization performance of the model; the E-ELAN module is a module used to improve the feature representation ability in deep neural networks, enhancing the model's attention to and modeling ability of long sequences; the MPConv module adds the MaxPool layer to form two upper and lower branches, and uses the Concat operation to fuse the extracted features to improve the generalization ability of the network; the neck part adopts the SPPCSPC structure, and according to the pixel information in the input Feature Map, the target category, position, size and other information are calculated through the convolution operation, and feature extraction is performed; the prediction head part processes the feature maps extracted by the backbone network and the neck part, and then combines all the prediction results into a target box to obtain the final predicted target.

[0050] Based on the YOLOv7 target detection network, the PIoU loss function is introduced to replace the original CIoU loss function as the target rotation box regression loss function. The PIoU loss function calculation formula is: (1)

[0051] (2)

[0052] in, is the set of all positive samples; is the number of positive samples; For the real goal; To predict the target box; is the true target after being processed by the binary constraint function and predicted target box The number of pixels of the intersection The number of pixels in the union ratio.

[0053] Specifically, the original CIoU loss function of YOLOv7 has the disadvantage of inaccurate positioning when encountering targets with large aspect ratios and dense features, which greatly affects the regression efficiency of target recognition. In order to solve the above problems and make the original processing annotation box more compact to select the target, the Pixels-IoU (PIoU) function is introduced. This loss function adds a rotation parameter and calculates the target IoU by counting pixels. This makes the loss value sensitive to the size, position and rotation of the processing annotation box, and can more accurately calculate the target intersection over union (IoU), improving the regression efficiency of target recognition.

[0054] Furthermore, when the target rotation box regression loss function tends to be stable and no longer decreases significantly, the model verification set is input into the trained YOLOv7 target detection model, and the model verification set is predicted to obtain the first predicted rotation box of the tobacco filter rod box output by the model.

[0055] Furthermore, the annular smooth label method transforms the angle regression problem into a classification problem, which can measure the angle distance between the predicted box and the true box. The formula is: (2)

[0056] (4)

[0057] in, is the angle in the target label; is the angle of the current bounding box; is the radius of the window function; is the Gaussian window function; , , All are constant terms.

[0058] Specifically, the circular smooth label method can be used to classify the angles into fixed categories, such as Figure 6 As shown, one degree is classified into one category, and the tilt angle A total of 180 categories are divided into -90~89. Discrete periodicity is introduced to avoid the problem of angle periodicity. A window function is set to smooth the tilt angle of the prediction box and the target box, and the angular distance between the prediction box and the true box (labeled rotation box) is calculated. The closer the predicted value is to the true value, the smaller the loss value.

[0059] Based on the YOLOv7 target detection network, the fusion of long edge representation and circular smooth labeling method is conducive to solving the boundary problem that may be caused by rotating rectangular boxes during network training.

[0060] Furthermore, the model performance of the trained YOLOv7 target detection model is evaluated based on the angular distance to obtain a model evaluation result. When the angular distance is less than a preset threshold, it is determined that the model evaluation result meets the first preset condition; optionally, when the angular distance is greater than the preset threshold, the weight parameter of the YOLOv7 target detection model is adjusted, and the YOLOv7 target detection model is retrained using the model training set.

[0061] S24. When the model evaluation result meets the first preset condition, the evaluated YOLOv7 target detection model is tested by the model test set to obtain a model test result.

[0062] The circular smooth label method is used to compare the angular distance between the predicted rotation box output by the model and the annotated rotation box of the model verification set. When the angular distance is less than a preset threshold, it is determined that the model evaluation result meets the first preset condition.

[0063] Furthermore, the evaluated YOLOv7 target detection model is tested through the model test set to obtain the model test results.

[0064] S5. When the model test result meets the second preset condition, it is determined that the training of the YOLOv7 target detection model is completed, and the trained YOLOv7 target detection model is used as a tobacco filter rod box rotation target detection model.

[0065] When the evaluated YOLOv7 target detection model is tested on the test set and the model performance reaches the expectation (satisfying the second preset condition), it can be determined that the training of the YOLOv7 target detection model is completed, and the trained YOLOv7 target detection model is used as the tobacco filter rod box rotation target detection model.

[0066] Specifically, the model test set is input into the evaluated YOLOv7 target detection model to obtain the second predicted rotation frame of the tobacco filter rod box output by the model. When the angle distance between the second predicted rotation frame and the labeled rotation frame of the model test set is less than a preset threshold, it is determined that the YOLOv7 target detection model training is completed, and the trained YOLOv7 target detection model is used as the tobacco filter rod box rotation target detection model.

[0067] Specifically, the model test set is input into the evaluated YOLOv7 target detection model to obtain the second predicted rotation frame of the tobacco filter rod box output by the model. The angular distance between the second predicted rotation frame and the real labeled rotation frame of the model test set is determined. When the angular distance is less than a preset threshold, it is characterized that the performance of the model in the test set meets expectations, and it is determined that the training of the YOLOv7 target detection model is completed. After the training is completed, the optimal weight parameters are saved, and the trained YOLOv7 target detection model is used as the tobacco filter rod box rotation target detection model.

[0068] Based on the YOLOv7 target detection network, a tobacco filter rod box rotating target detection model was constructed by defining the representation of a rectangular rotation box, combining annular smooth labels, and introducing the PIoU loss function as the target rotation box regression loss function. Its recognition performance and bounding box compatibility performance were adapted to the detection requirements of tobacco filter rod boxes.

[0069] The method for constructing a tobacco filter rod box rotation target detection model provided by the present invention comprises the following steps: obtaining tobacco filter rod box sample images with different rotation postures; annotating the tobacco filter rod box sample images with rotation frames based on the rotation postures of the tobacco filter rod boxes, and dividing the annotated tobacco filter rod box sample images into a model training set, a model verification set, and a model test set; training a YOLOv7 target detection model with the model training set so that the YOLOv7 target detection model learns the rotation postures and corresponding rotation frame positions of the tobacco filter rod boxes in the model training set, and evaluating the model performance of the trained YOLOv7 target detection model with the model verification set to obtain a model evaluation result; when the model evaluation result meets a first preset condition, testing the evaluated YOLOv7 target detection model with the model test set to obtain a model test result; and when the model test result meets a second preset condition, determining that the training of the YOLOv7 target detection model is completed, and using the trained YOLOv7 target detection model as a tobacco filter rod box rotation target detection model. Compared with the existing technology, when the position and direction of the tobacco filter rod box in the production area are greatly offset between the bracket area and the entrance and exit conveyor of the automatic cartoning machine, the palletizing robot without a camera cannot normally grasp the tobacco filter rod box. Therefore, the detection of tobacco filter rod boxes in any direction is realized, and the palletizing robot adjusts the grasping direction accordingly, ensuring production safety and improving production efficiency.

[0070] The following is a description of the grabbing device for the tobacco filter rod box provided by the present invention. The grabbing device for the tobacco filter rod box described below and the grabbing method for the tobacco filter rod box described above can be referred to each other.

[0071] Figure 8 It is a structural schematic diagram of the grabbing device of the tobacco filter rod box provided by the present invention, which specifically comprises: An acquisition module 801 is used to acquire image data of a target tobacco filter rod box to be currently grasped by the palletizing robot; A positioning module 802 is used to input the image data into a tobacco filter plug box rotating target detection model so that the tobacco filter plug box rotating target detection model performs a rotating frame positioning on the tobacco filter plug box in the image data to obtain a positioning result; A grasping module 803, for adjusting the grasper of the palletizing robot to grasp the target tobacco filter rod box to be grasped based on the positioning result; The model training module 804 is used to obtain sample images of tobacco filter rod boxes with different rotation postures; annotate the sample images of tobacco filter rod boxes with rotation boxes based on the rotation posture of the tobacco filter rod boxes, and divide the annotated sample images of tobacco filter rod boxes into a model training set, a model verification set and a model test set; train the YOLOv7 target detection model through the model training set so that the YOLOv7 target detection model learns the rotation posture and corresponding rotation box position of the tobacco filter rod boxes in the model training set, and evaluate the model performance of the trained YOLOv7 target detection model through the model verification set to obtain a model evaluation result; when the model evaluation result meets the first preset condition, test the evaluated YOLOv7 target detection model through the model test set to obtain a model test result; when the model test result meets the second preset condition, determine that the training of the YOLOv7 target detection model is completed, and use the trained YOLOv7 target detection model as the tobacco filter rod box rotation target detection model.

[0072] Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute a method for grasping a tobacco filter rod box, the method comprising: obtaining image data of a target tobacco filter rod box to be grasped by a palletizing robot; inputting the image data into a tobacco filter rod box rotating target detection model so that the tobacco filter rod box rotating target detection model performs a rotation frame positioning on the tobacco filter rod box in the image data to obtain a positioning result; adjusting the gripper of the palletizing robot toward the target tobacco filter rod box to be grasped based on the positioning result to grasp the target tobacco filter rod box to be grasped; wherein the tobacco filter rod box rotating target detection model is trained based on the following steps: obtaining sample images of tobacco filter rod boxes with different rotation postures; performing rotation frame annotation on the sample images of tobacco filter rod boxes based on the rotation posture of the tobacco filter rod boxes, and marking the sample images of tobacco filter rod boxes with the annotation completed The invention is divided into a model training set, a model verification set and a model test set; a YOLOv7 target detection model is trained by the model training set so that the YOLOv7 target detection model learns the rotation posture and the corresponding rotation frame position of the tobacco filter rod box in the model training set, and the model performance of the trained YOLOv7 target detection model is evaluated by the model verification set to obtain a model evaluation result; when the model evaluation result meets a first preset condition, the evaluated YOLOv7 target detection model is tested by the model test set to obtain a model test result; when the model test result meets a second preset condition, it is determined that the training of the YOLOv7 target detection model is completed, and the trained YOLOv7 target detection model is used as a tobacco filter rod box rotation target detection model.

[0073] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0074] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the tobacco filter rod box grasping method provided by the above-mentioned methods, the method including: obtaining image data of the target tobacco filter rod box to be grasped by the stacking robot; inputting the image data into a tobacco filter rod box rotating target detection model, so that the tobacco filter rod box rotating target detection model performs rotation frame positioning on the tobacco filter rod box in the image data to obtain a positioning result; based on the positioning result, adjust the gripper of the stacking robot toward the target tobacco filter rod box to be grasped; wherein, the tobacco filter rod box rotating target detection model is trained based on the following steps: obtaining sample images of tobacco filter rod boxes with different rotation postures; based on the rotation posture of the tobacco filter rod box, adjusting the gripper of the stacking robot to grasp the target tobacco filter rod box; wherein, the tobacco filter rod box rotating target detection model is trained based on the following steps: obtaining sample images of tobacco filter rod boxes with different rotation postures; The sample images of tobacco filter rod boxes are annotated with rotation boxes, and the annotated sample images of tobacco filter rod boxes are divided into a model training set, a model verification set and a model test set; the YOLOv7 target detection model is trained through the model training set so that the YOLOv7 target detection model learns the rotation posture and the corresponding rotation box position of the tobacco filter rod boxes in the model training set, and the model performance of the trained YOLOv7 target detection model is evaluated through the model verification set to obtain a model evaluation result; when the model evaluation result meets the first preset condition, the evaluated YOLOv7 target detection model is tested through the model test set to obtain a model test result; when the model test result meets the second preset condition, it is determined that the training of the YOLOv7 target detection model is completed, and the trained YOLOv7 target detection model is used as the tobacco filter rod box rotation target detection model.

[0075] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the tobacco filter rod box grasping method provided by the above-mentioned methods, the method comprising: obtaining image data of a target tobacco filter rod box to be grasped by a stacking robot; inputting the image data into a tobacco filter rod box rotating target detection model, so that the tobacco filter rod box rotating target detection model performs rotation frame positioning on the tobacco filter rod box in the image data to obtain a positioning result; based on the positioning result, adjusting the gripper of the stacking robot toward the target tobacco filter rod box to be grasped; wherein the tobacco filter rod box rotating target detection model is trained based on the following steps: obtaining sample images of tobacco filter rod boxes in different rotation postures; performing rotation frame positioning on the sample images of tobacco filter rod boxes based on the rotation posture of the tobacco filter rod boxes The YOLOv7 target detection model is trained by the model training set, so that the YOLOv7 target detection model learns the rotation posture and the corresponding rotation box position of the tobacco filter rod box in the model training set, and the model performance of the trained YOLOv7 target detection model is evaluated by the model verification set to obtain a model evaluation result; when the model evaluation result meets the first preset condition, the evaluated YOLOv7 target detection model is tested by the model test set to obtain a model test result; when the model test result meets the second preset condition, it is determined that the training of the YOLOv7 target detection model is completed, and the trained YOLOv7 target detection model is used as the tobacco filter rod box rotation target detection model.

[0076] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0077] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for grabbing a tobacco filter rod box, characterized in that: include: Acquire image data of a target tobacco filter rod box to be currently grasped by the palletizing robot; The image data is input into a tobacco filter rod box rotating target detection model, so that the tobacco filter rod box rotating target detection model performs a rotating frame positioning on the tobacco filter rod box in the image data to obtain a positioning result; Based on the positioning result, the gripper of the palletizing robot is adjusted to grasp the target tobacco filter rod box to be grasped; The tobacco filter box rotating target detection model is trained based on the following steps: Acquire tobacco filter rod box sample images in different rotation postures; Annotating the sample images of the tobacco filter rod box with a rotation frame based on the rotation posture of the tobacco filter rod box, and dividing the annotated sample images of the tobacco filter rod box into a model training set, a model verification set, and a model test set; The YOLOv7 target detection model is trained by the model training set, so that the YOLOv7 target detection model learns the rotation posture and the corresponding rotation frame position of the tobacco filter rod box in the model training set, and the model performance of the trained YOLOv7 target detection model is evaluated by the model verification set to obtain a model evaluation result; When the model evaluation result meets the first preset condition, the evaluated YOLOv7 target detection model is tested by the model test set to obtain a model test result; When the model test result meets the second preset condition, it is determined that the training of the YOLOv7 target detection model is completed, and the trained YOLOv7 target detection model is used as a tobacco filter rod box rotation target detection model.

2. The method according to claim 1, characterized in that The method of performing rotation frame annotation on the sample image of the tobacco filter rod box based on the rotation posture of the tobacco filter rod box includes: The rotation box is defined using the long side representation; The sample image of the tobacco filter rod box is annotated with a rotation frame based on the rotation posture and definition method of the tobacco filter rod box.

3. The method according to claim 2, characterized in that The YOLOv7 target detection model is trained by the model training set so that the YOLOv7 target detection model learns the rotation posture and the corresponding rotation frame position of the tobacco filter rod box in the model training set, and the model performance of the trained YOLOv7 target detection model is evaluated by the model verification set to obtain a model evaluation result, including: The YOLOv7 target detection model is trained using the model training set, and the target rotation box regression loss function is calculated after each training; When the target rotation frame regression loss function tends to be stable, the model verification set is input into the trained YOLOv7 target detection model to obtain a first predicted rotation frame of the tobacco filter rod box output by the model; Using a circular smooth label method to compare the angular distance between the first predicted rotation box and the labeled rotation box of the model verification set; The model performance of the trained YOLOv7 target detection model is evaluated based on the angular distance to obtain a model evaluation result.

4. The method according to claim 3, characterized in that When the model evaluation result meets the first preset condition, the evaluated YOLOv7 target detection model is tested by the model test set to obtain a model test result, including: When the angular distance is less than a preset threshold, determining that the model evaluation result meets a first preset condition; Inputting the model test set into the evaluated YOLOv7 target detection model to obtain a second predicted rotation frame of the tobacco filter rod box output by the model; A model test result is determined based on the second predicted rotation box and the labeled rotation box of the model test set.

5. The method according to claim 4, characterized in that When the model test result satisfies the second preset condition, determining that the YOLOv7 target detection model training is completed, and using the trained YOLOv7 target detection model as a tobacco filter rod box rotation target detection model, including: When the angular distance between the second predicted rotation box and the labeled rotation box of the model test set is less than the preset threshold, it is determined that the training of the YOLOv7 target detection model is completed, and the trained YOLOv7 target detection model is used as the tobacco filter rod box rotation target detection model.

6. The method according to claim 4, characterized in that The method further comprises: When the angular distance is greater than a preset threshold, the weight parameters of the YOLOv7 target detection model are adjusted, and the YOLOv7 target detection model is retrained using the model training set.

7. A grabbing device for a tobacco filter rod box, characterized in that: include: An acquisition module, used to acquire image data of a target tobacco filter rod box to be currently grasped by the palletizing robot; A positioning module, used for inputting the image data into a tobacco filter plug box rotating target detection model, so that the tobacco filter plug box rotating target detection model performs a rotating frame positioning on the tobacco filter plug box in the image data to obtain a positioning result; A grasping module, used for adjusting the grasper of the palletizing robot to grasp the target tobacco filter rod box to be grasped based on the positioning result; A model training module is used to obtain sample images of tobacco filter rod boxes with different rotation postures; annotate the sample images of tobacco filter rod boxes with rotation boxes based on the rotation postures of the tobacco filter rod boxes, and divide the annotated sample images of tobacco filter rod boxes into a model training set, a model verification set, and a model test set; train a YOLOv7 target detection model through the model training set so that the YOLOv7 target detection model learns the rotation postures and corresponding rotation box positions of the tobacco filter rod boxes in the model training set, and evaluate the model performance of the trained YOLOv7 target detection model through the model verification set to obtain a model evaluation result; When the model evaluation result meets the first preset condition, the evaluated YOLOv7 target detection model is tested by the model test set to obtain the model test result; when the model test result meets the second preset condition, it is determined that the training of the YOLOv7 target detection model is completed, and the trained YOLOv7 target detection model is used as the tobacco filter rod box rotation target detection model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for grabbing a tobacco filter rod box according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for grabbing a tobacco filter plug box according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for grabbing a tobacco filter plug box according to any one of claims 1 to 6 is implemented.

Images