A packaging bag opening detection method based on YOLOv8 model

Abstract

A method for detecting the bag opening of a packaging bag based on a YOLOv8 model comprises the following steps: obtaining bag opening image information of a packaging bag in a transmission process; annotating the obtained bag opening image information, constructing a bag opening data set, and dividing the data set into a training set, a validation set, and a test set; configuring a training set image enhancement algorithm to perform data enhancement on the image; training the constructed YOLOv8 rotating target detection model, and outputting the detection result of the predicted rotating frame; identifying the bag opening coordinates based on the trained YOLOv8 target detection model, and sending the obtained offset information to a control system, and the control system controls a grasping device to grasp the bag opening. The present invention can prevent the model from overfitting the training set, improve the overall generalization performance and robustness, and realize high-precision and high-speed detection of the bag opening of the packaging bag.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, specifically to the field of image detection and recognition, and in particular to a packaging bag opening detection method based on a YOLOv8 model. Background Art

[0002] In the process of automated bag opening and bagging of packaging bags (such as ton bags), in order to abolish the previous color code positioning solution and reduce the cost of ton bags, it was finally decided to use AI visual recognition technology to automatically load ton bags. However, the existing ton bags, especially aluminum foil bags, have the following defects when using visual recognition technology: on the one hand, there are many interference items of feature objects (corner indentation, edge curling, and light influence), which leads to a decrease in recognition accuracy; on the other hand, the color difference contrast of the identified object is low, resulting in the inability to accurately identify effective features when the identified object is offset by the angle of illumination of the light source. Furthermore, there is no recognition error standard. In certain circumstances (such as a single corner curling down and the degree of curling down is small), the robot arm can ignore the error. It only needs to give specific coordinates and angles to grab the target object and execute the grab task through the PLC. At this time, the recognition result of the feature object will directly determine whether to send a grab command transmission.

[0003] CN 113887395A discloses a method for detecting the position of a filter bag opening based on a deep separable convolution YOLOv4 model. The method firstly collects filter bag opening images on a conveyor belt through a camera; then performs data enhancement on all images, annotates the images, and constructs a filter bag opening data set; then constructs a YOLOv4 target detection model based on deep separable convolution, sets training parameters, and uses the filter bag opening data set to train the YOLOv4 target detection model based on deep separable convolution; finally, inputs the filter bag opening image to be detected into the trained YOLOv4 target detection model based on deep separable convolution, and outputs the filter bag opening image to be detected with the detection frame position and the filter bag opening category annotated.

[0004] However, this detection method has the following defects: (1) First, data enhancement is performed on all images, and then the images are annotated and divided into training sets, validation sets, and test sets. On the one hand, when the data set is directly enhanced, the number of data will become the enhancement factor multiplied by the number of original data sets. This method is often used in fields with very small data sets. On the other hand, since this enhancement method is to enhance the samples first and then divide the training set, the same image may be enhanced and divided into training sets, validation sets, and test sets during the data enhancement process, which will cause the model to have falsely high accuracy during the training process. In actual tests, the generalization of the model is not high. (2) The YOLOv4 model cannot accurately handle tilted or rotated objects, resulting in reduced detection accuracy.

[0005] Therefore, the present invention urgently needs to develop a more efficient bag opening detection method to automatically and accurately locate the bag opening of the packaging bag, thereby improving the automation level and production efficiency of the assembly line. Summary of the invention

[0006] The purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art and provide a packaging bag mouth detection method based on the YOLOv8 model with high generalization performance, robustness and detection accuracy.

[0007] The technical solution of the present invention is: a packaging bag opening detection method based on the YOLOv8 model, comprising the following steps:

[0008] S1: Acquire bag opening image information of the packaging bag during the conveying process;

[0009] S2: annotating the acquired bag opening image information, constructing a bag opening data set, and dividing the data set into a training set, a validation set, and a test set;

[0010] S3: Configure the training set image enhancement algorithm to perform data enhancement on the images;

[0011] S4: construct a YOLOv8 rotation target detection model, set training parameters, use the enhanced training set as training data, train the constructed YOLOv8 rotation target detection model, and calculate the classification loss, positioning loss and rotation target distribution focus loss respectively during the training process; based on the loss function calculation result, the YOLOv8 target detection model outputs the detection result of the predicted rotation box;

[0012] S5: Based on the detection result of the predicted rotation frame, obtaining the offset information of the predicted rotation frame relative to the camera calibration position;

[0013] S6: Based on the trained YOLOv8 target detection model, the coordinates of the bag opening are identified, and the obtained offset information is sent to the control system, which controls the grasping device to align with the bag opening for grasping.

[0014] Further, step S2 specifically includes the following steps:

[0015] The obtained bag opening image information is labeled and the width and height of the image are proportionally compressed;

[0016] Normalize the labeled data and scale the position coordinates of the bag opening to a specific range;

[0017] The normalized images are divided into training set, validation set and test set according to the ratio of 7:2:1.

[0018] Further, in step S3, the data enhancement method is to perform online data enhancement on the acquired packaging bag mouth training set images through an image segmentation algorithm, an image enhancement algorithm or an image fusion algorithm, including at least one of mosaic enhancement, mixed enhancement, random perturbation and color perturbation.

[0019] Further, S4 and S5 specifically include the following steps:

[0020] The parameters of the predicted rotation box are calculated through the loss function, and the coordinates of the four vertices of the predicted rotation box are obtained;

[0021] Acquire the posture information of the predicted rotating frame according to the coordinates of the four vertices of the predicted rotating frame;

[0022] According to the posture information of the predicted rotating frame and the camera calibration information, the offset information of the predicted rotating frame relative to the camera calibration position is obtained.

[0023] Furthermore, the predicted rotation box parameters calculated by the loss function are: [classid, (x1, y1), (x2, y2), (x3, y3), (x4, y4)]; wherein classid is the category number, and (x1, y1), (x2, y2), (x3, y3), and (x4, y4) are the coordinates of the four vertices of the rotation box.

[0024] Furthermore, the posture information of the predicted rotation box is obtained by the following method: according to the acquired coordinates of the four vertices of the predicted rotation box, the coordinate order of the four vertices is calculated, and the positions of two target vertices are selected to calculate the angle θ` between the two target vertices and the X-axis of the bag opening image and the center point coordinates (x`, y`) of the two target vertices, which is the posture information of the predicted rotation box.

[0025] Furthermore, the camera calibration information is obtained by:

[0026] Place a circular calibration object at the grabbing center of the bag opening;

[0027] Determine the standard camera position;

[0028] Take at least one photo of the bag opening and the circular calibration object as the first image information, and predict the standard center point coordinates (x, y) and the standard angle θ of the standard position calibration according to the rotation target detection model trained subsequently;

[0029] After controlling the camera to move the target distance s in the horizontal direction, at least one photo of the bag opening and the circular calibration object is taken as the second image information;

[0030] The center pixel coordinates of the circular calibration object are extracted from the first image information and the second image information by using the image tool, and are recorded as (x a ,y a )、(x b ,y b );

[0031] Calculate the distance / pixel ratio r = s / (x a -x b ).

[0032] Further, according to the center point coordinates (x', y'), θ', combined with the standard center point coordinates (x, y) and the standard angle θ, and the distance / pixel ratio r, the offset distance d and the offset angle angle of x' and y' are calculated, and the calculation formula is:

[0033] d=(x`-x)*r or d=(y`-y)*r;

[0034] angle = θ`-θ.

[0035] Further, S6 specifically includes the following steps:

[0036] Run the YOLOv8 target detection model, communicate with the camera and control system, and wait for the control system to send a photo signal;

[0037] When the bag opening is detected to be in place, the control system sends a photo signal;

[0038] After the YOLOv8 target detection model receives the photographing signal, it obtains the bag opening image information taken by the camera, predicts the position of the bag opening, and then returns the result to the control system based on the offset information calculated in step S5, and the control system controls the grasping device to grasp the bag opening.

[0039] Furthermore, the enhancement methods in S3 specifically include: when training the target detection model, first randomly perturb the training set images in the data set through three methods: mixed enhancement, random perturbation, and color perturbation, and then randomly select a certain number of images in the training set, randomly scale and randomly splice and mix them through mosaic enhancement, and after obtaining a new spliced ​​image, use the spliced ​​image to train the target detection model.

[0040] The principle of the present invention is: first collect the bag mouth pictures of the packaging bag, then annotate all the pictures obtained, construct the bag mouth data set, divide the data set into training set, verification set and test set, and compress the width and height of the collected and annotated pictures in equal proportion to ensure that the accuracy of the training model remains unchanged, and improve the training and reasoning speed of the model. Then configure the training set picture enhancement algorithm to perform data enhancement on the pictures; construct a YOLOv8 OBB rotating target detection model, set the training parameters, and train the constructed YOLOv8 OBB rotating target detection model based on the packaging bag mouth training set. Finally, verify the performance of the YOLOv8 OBB rotating target detection model on the test set, and verify the bag mouth classification and bag mouth positioning effects on the test set.

[0041] Among them, YOLOv8 OBB (Oriented Bounding Box) is an important extension of the YOLOv8 model in handling rotated object detection tasks. YOLOv8 is the latest version of the YOLO series model. It introduces new features and improvements to further improve performance and flexibility, especially in handling rotated object detection tasks. OBB can provide a bounding box that fits the actual shape and orientation of the target object more closely. Compared with the traditional horizontal bounding box (HBB), OBB provides higher detection accuracy. The main features and functions of YOLOv8 OBB include: Anchor-free design: YOLOv8 OBB adopts an anchor-free design, which helps reduce the model's dependence on prior boxes, thereby simplifying the model structure and training process. Rotated box target detection: By introducing OBB, YOLOv8 can identify and accurately locate rotated objects in the image, and provide a rotated bounding box (OBB) for each detected object, which can more closely fit the actual shape and orientation of the target object. Pruning and tracking: YOLOv8 OBB also supports pruning and tracking functions of the model, which helps to improve the computational efficiency and real-time performance of the model while maintaining high detection accuracy. Dataset adaptability: For different data sets, YOLOv8 OBB needs to make some parameter adjustments to adapt to specific data distribution and target characteristics. In addition, the network structure of YOLOv8OBB includes a backbone network, a Neck layer, and a Head layer, in which the Head layer specifically adds a feature map for predicting the rotation angle to meet the needs of the rotated target detection task. The design of this network structure makes YOLOv8OBB more efficient and accurate when processing rotated target detection tasks.

[0042] Therefore, YOLOv8 OBB provides a powerful tool for rotating target detection tasks through its unique design and functions. For improving the YOLOv8 OBB algorithm to improve the detection accuracy and speed of the bag mouth, the present invention enhances the model's expression ability and detail capture ability by improving the feature extraction layer and the detection layer, thereby more effectively processing and analyzing rotating objects in the image to obtain the best detection effect.

[0043] The beneficial effects of the present invention are as follows: on the one hand, increasing the diversity of training data helps prevent the model from overfitting the training set, that is, avoiding the model from being overly sensitive to specific details or noise in the training data and ignoring more general and essential features; through online data enhancement, the model is forced to learn more general feature representations, so that it can still accurately detect the target object when facing unseen test data, thereby improving the overall generalization performance and robustness; on the other hand, through the improved YOLOv8 algorithm, high-precision and high-speed detection of the bag opening of the packaging bag can be achieved, and the inclination of the bag opening can be accurately identified, thereby minimizing the consumption of manpower, material resources and financial resources, and maximizing production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a schematic structural diagram of an embodiment of the present invention;

[0045] Figure 2 It is a schematic diagram of the test results of 2000 rounds of model training in an embodiment of the present invention;

[0046] Figure 3 It is a schematic diagram of the test results of 4000 rounds of model training in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] like Figure 1 As shown: The following is a preferred implementation of the present invention for detecting the bag opening using an aluminum foil bag as an example, specifically comprising the following steps:

[0049] S101: Acquire the image information of the bag opening of the aluminum foil bag during the transmission process.

[0050] Specifically, the camera acquisition parameters, such as image sharpening, are set, and the camera is used to collect images of the openings of the aluminum foil bags on the conveyor belt, and obtain images of various scenes with the aluminum foil bags as the target, including but not limited to reflections, up and down edge warping, edge indentation, etc. The camera may be an industrial camera.

[0051] S102: Annotate the acquired bag opening image information, construct a bag opening data set, and divide the data set into a training set, a validation set, and a test set according to a target ratio.

[0052] Specifically, first, all the pictures obtained in S101 are data labeled, and the width and height of the labeled pictures are proportionally compressed. This embodiment uses a special labeling tool X-anylabeling for image labeling. After collecting and formulating target object feature standards for training, it can automatically generate labeled data. For example, this embodiment takes the bag mouth of an aluminum foil bag as the target feature, and calibrates it according to the inclination angle of the bag mouth to form bag mouth information (coordinates, categories) containing picture information and annotations. This embodiment can obtain the position coordinates and category information of the bag mouth through data labeling, so that the model can learn how to identify and locate the target, and improve the accuracy and reliability of the prediction. Through proportional compression, on the one hand, images of different sizes can be unified to the input size required by the model, ensuring that the model can process inputs of the same size during training and reasoning, reducing the demand for computing resources and reducing memory usage; on the other hand, the aspect ratio of the image can be maintained, thereby avoiding image deformation and ensuring that the shape and features of the bag mouth are not distorted.

[0053] Next, the labeled data is normalized, which helps the model better understand and process the target location in the image. The coordinate normalization of this embodiment is to scale the position coordinates of the bag opening (such as the center point coordinates, the width and height of the bounding box) to a specific range, such as between 0 and 1, to facilitate model training and reasoning. The general mathematical formula for coordinate normalization can be expressed as:

[0054] new_x=(x-x_min) / (x_max-x_min)

[0055] new_y=(y-y_min) / (y_max-y_min)

[0056] Among them, x and y are the coordinates of the position of the target (i.e. the bag opening) in the image; x_min and y_min are the minimum values ​​of the x and y coordinates in the image or relevant area (usually 0, but if the image is cropped or translated, it may not be), and x_max and y_max are the maximum values ​​of the x and y coordinates (for the entire image, this will be the width and height of the image).

[0057] Finally, the training set, validation set, and test set are divided into 7:2:1. Setting this ratio has the following advantages:

[0058] 70% of the training set can provide enough samples to enable the model to fully learn, thereby improving the generalization ability of the model; 20% of the validation set can be used to tune the hyperparameters of the model, help avoid overfitting, and ensure that the model performs well on unseen data; 10% of the test set provides an independent evaluation standard for the final verification of the performance of the model, which ensures the fairness of the model evaluation because the test set is not used in the training and verification process. If the proportion of the validation set is reduced to 1%, it is easy to be insufficient to effectively evaluate the performance of the model, especially in the case of uneven sample distribution, which may lead to instability in the evaluation results; and if the proportion of the training set is increased, although the training set is larger, it is easy to lead to instability in the evaluation of too small validation and test sets. Therefore, this embodiment is divided into training set, validation set and test set according to 7:2:1, which balances the use of training, validation and test data, avoids data waste, and ensures that each part has enough samples for effective analysis.

[0059] S103: Configure a training set image enhancement algorithm to perform data enhancement on the image.

[0060] Specifically, since the prior art directly enhances the data set and then performs annotation and data set division, this belongs to offline enhancement, while this embodiment uses an image segmentation algorithm, an image enhancement algorithm or an image fusion algorithm to randomly enhance or fuse the acquired aluminum foil bag opening training set images to perform data online enhancement on the images. The enhancement methods mainly include: mosaic enhancement, mixup, random perspective and color perturbation (HSV augment), etc.

[0061] When training the model, after obtaining the training set data, a series of random transformations (such as flipping, cropping, rotating, scaling, color adjustment, translation, etc.) are applied to the training set data to randomly generate a large number of new samples with different viewpoints, sizes, lighting conditions and backgrounds. These transformations simulate the various forms and environmental changes that objects may appear in the real world, so that the model can be exposed to more diverse scenes during the training stage, thereby improving the model's adaptability to different situations, increasing the diversity of training data, and improving the model's generalization ability.

[0062] In order to effectively improve the robustness of the model, when training the target detection model, this embodiment first randomly perturbs the training set images in the data set through mixed enhancement (Mixup), random perturbation (random perspective) and color perturbation (HSV augment), and then randomly selects 4 images from the training set, and randomly scales and randomly splices and mixes them through the mosaic enhancement (Mosaic) data enhancement method. After obtaining a new spliced ​​image, the spliced ​​image is used to train the target detection model.

[0063] The data enhancement method of this embodiment has the following advantages:

[0064] (1) Improving the generalization ability of the model: Increasing the diversity of training data helps prevent the model from overfitting the training set, that is, preventing the model from being overly sensitive to specific details or noise in the training data and ignoring more general and essential features. Through online data augmentation, the model is forced to learn more general feature representations, so that it can still accurately detect the target object when faced with unseen test data, thereby improving the overall generalization performance.

[0065] (2) Enhance the robustness of the model: The noise and changes introduced by online data augmentation help the model cope with various potential interference factors during training. For example, adding slight blur, brightness changes, or color distortion can make the model more resistant to changes in image quality, fluctuations in lighting conditions, and camera imaging deviations in practical applications. In this way, the model can stably detect objects even in complex or non-ideal environments.

[0066] (3) Reduce the reliance on large amounts of labeled data: In practical applications, it is costly and time-consuming to obtain large-scale, high-quality labeled object detection datasets. Online data augmentation can compensate for the problem of insufficient data to a certain extent by effectively using existing data to generate new training samples, making full use of limited data resources and reducing the rigid demand for large-scale labeled datasets.

[0067] S104: construct a YOLOv8 rotated target detection model, set training parameters, use the enhanced training set as training data, train the constructed YOLOv8 rotated target detection model, and calculate the classification loss (C1s Loss), positioning loss (Bbox Loss) and rotated target distribution focus loss (RDFL Loss) during the training process; based on the loss function calculation result, the YOLOv8 target detection model outputs the detection result of the predicted rotation box.

[0068] Specifically, the model constructed in this embodiment is a rotating target detection model based on YOLOv8 OBB. After setting the training parameters, the constructed YOLOv8 OBB rotating target detection model is trained based on the aluminum foil bag mouth training set. The classification loss is used to evaluate the classification accuracy, the positioning loss is used to evaluate the positioning accuracy of the rotating frame, and the rotating target distribution focus loss focuses on the detection of rotating targets. The predicted rotating frame parameters calculated by the loss function in this embodiment are: [classid, (x1, y1), (x2, y2), (x3, y3), (x4, y4)]. Among them, classid is the category number, (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the coordinates of the four vertices of the rotating frame.

[0069] After obtaining the coordinates of the four vertices, it is necessary to define the order of the four vertices (e.g., clockwise or counterclockwise), and confirm the rotation angle at the same time to reflect the true direction of the object, so when detecting and grasping the bag mouth of the aluminum foil bag, it is essential to ensure the consistency of the coordinate order and the angle definition for the accuracy of the algorithm. In the present embodiment, the angle definition method adopted by the rotating target detection model of YOLOv8 OBB is preferably: with the positive direction (right) of the X-axis as the zero degree direction, the counterclockwise rotation of the X-axis is a positive angle, and the clockwise rotation is a negative angle, and the value range of the angle is [-90, 90]. According to the coordinates of the four vertices returned by the YOLOv8 target detection model, the coordinate order of the four vertices (upper left corner, lower left corner, lower right corner, upper right corner) is calculated, and then the angle θ' of the two vertices and the image X-axis and the center point coordinates (x', y') of the two coordinate points are calculated according to the vertex positions of the upper left corner and the upper right corner or the lower left corner and the lower right corner, which is the posture information of the predicted rotation frame.

[0070] Among them, the aforementioned positive direction of the X-axis is the zero-degree direction, which means that the selection of the vertex order needs to be consistent with this reference direction. For example, when selecting the upper left corner and the upper right corner to calculate the angle, it is necessary to ensure that the relationship between these vertices and the X-axis is clear. When calculating the angle θ`, the order and reference direction determine whether the calculation result is within the range of [-90, 90]. If the vertex order is improper, the calculation result may exceed this range, which requires correction. This embodiment provides a clear framework and direction by defining the coordinate order and angle, which helps to better image processing and object tracking. By calculating the angle θ`, the degree of inclination or direction of the rotating box can be determined; by calculating the coordinates of the center point, a reference position is provided, which is convenient for comparison and calculation between different coordinate systems.

[0071] S105: Based on the detection result of the predicted rotation frame, obtain the offset information of the predicted rotation frame relative to the camera calibration position.

[0072] Specifically, the camera calibration of this embodiment uses a flat calibration object for calibration, and the calibration method is:

[0073] A. Place a circular calibration object at the grab center of the aluminum foil bag opening;

[0074] B. Determine the standard camera position (the camera moves with the robotic arm);

[0075] C. Take a photo of the bag opening and the circular calibration object as the first picture. At this time, the standard center point coordinates (x, y) and the standard angle θ of the standard position calibration can be predicted based on the rotation target detection model trained later.

[0076] After controlling the robot arm to drive the camera to move 100 mm in the horizontal direction, take a photo of the opening of the aluminum foil bag and the circular calibration object as the second picture;

[0077] The center pixel coordinates of the circular calibration object are extracted from the first and second images through the image tool and recorded as (x a ,y a )、(x b ,y b );

[0078] Calculate the distance / pixel ratio r = 100 / (x a -x b ).

[0079] Next, according to the center point coordinates (x`, y`) and θ` calculated in step S104, combined with the standard center point coordinates (x, y) and the standard angle θ, and the distance / pixel ratio r, the offset distance d and the offset angle angle of x` and y` are calculated, and the calculation formula is:

[0080] d=(x`-x)*r or d=(y`-y)*r;

[0081] angle = θ`-θ.

[0082] S106: Identify the bag opening coordinates based on the trained YOLOv8 target detection model, and send the acquired offset information to the control system, which controls the grasping device to grasp the bag opening. Specifically, the following steps are included:

[0083] (1) Run the YOLOv8 target detection model, connect the camera and the PLC controller of the automatic packaging machine via TCP communication, and wait for the PLC controller to send a photo signal;

[0084] (2) When the aluminum foil bag opening is detected to be in place, the PLC controller sends a photo-taking signal;

[0085] (3) After the YOLOv8 target detection model receives the camera signal, it obtains the camera image through TCP communication, predicts the position of the bag opening, and then returns the result to the PLC controller based on the offset calculated in step S105. The PLC controller controls the grasping device to grasp the bag opening.

[0086] Figure 2 and Figure 3 This is the model training test result obtained by the present invention according to the above detection method.

[0087] in, Figure 2 This is the test result of 2000 rounds of model training. The total number of test images in the first round is 223. The model has been inferred for 2000 rounds. The test results are as follows:

[0088] Target without interference: 54 images in total, recognition error rate: 100%;

[0089] The target object has warped edges: 78 images in total, recognition error rate: 96.2%;

[0090] The target object has wrinkles: 43 images, recognition error rate: 97.3%;

[0091] Target has backlight: 18 images are counted; recognition error rate: 100%.

[0092] Figure 3 The following are the test results of 4000 rounds of model training. The total number of test images in the first round is 338. The model has been inferred for 4000 rounds. The test results are as follows:

[0093] Target without interference: 113 images in total, recognition error rate: 100%;

[0094] The target object has warped edges: 96 images in total, recognition error rate: 100%;

[0095] The target object has wrinkles: 82 images, recognition error rate: 99%;

[0096] Target has backlight: 22 images are counted, recognition error rate: 100%.

[0097] Judging from the overall test results, except for the severe wrinkles of the aluminum foil bag that will affect the recognition accuracy, the YOLOv8 target detection model can accurately identify and infer the corresponding angles and coordinates in other cases. The current accuracy rate can reach 99.6%, which has met the recognition requirements of the fully automatic packaging machine.

[0098] In summary, on the one hand, the present invention helps prevent the model from overfitting the training set by increasing the diversity of training data, that is, avoiding the model from being overly sensitive to specific details or noise in the training data and ignoring more general and essential features; through online data enhancement, the model is forced to learn more general feature representations, so that it can still accurately detect the target object when facing unseen test data, thereby improving the overall generalization performance and robustness; on the other hand, through the improved YOLOv8 algorithm, high-precision and high-speed detection of the bag opening of the packaging bag can be achieved, and the inclination of the bag opening can be accurately identified, which can minimize the consumption of manpower, material resources and financial resources, and maximize production efficiency.

Claims

1. A packaging bag opening detection method based on the YOLOv8 model, characterized in that: The following steps are involved: S1: Acquire bag opening image information of the packaging bag during the conveying process; S2: annotating the acquired bag opening image information, constructing a bag opening data set, and dividing the data set into a training set, a validation set, and a test set; S3: Configure the training set image enhancement algorithm to perform data enhancement on the images; S4: construct a YOLOv8 rotation target detection model, set training parameters, use the enhanced training set as training data, train the constructed YOLOv8 rotation target detection model, and calculate the classification loss, positioning loss and rotation target distribution focus loss respectively during the training process; based on the loss function calculation result, the YOLOv8 target detection model outputs the detection result of the predicted rotation box; S5: Based on the detection result of the predicted rotation frame, obtaining the offset information of the predicted rotation frame relative to the camera calibration position; S6: Based on the trained YOLOv8 target detection model, the coordinates of the bag opening are identified, and the obtained offset information is sent to the control system, which controls the grasping device to align with the bag opening for grasping.

2. The packaging bag opening detection method based on the YOLOv8 model according to claim 1 is characterized in that: Step S2 specifically includes the following steps: The obtained bag opening image information is labeled and the width and height of the image are proportionally compressed; Normalize the labeled data and scale the position coordinates of the bag opening to a specific range; The normalized images are divided into training set, validation set and test set according to the ratio of 7:2:

1.

3. The packaging bag opening detection method based on the YOLOv8 model according to claim 1 is characterized in that: In step S3, the data enhancement method is to perform online data enhancement on the acquired packaging bag mouth training set images through an image segmentation algorithm, an image enhancement algorithm or an image fusion algorithm, including at least one of mosaic enhancement, mixed enhancement, random perturbation and color perturbation.

4. The packaging bag opening detection method based on the YOLOv8 model according to claim 1 is characterized in that: S4 and S5 specifically include the following steps: The parameters of the predicted rotation box are calculated through the loss function, and the coordinates of the four vertices of the predicted rotation box are obtained; Acquire the posture information of the predicted rotating frame according to the coordinates of the four vertices of the predicted rotating frame; According to the posture information of the predicted rotating frame and the camera calibration information, the offset information of the predicted rotating frame relative to the camera calibration position is obtained.

5. The packaging bag opening detection method based on the YOLOv8 model according to claim 4 is characterized in that: The predicted rotation box parameters calculated by the loss function are: [classid, (x1, y1), (x2, y2), (x3, y3), (x4, y4)]; where classid is the category number, (x1, y1), (x2, y2), (x3, y3), and (x4, y4) are the coordinates of the four vertices of the rotation box.

6. The packaging bag opening detection method based on the YOLOv8 model according to claim 4 or 5, characterized in that: The posture information of the predicted rotation frame is obtained by the following method: according to the acquired coordinates of the four vertices of the predicted rotation frame, the coordinate order of the four vertices is calculated, and the positions of two target vertices are selected to calculate the angle θ` between the two target vertices and the X-axis of the bag opening image and the center point coordinates (x`, y`) of the two target vertices, which is the posture information of the predicted rotation frame.

7. The packaging bag opening detection method based on the YOLOv8 model according to claim 6 is characterized in that: The camera calibration information is obtained in the following way: Place a circular calibration object at the grabbing center of the bag opening; Determine the standard camera position; Take at least one photo of the bag opening and the circular calibration object as the first image information, and predict the standard center point coordinates (x, y) and the standard angle θ of the standard position calibration according to the rotation target detection model trained subsequently; control the camera to move the target distance s in the horizontal direction, and then take at least one photo of the bag opening and the circular calibration object as the second image information; The center pixel coordinates of the circular calibration object are extracted from the first image information and the second image information by using the image tool, and are recorded as (x a ,y a )、(x b ,y b ); Calculate the distance / pixel ratio r = s / (x a -x b ).

8. The packaging bag opening detection method based on the YOLOv8 model according to claim 7 is characterized in that: According to the center point coordinates (x`, y`), θ`, combined with the standard center point coordinates (x, y) and the standard angle θ, and the distance / pixel ratio r, the offset distance d and the offset angle angle of x` and y` are calculated. The calculation formula is: d=(x`-x)*r or d=(y`-y)*r; angle = θ`-θ.

9. The packaging bag opening detection method based on the YOLOv8 model according to claim 1, characterized in that: S6 specifically includes the following steps: Run the YOLOv8 target detection model, communicate with the camera and control system, and wait for the control system to send a photo signal; When the bag opening is detected to be in place, the control system sends a photo signal; After the YOLOv8 target detection model receives the photographing signal, it obtains the bag opening image information taken by the camera, predicts the position of the bag opening, and then returns the result to the control system based on the offset information calculated in step S5, and the control system controls the grasping device to grasp the bag opening.

10. The packaging bag opening detection method based on the YOLOv8 model according to claim 3 is characterized in that: The enhancement methods in S3 specifically include: when training the target detection model, first randomly perturb the training set images in the data set through three methods: mixed enhancement, random perturbation, and color perturbation. Then, a certain number of images are randomly selected from the training set, and they are randomly scaled and randomly spliced ​​and mixed through mosaic enhancement. After obtaining a new spliced ​​image, the spliced ​​image is used to train the target detection model.

Images