Vision-based wine bottle feeding angle correction method and system

Abstract

The invention discloses a method and a system for correcting a feeding angle of a wine bottle based on vision, belongs to the technical field of industrial image processing, and solves the problem that the recognition precision of a silk-screen defect detector is influenced due to difficulty in clearly and completely imaging printed contents caused by uncertainty of the feeding angle of the wine bottle in the prior art. The method comprises the following steps: acquiring a bottom image of a wine bottle in a standard feeding direction, and acquiring a standard feeding direction angle of the wine bottle based on the bottom image; bottom images of wine bottles in any feeding direction are collected to construct a data set, and after the data set is marked, a direction discrimination mark area rotation rectangular positioning model is trained; obtaining a bottom image of the to-be-analyzed wine bottle, and calculating a feeding direction angle of the to-be-analyzed wine bottle based on the bottom image of the to-be-analyzed wine bottle; calculating an actual correction angle according to the standard feeding direction angle of the wine bottle and the feeding direction angle of the to-be-analyzed wine bottle; and correcting the entrance angle of the to-be-analyzed wine bottle based on the actual correction angle. The device is used for correcting the feeding angle of the wine bottle.

Description

Technical Field

[0001] A method and system for correcting the feeding angle of wine bottles based on vision, which are used for correcting the feeding angle of wine bottles and belong to the technical field of industrial image processing. Background Art

[0002] Industrial wine bottle printing defect detection is an important part in wine bottle production and quality control. Manual defect detection has problems such as slow detection speed, low accuracy affected by fatigue, concentration and other aspects, and large subjective factors. At present, vision-based wine bottle printing defect detection machines are gradually replacing manual detection. This vision-based intelligent recognition and detection method can not only ensure the detection accuracy but also improve the detection efficiency.

[0003] The clear and complete imaging of the printed graphic area on the wine bottle is the premise for the normal and effective operation of the wine bottle silk screen defect detection machine, that is, the printed area to be detected of the wine bottle needs to be located in the camera shooting field of view. In the actual application process, after the wine bottle printing process is completed, whether it is grabbed manually or by a robotic arm and placed on the conveyor belt and sent into the defect detection machine, it is difficult to ensure the accuracy and consistency of the wine bottle feeding angle, and thus it is difficult to ensure that the printed content is clearly and completely in the camera field of view. Therefore, the correction of the wine bottle feeding angle is a very important precondition in the wine bottle printing defect detection task.

[0004] For some bottle shapes, a specific guiding mechanism can be designed to correct the feeding direction, but for round bottles or bottles with completely symmetrical multi-faces, a corresponding guiding mechanism cannot be designed to correct the feeding direction. Although for round bottles, using a line scan camera for imaging does not need to care about the feeding angle problem, this solution has the following problems in actual application:

[0005] (1) There is a situation where part of the printed graphics are printed on the bottleneck of the round wine bottle, and the line scan camera alone cannot completely and clearly image this area;

[0006] (2) The time taken by the line scan camera to collect images is much longer than that of the area array camera, so it cannot meet the requirements of the actual production beat.

[0007] In actual application, through-type silk screen defect detection machines are mostly used, and the correction of the feeding angle is a necessary precondition.

[0008] To sum up, the current silk screen defect detection machines have the following technical problems in accommodating various bottle shapes:

[0009] The silk screen defect detection machine is difficult to ensure the accuracy and consistency of the feeding angles of round and multi-face symmetric wine bottles, and thus it is difficult to ensure that the printed content is clearly and completely in the camera field of view, that is, the uncertainty of the wine bottle feeding angle leads to the difficulty of clearly and completely imaging the printed content, which in turn affects the recognition accuracy of the silk screen defect detection machine. Summary of the Invention

[0010] For the problems in the above research, the purpose of the present invention is to provide a method and system for correcting the feeding angle of wine bottles based on vision, so as to solve the problem that the uncertainty of the feeding angle of wine bottles in the prior art makes it difficult to clearly and completely image the printed content, thereby affecting the recognition accuracy of the screen printing defect detector.

[0011] To achieve the above purpose, the present invention adopts the following technical solutions:

[0012] A method for correcting the feeding angle of wine bottles based on vision, comprising the following steps:

[0013] Step 1: Obtain the bottom image of the wine bottle in the standard feeding direction, and obtain the standard feeding direction angle of the wine bottle based on this bottom image;

[0014] Step 2: Collect the bottom images of wine bottles in any feeding direction to construct a data set, and after annotating the data set, train a direction discrimination flag region rotated rectangle positioning model;

[0015] Step 3: Obtain the bottom image of the wine bottle to be analyzed, and calculate the feeding direction angle of the wine bottle to be analyzed based on the bottom image of the wine bottle to be analyzed;

[0016] Step 4: Calculate the actual correction angle according to the standard feeding direction angle of the wine bottle and the feeding direction angle of the wine bottle to be analyzed;

[0017] Step 5: Correct the incoming angle of the wine bottle to be analyzed based on the actual correction angle.

[0018] Furthermore, the specific steps of step 1 are as follows:

[0019] Step 1.1: Obtain the bottom image of the wine bottle in the standard feeding direction:

[0020] Step 1.2: Obtain the pixel coordinates of the center point of the wine bottle bottom based on the bottom image of the wine bottle in the standard feeding direction. The specific steps are as follows:

[0021] First, perform mean filtering processing on the bottom image of the wine bottle in the standard feeding direction;

[0022] Next, perform image enhancement preprocessing on the result of the mean filtering processing;

[0023] Next, perform image segmentation operation on the result of the image enhancement preprocessing to obtain a binary image. Among them, for the imaging environment with relatively good conditions, direct threshold segmentation is used, and for the imaging with relatively poor quality, a deep learning network is used. The imaging environment with relatively good conditions means that there is no reflection or noise after imaging the bottom image of the wine bottle in the standard feeding direction, and the imaging with relatively poor quality means that there is reflection or noise after imaging the bottom image of the wine bottle in the standard feeding direction;

[0024] Use the contour finding function findContours supported by the OpenCV library to find the contours of the binary image and calculate the contour area. Use the calculated contour area to filter out and delete the reflective interference smaller than the given threshold. After deletion, the remaining is the bottom contour of the wine bottle in the standard feeding direction;

[0025] Finally, use the minAreaRect function in OpenCV to locate the minimum bounding rectangle of the bottom contour of the wine bottle, and obtain the center point coordinates of the minimum bounding rectangle of the bottom contour of the wine bottle as the pixel coordinates of the bottom center point of the wine bottle in the standard feeding direction;

[0026] Step 1.3: Based on the bottom image of the wine bottle in the standard feeding direction, determine the boundary of the direction discrimination flag area through manual annotation, and obtain the center point pixel coordinates of this area. Among them, the boundary of the direction discrimination flag area is a rotated rectangle frame composed of four vertex coordinates. After annotation, the rotated rectangle frame (class_idx, x1, y1, x2, y2, x3, y3, x4, y4) and the center point pixel coordinates of the rotated rectangle = ((x1 + x2 + x3 + x4) / 4, (y1 + y2 + y3 + y4) / 4) can be obtained. class_idx represents the category id, and (x1, y1), (x2, y2), (x3, y3), (x4, y4) respectively represent the four vertex coordinates of the rotated rectangle frame;

[0027] Step 1.4: Based on the center point pixel coordinates of the boundary of the direction discrimination flag area in the bottom image of the wine bottle in the standard feeding direction, point to the center point pixel coordinates of the bottom of the wine bottle in the standard feeding direction to form a direction vector, and calculate the angle between this direction vector and the horizontal x-axis as the standard feeding direction angle of the wine bottle. The formula is:

[0028]

[0029] In the formula, is the direction vector, which is calculated through the center point pixel coordinates flag_o(fcx, fcy) of the direction discrimination flag area in the bottom image of the wine bottle in the standard feeding direction and the center point pixel coordinates bottle_o(bcx, bcy) of the bottom of the wine bottle in the standard feeding direction. A negative sign is added in front of y because the change of the image coordinate and the Cartesian coordinate system in the y-axis direction is opposite. θ(y, x) represents the angle between this direction vector and the horizontal x-axis, that is, the standard feeding direction angle of the wine bottle. The formula compensates the calculated angle by judging the quadrant of the positive and negative signs of x and y, so that the angle range is [-180, 180].

[0030] Furthermore, the rotated rectangle positioning model of the direction discrimination flag area in step 2 is the yolo11-obb model, which includes a feature extraction module, a feature fusion and enhancement module, and a task head connected in sequence.

[0031] Furthermore, the specific steps of Step 2 are as follows:

[0032] Collect the bottom images of wine bottles in any feeding direction to construct a dataset;

[0033] Use the rotating rectangle annotation tool X-AnyLabeling to annotate the boundaries of the direction discriminant marker regions for the bottom images of wine bottles in the dataset;

[0034] Based on the annotated dataset, train a rotating rectangle positioning model for the direction discriminant marker region to obtain a trained rotating rectangle positioning model for the direction discriminant marker region.

[0035] Furthermore, the specific steps of Step 3 are as follows:

[0036] Step 3.1: Use the trained rotating rectangle detection model for the direction discriminant marker region to perform positioning and recognition of the direction discriminant marker region on the obtained bottom image of the wine bottle to be analyzed, and then post-process the recognition result to obtain the final rotating rectangle box. Among them, the final rotating rectangle box is the smallest circumscribed rectangle of the boundary of the direction discriminant marker region. The pixel coordinates of the center point output by this smallest circumscribed rectangle are the pixel coordinates of the center point of the direction discriminant annotation region. Post-processing means that when the number of recognition results is greater than 1, perform confidence sorting on the boundaries of all direction discriminant marker regions, and output the one with the highest confidence probability as the final rotating rectangle box; when the number of recognition results is 0, directly feedback that the input image is abnormal and no further operation steps are performed; otherwise, directly obtain the final rotating rectangle box;

[0037] Step 3.2: Calculate the pixel coordinates of the center point of the bottom of the wine bottle to be analyzed;

[0038] Step 3.3: Use the direction vector from the pixel coordinates flag_o1(fcx1, fcy1) of the center point of the direction discriminant marker region to the pixel coordinates bottle_o1(bcx1, bcy1) of the center point of the bottom of the wine bottle to be analyzed to calculate the angle with the horizontal x-axis as the feeding direction angle of the wine bottle to be analyzed.

[0039] Furthermore, the specific steps of Step 4 are as follows:

[0040] First, calculate the difference between the feeding direction angle of the wine bottle to be analyzed and the standard feeding direction angle of the wine bottle to obtain the accurate angle for adjusting the incoming angle of the wine bottle to be analyzed to the desired incoming angle. The calculation formula is:

[0041] angle = theta2 - theta1

[0042] Among them, angle represents the accurate angle at which the incoming angle of the wine bottle to be analyzed is adjusted to the desired incoming angle, that is, the corrected angle is obtained. theta2 represents the incoming material direction angle of the wine bottle to be analyzed, and theta1 represents the standard incoming material direction angle of the wine bottle.

[0043] Perform a normalization operation on the corrected angle to normalize the corrected angle to the range of [-180, 180]. Finally, output the actual corrected angle angle_norm. The formula is:

[0044]

[0045] Furthermore, the specific steps of step 5 are as follows:

[0046] Send the actual corrected angle to the plc side. If the actual corrected angle is less than 0, rotate the incoming angle of the wine bottle to be analyzed clockwise based on the actual corrected angle for correction, that is, rotate clockwise by -angle_norm degrees. If the actual corrected angle is greater than 0, rotate the incoming angle of the wine bottle to be analyzed counterclockwise based on the actual corrected angle for correction, that is, rotate counterclockwise by angle_norm degrees.

[0047] A system for correcting the incoming material angle of a wine bottle based on vision, comprising:

[0048] Module for obtaining the standard incoming material direction angle of the wine bottle: Obtain the bottom image of the wine bottle with the standard incoming material direction, and obtain the standard incoming material direction angle of the wine bottle based on this bottom image;

[0049] Model training module: Collect the bottom images of wine bottles with arbitrary incoming material directions to construct a dataset, and after annotating this dataset, train the rotation rectangle positioning model for the direction discrimination flag area;

[0050] Module for obtaining the incoming material direction angle of the wine bottle to be analyzed: Obtain the bottom image of the wine bottle to be analyzed, and calculate the incoming material direction angle of the wine bottle to be analyzed based on the bottom image of the wine bottle to be analyzed;

[0051] Module for obtaining the actual corrected angle: Calculate the actual corrected angle according to the standard incoming material direction angle of the wine bottle and the incoming material direction angle of the wine bottle to be analyzed;

[0052] Correction module: Correct the incoming angle of the wine bottle to be analyzed based on the actual corrected angle.

[0053] Furthermore, the specific implementation steps of the module for obtaining the standard incoming material direction angle of the wine bottle are as follows:

[0054] Step 1.1: Obtain the bottom image of the wine bottle with the standard incoming material direction:

[0055] Step 1.2: Obtain the pixel coordinates of the center point of the wine bottle bottom based on the bottom image of the wine bottle with the standard incoming material direction. The specific steps are as follows:

[0056] First, perform mean filtering on the bottom image of the wine bottle in the standard feeding direction;

[0057] Next, perform image enhancement preprocessing on the result of the mean filtering;

[0058] Next, perform image segmentation on the result of the image enhancement preprocessing to obtain a binary image. Among them, for a relatively good imaging environment, direct threshold segmentation is used, and for a relatively poor imaging quality, a deep learning network is used. A relatively good imaging environment means that there is no reflection or noise in the bottom image of the wine bottle in the standard feeding direction after imaging, and a relatively poor imaging quality means that there is reflection or noise in the bottom image of the wine bottle in the standard feeding direction after imaging;

[0059] Use the contour finding function findContours supported by the OpenCV library to find the contours of the binary image and calculate the contour areas. Use the calculated contour areas to filter out and delete the reflection interference smaller than a given threshold. After deletion, what remains is the bottom contour of the wine bottle in the standard feeding direction;

[0060] Finally, use the minAreaRect function in OpenCV to locate the minimum bounding rectangle of the bottom contour, and obtain the center point coordinates of the minimum bounding rectangle of the bottom contour as the pixel coordinates of the center point of the bottom of the wine bottle in the standard feeding direction;

[0061] Step 1.3: Based on the bottom image of the wine bottle in the standard feeding direction, determine the boundary of the direction discrimination mark area through manual annotation, and obtain the pixel coordinates of the center point of this area. Among them, the boundary of the direction discrimination mark area is a rotated rectangle frame composed of four vertex coordinates. After annotation, the rotated rectangle frame (class_idx, x1, y1, x2, y2, x3, y3, x4, y4) and the pixel coordinates of the center point of the rotated rectangle = ((x1 + x2 + x3 + x4) / 4, (y1 + y2 + y3 + y4) / 4) can be obtained. class_idx represents the category id, and (x1, y1), (x2, y2), (x3, y3), (x4, y4) respectively represent the four vertex coordinates of the rotated rectangle frame;

[0062] Step 1.4: Based on the pixel coordinates of the center point of the boundary of the direction discrimination mark area in the bottom image of the wine bottle in the standard feeding direction, point to the pixel coordinates of the center point of the bottom of the wine bottle in the standard feeding direction to form a direction vector, and calculate the angle between this direction vector and the horizontal x-axis as the standard feeding direction angle of the wine bottle. The formula is:

[0063]

[0064] In the formula, Let \(\vec{v}\) be the direction vector. It is calculated by the center point pixel coordinates \(flay_o(fcx, fcy)\) of the direction discrimination mark area in the bottom image of the wine bottle in the standard feeding direction and the center point pixel coordinates \(bottle_o(bcx, bcy)\) of the bottom of the wine bottle in the standard feeding direction. A negative sign is added in front of \(y\) because the change of the image coordinate and the Cartesian coordinate system in the \(y\)-axis direction is opposite. \(\theta(y, x)\) represents the angle between the calculated direction vector and the horizontal \(x\)-axis, which is the standard feeding direction angle of the wine bottle. The formula compensates the calculated angle by judging the quadrant of the positive and negative signs of \(x\) and \(y\) so that the angle range is \([-180, 180]\);

[0065] Furthermore, the specific implementation steps of the module for obtaining the feeding direction angle of the wine bottle to be analyzed are as follows:

[0066] Step 3.1: Use the trained rotation rectangle detection model of the direction discrimination mark area to locate and identify the direction discrimination mark area in the bottom image of the wine bottle to be analyzed, and then post-process the recognition result to obtain the final rotation rectangle. Among them, the final rotation rectangle is the smallest circumscribed rectangle of the boundary of the direction discrimination mark area. The center point pixel coordinates output by this smallest circumscribed rectangle are the center point pixel coordinates of the direction discrimination mark area. Post-processing means that when the number of recognition results is greater than 1, the confidence levels of the boundaries of all direction discrimination mark areas are sorted, and the one with the highest confidence probability is output as the final rotation rectangle; when the number of recognition results is 0, at this time, it is directly fed back that the input image is abnormal and no subsequent operation steps are performed. Otherwise, the final rotation rectangle is directly obtained;

[0067] Step 3.2: Calculate the center point pixel coordinates of the bottom of the wine bottle to be analyzed;

[0068] Step 3.3: Calculate the angle between the direction vector from the center point pixel coordinates \(flag_o1(fcx1, fcy1)\) of the direction discrimination mark area to the center point pixel coordinates \(bottle_o1(bcx1, bcy1)\) of the bottom of the wine bottle to be analyzed and the horizontal \(x\)-axis as the feeding direction angle of the wine bottle to be analyzed;

[0069] The specific implementation steps of the actual correction angle acquisition module are as follows:

[0070] First, calculate the difference between the feeding direction angle of the wine bottle to be analyzed and the standard feeding direction angle of the wine bottle to obtain the accurate angle for adjusting the feeding angle of the wine bottle to be analyzed to the desired feeding angle. The calculation formula is:

[0071] angle = theta2 - theta1

[0072] Among them, angle represents the accurate angle at which the incoming angle of the wine bottle to be analyzed is adjusted to the desired incoming angle, that is, the corrected angle is obtained. theta2 represents the incoming material direction angle of the wine bottle to be analyzed, and theta1 represents the standard incoming material direction angle of the wine bottle;

[0073] Perform a normalization operation on the corrected angle to normalize the corrected angle to the range of [-180, 180]. Finally, output the actual corrected angle angle_norm. The formula is as follows:

[0074]

[0075] The specific implementation steps of the correction module are as follows:

[0076] Send the actual corrected angle to the plc side. If the actual corrected angle is less than 0, rotate the incoming angle of the wine bottle to be analyzed clockwise based on the actual corrected angle for correction, that is, rotate clockwise by -angle_norm degrees. If the actual corrected angle is greater than 0, rotate the incoming angle of the wine bottle to be analyzed counterclockwise based on the actual corrected angle for correction, that is, rotate counterclockwise by angle_norm degrees.

[0077] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0078] The present invention proposes a method for correcting the incoming material angle of wine bottles based on vision, which has the characteristics of universality, anti-interference, accuracy, and stability, and thus effectively ensures the detection effect of the wine bottle printing defect detector. Specifically, it is embodied as follows:

[0079] 1. The present invention is not only applicable to round bottles, but also applicable to other various bottle types such as square bottles and rectangular bottles. The printed graphics are clear, the imaging is complete, and the image acquisition is time-saving, which can meet the requirements of the on-site production beat, effectively avoiding the problems that the design method based on the guiding mechanism can only handle some bottle types and different bottle types require targeted design of the hardware structure.

[0080] 2. The present invention can obtain accurate corrected angle information based on vision analysis and processing, and then accurately correct the incoming material direction to ensure the accuracy and consistency of the incoming material angles of round and multi-face symmetric wine bottles, and thus ensure that the printed content is clearly and completely within the camera's field of view.

[0081] 3. When calculating the image coordinates of the center point of the direction landmark area in the present invention, the yolo-obb model is used to detect this area. The yolo-obb can perform complex feature expression using multiple convolutional modules, and combined with various data augmentation means to provide rich training data to suppress the light source reflection interference generated by the white glass of the wine bottle. Secondly, when calculating the coordinates of the center point of the bottom of the wine bottle, environmental interference is suppressed by adding filtering processing and contour analysis and filtering operations.

[0082] IV. The present invention calculates the feeding deflection angle ratio based on the center point coordinates of the rotating rectangle and the center point coordinates of the bottom of the wine bottle, which is more stable than calculating the feeding deflection angle information using the angle information of the directly rotating rectangle frame. Description of the Drawings

[0083] Figure 1 The process of calculating the center of the wine bottle from the bottom image of the wine bottle to be analyzed in the present invention;

[0084] Figure 2 Schematic diagram of the direction vector of the bottom image of the wine bottle in the standard feeding direction in the present invention. Detailed Embodiment

[0085] The present invention will be further described below in conjunction with the drawings and specific embodiments.

[0086] The bottom of the wine bottle body generally has a front or front view discrimination mark. Therefore, the present invention triggers a specific stroboscopic light source through the product of the optoelectronic switch and analyzes the pattern of the bottom of the wine bottle by taking pictures with a camera. The specific steps are as follows:

[0087] A method for correcting the feeding angle of a wine bottle based on vision, comprising the following steps:

[0088] Step 1: Obtain the bottom image of the wine bottle in the standard feeding direction, and obtain the standard feeding direction angle of the wine bottle based on this bottom image; the specific steps are as follows:

[0089] Step 1.1: Obtain the bottom image of the wine bottle in the standard feeding direction:

[0090] Step 1.2: Obtain the pixel coordinates of the center point of the bottom of the wine bottle based on the bottom image of the wine bottle in the standard feeding direction. The specific steps are as follows:

[0091] First, perform mean filtering on the bottom image of the wine bottle in the standard feeding direction;

[0092] Next, perform image enhancement preprocessing on the result of the mean filtering;

[0093] Next, perform image segmentation on the result of the image enhancement preprocessing to obtain a binary image. Among them, for a relatively good imaging environment, direct threshold segmentation is used, and for a relatively poor imaging quality, a deep learning network is used. A relatively good imaging environment means that there is no reflection or noise after the bottom image of the wine bottle in the standard feeding direction is imaged, and a relatively poor imaging quality means that there is reflection or noise after the bottom image of the wine bottle in the standard feeding direction is imaged;

[0094] Use the contour finding function findContours supported by the OpenCV library to find the contours of the binary image and calculate the contour area. Use the calculated contour area to filter out and delete the reflection interference smaller than a given threshold. After deletion, the remaining is the bottom contour of the wine bottle in the standard feeding direction;

[0095] Finally, the minimum circumscribed rectangle of the bottom contour of the bottle is located by the minAreaRect function in OpenCV, and the center point coordinates of the minimum circumscribed rectangle of the bottom contour of the bottle are obtained as the pixel coordinates of the bottom center point of the wine bottle in the standard feeding direction.

[0096] Step 1.3: Determine the boundary of the direction discrimination flag area through manual annotation based on the bottom image of the wine bottle in the standard feeding direction, and obtain the pixel coordinates of the center point of this area. Among them, the boundary of the direction discrimination flag area is a rotated rectangle frame composed of four vertex coordinates. After annotation, the rotated rectangle frame (class_idx, x1, y1, x2, y2, x3, y3, x4, y4) and the pixel coordinates of the center point of the rotated rectangle can be obtained = ((x1 + x2 + x3 + x4) / 4, (y1 + y2 + y3 + y4) / 4). class_idx represents the category id, and (x1, y1), (x2, y2), (x3, y3), (x4, y4) respectively represent the four vertex coordinates of the rotated rectangle frame.

[0097] Step 1.4: A direction vector is formed by the pixel coordinates of the center point of the boundary of the direction discrimination flag area in the bottom image of the wine bottle in the standard feeding direction pointing to the pixel coordinates of the bottom center point of the wine bottle in the standard feeding direction, and the included angle between this direction vector and the horizontal x-axis is calculated as the standard feeding direction angle of the wine bottle. The formula is:

[0098]

[0099] In the formula, is the direction vector, which is calculated through the pixel coordinates of the center point flag_o(fcx, fcy) of the direction discrimination flag area in the bottom image of the wine bottle in the standard feeding direction and the pixel coordinates of the bottom center point bottle_o(bcx, bcy) of the wine bottle in the standard feeding direction. A negative sign is added in front of y because the change of the image coordinates and the Cartesian coordinate system in the y-axis direction is opposite. θ(y, x) represents the included angle between this direction vector and the horizontal x-axis, which is defined as the standard feeding direction angle of the wine bottle. The formula compensates for the calculated angle by judging the quadrant of the positive and negative signs of x and y, so that the angle range is [-180, 180];

[0100] Specifically: Randomly select a wine bottle and place it according to the expected standard feeding direction, and collect the bottom image of the wine bottle in the standard feeding direction. First, analyze and process the image to obtain the pixel coordinates of the bottom center point bottle_o(bcx, bcy) of the wine bottle in the standard feeding direction; secondly, manually determine the boundary of the direction discrimination annotation area and annotate it, and then obtain the pixel coordinates of the center point flag_o(fcx, fcy) of this area; as Figure 2As shown in the figure, finally, the angle between the direction vector formed by flag_o pointing to bottle_o and the horizontal x-axis is calculated as the standard feeding direction angle theta1. Definition of the angle direction in this article: The positive x-axis direction of the horizontal is the starting point of 0 degrees, and the counterclockwise rotation is the positive direction. The range of the standard feeding direction angle is [-180, 180].

[0101] The calculation process of the pixel coordinates of the center point of the bottom of the wine bottle in the standard feeding direction is as follows Figure 1 As shown in the figure: First, perform mean filtering to reduce the interference of camera noise and environmental noise; then perform image enhancement preprocessing operations to improve the image contrast and expand the difference between the foreground and the background, which is convenient for subsequent segmentation processing; then perform image segmentation operations. Here, the algorithm is selected according to the complexity of the pattern. If the imaging environment is relatively good, threshold segmentation can be directly performed. If the imaging quality is poor, a more robust deep learning network such as unet or deeplabv3 can be selected; further post-processing is performed on the binary image obtained based on image segmentation: perform contour search and contour analysis on the binary image, and use the contour area to filter and retain the bottom contour of the bottle to filter out the reflection interference; finally, locate the minimum circumscribed rectangle of the bottom contour of the bottle, and then determine the pixel coordinates bottle_o of the center point of the bottom of the wine bottle in the standard feeding direction. Step 2: Collect the bottom images of wine bottles in any feeding direction to construct a dataset, and train a direction discriminant flag area rotated rectangle positioning model after annotating the dataset;

[0102] The direction discriminant flag area rotated rectangle positioning model is the yolo11-obb model, which includes a feature extraction module (backbone), a feature fusion and enhancement module (neck), and a task head (head) connected in sequence;

[0103] Among them, the feature layer model uses the C3K2 module to replace the traditional C2f module of yolov8. This module still uses the CSP structure. This module is compatible with the C2f module by using the c3k identifier. When c3k = False, that is, C3K2 is equivalent to the C2f module. When c3k = True, the Bottleneck is replaced by the C3k module, and the C3k module supports the configuration of the convolutional kernel size.

[0104] The feature fusion and enhancement module uses the C2PSA module;

[0105] The task head contains 3 branches, including a class prediction branch, a bounding box branch, and an angle regression branch respectively.

[0106] The rotation rectangle positioning model for the direction discrimination flag area can be the yolo11-obb model. Considering the uncertainty of the incoming material direction, as well as the interference of camera noise and environmental noise in the actual situation of this application, data augmentation preprocessing operations such as online image rotation, color transformation, and adding noise are added during the training process. This can not only improve the generalization of the model but also reduce the dependence on the amount of labeled data and thus reduce the manual labeling cost.

[0107] The specific steps are as follows:

[0108] Collect the bottom images of wine bottles in any incoming material direction to construct a dataset;

[0109] Use the rotation rectangle annotation tool X-AnyLabeling to annotate the boundaries of the direction discrimination flag areas for the bottom images of each wine bottle in the dataset;

[0110] Train the rotation rectangle positioning model for the direction discrimination flag area based on the annotated dataset to obtain a trained rotation rectangle positioning model for the direction discrimination flag area.

[0111] Step 3: Obtain the bottom image of the wine bottle to be analyzed, and calculate the incoming material direction angle of the wine bottle to be analyzed based on the bottom image of the wine bottle to be analyzed; the specific steps are as follows:

[0112] Step 3.1: Use the trained rotation rectangle detection model for the direction discrimination flag area to perform positioning and recognition of the direction discrimination flag area on the obtained bottom image of the wine bottle to be analyzed, and then post-process the recognition result to obtain the final rotation rectangle box. Among them, the final rotation rectangle box is the smallest circumscribed rectangle of the boundary of the direction discrimination flag area, and the pixel coordinates of the center point output by this smallest circumscribed rectangle are the pixel coordinates of the center point of the direction discrimination annotation area. Post-processing means that when the number of recognition results is greater than 1, the confidence levels of all the boundaries of the direction discrimination flag areas are sorted, and the one with the highest confidence probability is output as the final rotation rectangle box; when the number of recognition results is 0, at this time, directly feedback that the input image is abnormal and no subsequent operation steps are performed, otherwise, directly obtain the final rotation rectangle box;

[0113] Step 3.2: Calculate the pixel coordinates bottle_o1 of the center point of the bottom of the wine bottle to be analyzed, in the same way as calculating the pixel coordinates bottle_o of the center point of the bottom of the standard incoming material direction wine bottle;

[0114] Step 3.3: Calculate the angle between the direction vector pointing from the center pixel coordinates flag_o1(fcx1, fcy1) of the direction discrimination flag area to the center pixel coordinates bottle_o1(bcx1, bcy1) of the bottom of the wine bottle to be analyzed and the horizontal x-axis as the feeding direction angle theta2 of the wine bottle to be analyzed. The value range is [-180, 180], which is the same as the calculation method of the standard feeding direction angle of the wine bottle, except that x and y in the formula are replaced respectively, that is, x = bcx1 - fcx1, y = -(bcy1 - fcy1).

[0115] Step 4: Calculate the actual correction angle according to the standard feeding direction angle of the wine and the feeding direction angle of the wine bottle to be analyzed; the specific steps are as follows:

[0116] First, calculate the difference between the feeding direction angle of the wine bottle to be analyzed and the standard feeding direction angle of the wine bottle to obtain the accurate angle for adjusting the feeding angle of the wine bottle to be analyzed to the expected feeding angle. The calculation formula is:

[0117] angle = thata2 - theta1

[0118] Where, angle represents the accurate angle for adjusting the feeding angle of the wine bottle to be analyzed to the expected feeding angle, that is, the correction angle is obtained. thata2 represents the feeding direction angle of the wine bottle to be analyzed, and theta1 represents the standard feeding direction angle; perform a normalization operation on the correction angle to normalize the correction angle to the range of [-180, 180], and finally output the actual correction angle.

[0119] Step 5: Correct the feeding angle of the wine bottle to be analyzed based on the actual correction angle;

[0120] The specific steps are as follows:

[0121] Send the actual correction angle to the plc side. If the actual correction angle is less than 0, rotate the feeding angle of the wine bottle to be analyzed clockwise based on the actual correction angle for correction. If the actual correction angle is greater than 0, rotate the feeding angle of the wine bottle to be analyzed counterclockwise based on the actual correction angle for correction.

[0122] The above are only representative embodiments in the numerous specific application scopes of the present invention, and do not constitute any limitation to the protection scope of the present invention. Any technical solutions formed by transformation or equivalent replacement fall within the scope of the protection of the present invention's rights.

Claims

1. A method for correcting the feeding angle of wine bottles based on vision, characterized in that, It includes the following steps: Step 1: Obtain the bottom image of the wine bottle in the standard feeding direction, and obtain the wine bottle standard feeding direction angle based on this bottom image; Step 2: Collect the bottom images of wine bottles in any feeding direction to construct a dataset, and after annotating the dataset, train a direction discrimination marker area rotated rectangle positioning model; Step 3: Obtain the bottom image of the wine bottle to be analyzed, and calculate the feeding direction angle of the wine bottle to be analyzed based on the bottom image of the wine bottle to be analyzed; Step 4: Calculate the actual correction angle according to the wine bottle standard feeding direction angle and the feeding direction angle of the wine bottle to be analyzed; Step 5: Correct the inlet angle of the wine bottle to be analyzed based on the actual correction angle.

2. A method for correcting the feeding angle of a wine bottle based on vision according to claim 1, characterized in that, The specific steps of Step 1 are as follows: Step 1.1: Obtain the bottom image of the wine bottle in the standard feeding direction: Step 1.2: Obtain the pixel coordinates of the center point of the wine bottle bottom based on the bottom image of the wine bottle in the standard feeding direction. The specific steps are as follows: First, perform mean filtering on the bottom image of the wine bottle in the standard feeding direction; Next, perform image enhancement preprocessing on the result of the mean filtering; Next, perform image segmentation on the result of the image enhancement preprocessing to obtain a binary image. Among them, for a relatively good imaging environment, direct threshold segmentation is used, and for a relatively poor imaging quality, a deep learning network is used. A relatively good imaging environment means that there is no reflection or noise after the bottom image of the wine bottle in the standard feeding direction is imaged, and a relatively poor imaging quality means that there is reflection or noise after the bottom image of the wine bottle in the standard feeding direction is imaged; Use the contour finding function findContours supported by the OpenCV library to find the contours of the binary image and calculate the contour area. Use the calculated contour area to filter out and delete the reflection interference smaller than a given threshold. After deletion, what remains is the bottom contour of the wine bottle in the standard feeding direction; Finally, locate the minimum circumscribed rectangle of the bottom contour through the minAreaRect function in OpenCV, and obtain the center point coordinates of the minimum circumscribed rectangle of the bottom contour as the pixel coordinates of the center point of the bottom of the wine bottle in the standard feeding direction; Step 1.3: Determine the boundary of the direction discrimination marker area through manual annotation based on the bottom image of the wine bottle in the standard feeding direction, and obtain the pixel coordinates of the center point of this area. Among them, the boundary of the direction discrimination marker area is a rotated rectangle frame composed of four vertex coordinates. After annotation, the rotated rectangle frame (class_idx, x1, y1, x2, y2, x3, y3, x4, y4) and the pixel coordinates of the center point of the rotated rectangle can be obtained = ((x1 + x2 + x3 + x4) / 4, (y1 + y2 + y3 + y4) / 4), where class_idx represents the category id, and (x1, y1), (x2, y2), (x3, y3), (x4, y4) respectively represent the four vertex coordinates of the rotated rectangle frame; Step 1.4: Form a direction vector from the pixel coordinates of the center point of the boundary of the direction discrimination marker area in the bottom image of the wine bottle in the standard feeding direction to the pixel coordinates of the center point of the bottom of the wine bottle in the standard feeding direction, and calculate the included angle between this direction vector and the horizontal x-axis as the wine bottle standard feeding direction angle. The formula is: In the formula, is the direction vector, which is calculated through the central point pixel coordinates flag_o(fcx, fcy) of the direction discrimination mark area in the bottom image of the wine bottle in the standard feeding direction and the central point pixel coordinates bottle_o(bcx, bcy) of the bottom of the wine bottle in the standard feeding direction. A negative sign is added in front of y because the change of the image coordinates and the Cartesian coordinate system in the y-axis direction is opposite. θ(y, x) represents the angle between the calculated direction vector and the horizontal x-axis, which is the standard feeding direction angle of the wine bottle. The formula compensates the calculated angle by discriminating the quadrant of the positive and negative signs of x and y, so that the angle range is [-180, 180].

3. A method for correcting the feeding angle of a wine bottle based on vision according to claim 2, characterized in that, The rotated rectangle positioning model of the direction discrimination flag area in step 2 is the yolo11-obb model, which includes a feature extraction module, a feature fusion and enhancement module, and a task head connected in sequence.

4. A method for correcting the feeding angle of a wine bottle based on vision according to claim 2, characterized in that, The specific steps of step 2 are as follows: Collect the bottom images of wine bottles in any feeding direction to construct a dataset; Use the rotated rectangle annotation tool X-AnyLabel ing to annotate the boundaries of the direction discrimination flag areas in the bottom images of each wine bottle in the dataset; Train the rotated rectangle positioning model of the direction discrimination flag area based on the annotated dataset to obtain a trained rotated rectangle positioning model of the direction discrimination flag area.

5. A method for correcting the feeding angle of a wine bottle based on vision according to claim 4, characterized in that, The specific steps of step 3 are as follows: Step 3.1: Use the trained rotated rectangle detection model of the direction discrimination flag area to perform positioning and recognition of the direction discrimination flag area on the bottom image of the wine bottle to be analyzed obtained, and then post-process the recognition result to obtain the final rotated rectangle box. Among them, the final rotated rectangle box is the smallest circumscribed rectangle of the boundary of the direction discrimination flag area, and the pixel coordinates of the center point output by this smallest circumscribed rectangle are the pixel coordinates of the center point of the direction discrimination annotation area. Post-processing means that when the number of recognition results is greater than 1, the confidence levels of all the boundaries of the direction discrimination flag areas are sorted, and the one with the highest confidence probability is output as the final rotated rectangle box; when the number of recognition results is 0, at this time, directly feedback that the input image is abnormal and no subsequent operation steps are performed, otherwise, directly obtain the final rotated rectangle box; Step 3.2: Calculate the pixel coordinates of the center point of the bottom of the wine bottle to be analyzed; Step 3.3: Calculate the angle between the direction vector pointing from the pixel coordinates flag_o1(fcx1, fcy1) of the center point of the direction discrimination flag area to the pixel coordinates bottle_o1(bcx1, bcy1) of the center point of the bottom of the wine bottle to be analyzed and the horizontal x-axis, as the feeding direction angle of the wine bottle to be analyzed.

6. A method for correcting the feeding angle of a wine bottle based on vision according to claim 5, characterized in that, The specific steps of step 4 are as follows: First, calculate the difference between the feeding direction angle of the wine bottle to be analyzed and the standard feeding direction angle of the wine bottle to obtain the accurate angle for adjusting the incoming angle of the wine bottle to be analyzed to the desired incoming angle. The calculation formula is: angle = theta2 - theta1 where angle represents the accurate angle for adjusting the incoming angle of the wine bottle to be analyzed to the desired incoming angle, that is, the correction angle is obtained, theta2 represents the feeding direction angle of the wine bottle to be analyzed, and theta1 represents the standard feeding direction angle of the wine bottle; Perform a normalization operation on the correction angle to normalize the correction angle to the range of [-180, 180], and finally output the actual correction angle angle_norm. The formula is:

7. A method for correcting the feeding angle of a wine bottle based on vision, as claimed in claim 6, wherein The specific steps of step 5 are as follows: Send the actual correction angle to the plc side. If the actual correction angle is less than 0, then rotate the incoming angle of the wine bottle to be analyzed clockwise based on the actual correction angle for correction, that is, rotate clockwise by -angle_norm degrees. If the actual correction angle is greater than 0, then rotate the incoming angle of the wine bottle to be analyzed counterclockwise based on the actual correction angle for correction, that is, rotate counterclockwise by angle_norm degrees.

8. A system for correcting the feeding angle of wine bottles based on vision, characterized in that, Including: Bottle standard feeding direction angle acquisition module: Obtain the bottom image of the bottle with the standard feeding direction, and obtain the bottle standard feeding direction angle based on this bottom image; Model training module: Collect the bottom images of bottles with arbitrary feeding directions to construct a dataset, and after annotating this dataset, train the rotation rectangle positioning model for the direction discriminant flag area; Bottle feeding direction angle acquisition module to be analyzed: Obtain the bottom image of the bottle to be analyzed, and calculate the bottle feeding direction angle to be analyzed based on the bottom image of the bottle to be analyzed; Actual correction angle acquisition module: Calculate the actual correction angle according to the bottle standard feeding direction angle and the bottle feeding direction angle to be analyzed; Correction module: Correct the inlet angle of the bottle to be analyzed based on the actual correction angle.

9. A system for correcting the feeding angle of a wine bottle based on vision according to claim 8, wherein The specific implementation steps of the bottle standard feeding direction angle acquisition module are as follows: Step 1.1: Obtain the bottom image of the bottle with the standard feeding direction: Step 1.2: Obtain the pixel coordinates of the center point of the bottle bottom based on the bottom image of the bottle with the standard feeding direction. The specific steps are as follows: First, perform mean filtering on the bottom image of the bottle with the standard feeding direction; Next, perform image enhancement preprocessing on the result of the mean filtering; Next, perform image segmentation on the result of the image enhancement preprocessing to obtain a binary image. Among them, for a relatively good imaging environment, direct threshold segmentation is used, and for a relatively poor imaging quality, a deep learning network is used. A relatively good imaging environment means that there is no reflection or noise after the bottom image of the bottle with the standard feeding direction is imaged, and a relatively poor imaging quality means that there is reflection or noise after the bottom image of the bottle with the standard feeding direction is imaged; Use the contour finding function findContours supported by the OpenCV library to find the contours of the binary image and calculate the contour areas. Use the calculated contour areas to filter out and delete the reflection interference smaller than a given threshold. After deletion, what remains is the bottom contour of the bottle with the standard feeding direction; Finally, use the minAreaRect function in OpenCV to locate the minimum circumscribed rectangle of the bottle bottom contour, and obtain the center point coordinates of the minimum circumscribed rectangle of the bottle bottom contour as the pixel coordinates of the center point of the bottom of the bottle with the standard feeding direction; Step 1.3: Determine the boundary of the direction discriminant flag area through manual annotation based on the bottom image of the bottle with the standard feeding direction, and obtain the pixel coordinates of the center point of this area. Among them, the boundary of the direction discriminant flag area is a rotation rectangle frame composed of four vertex coordinates. After annotation, the rotation rectangle frame (class_idx, x1, y1, x2, y2, x3, y3, x4, y4) and the pixel coordinates of the center point of the rotation rectangle can be obtained = ((x1 + x2 + x3 + x4) / 4, (y1 + y2 + y3 + y4) / 4), where class_idx represents the category id, and (x1, y1), (x2, y2), (x3, y3), (x4, y4) respectively represent the four vertex coordinates of the rotation rectangle frame; Step 1.4: Based on the center point pixel coordinates of the boundary of the direction discriminant marker area in the bottom image of the wine bottle in the standard feeding direction, a direction vector is formed that points to the center point pixel coordinates of the bottom of the wine bottle in the standard feeding direction. The angle between this direction vector and the horizontal x-axis is calculated as the standard feeding direction angle of the wine bottle. The formula is as follows: In the formula, is the direction vector, which is calculated through the central point pixel coordinates flag_o(fcx, fcy) of the direction discrimination flag area in the bottom image of the wine bottle in the standard feeding direction and the central point pixel coordinates bottle_o(bcx, bcy) of the bottom of the wine bottle in the standard feeding direction. A negative sign is added in front of y because the change of the image coordinate and the Cartesian coordinate system in the y-axis direction is opposite. θ(y, x) represents the angle between the calculated direction vector and the horizontal x-axis, that is, the standard feeding direction angle of the wine bottle. The formula compensates the calculated angle by judging the quadrant of the positive and negative signs of x and y, so that the angle range is [-180, 180].

10. A system for correcting the feeding angle of wine bottles based on vision as claimed in claim 9, characterized in that, The specific implementation steps of the feeding direction angle acquisition module for the wine bottle to be analyzed are as follows: Step 3.1: Use the trained rotation rectangle detection model for the direction discriminant marker area to locate and identify the direction discriminant marker area in the bottom image of the wine bottle to be analyzed, and then post-process the recognition result to obtain the final rotation rectangle. Among them, the final rotation rectangle is the smallest circumscribed rectangle of the boundary of the direction discriminant marker area. The center point pixel coordinates output by this smallest circumscribed rectangle are the center point pixel coordinates of the direction discriminant annotation area. Post-processing means that when the number of recognition results is greater than 1, the confidence levels of all boundaries of the direction discriminant marker areas are sorted, and the one with the highest confidence probability is output as the final rotation rectangle; when the number of recognition results is 0, the input image is directly reported as abnormal and no subsequent operation steps are performed. Otherwise, the final rotation rectangle is directly obtained; Step 3.2: Calculate the center point pixel coordinates of the bottom of the wine bottle to be analyzed; Step 3.3: Calculate the angle between the horizontal x-axis and the direction vector formed by the center point pixel coordinates flag_o1(fcx1, fcy1) of the direction discriminant marker area pointing to the center point pixel coordinates bottle_o1(bcx1, bcy1) of the bottom of the wine bottle to be analyzed as the feeding direction angle of the wine bottle to be analyzed; The specific implementation steps of the actual correction angle acquisition module are as follows: First, calculate the difference between the feeding direction angle of the wine bottle to be analyzed and the standard feeding direction angle of the wine bottle to obtain the accurate angle for adjusting the feeding angle of the wine bottle to be analyzed to the expected feeding angle. The calculation formula is as follows: angle = thata2 - theta1 Where, angle represents the accurate angle for adjusting the feeding angle of the wine bottle to be analyzed to the expected feeding angle, that is, the correction angle is obtained. thata2 represents the feeding direction angle of the wine bottle to be analyzed, and theta1 represents the standard feeding direction angle of the wine bottle; Perform a normalization operation on the correction angle to normalize the correction angle to the range of [-180, 180]. Finally, output the actual correction angle angle_norm. The formula is as follows: The specific implementation steps of the correction module are as follows: Send the actual correction angle to the plc side. If the actual correction angle is less than 0, rotate the feeding angle of the wine bottle to be analyzed clockwise based on the actual correction angle for correction, that is, rotate clockwise by -angle_norm degrees. If the actual correction angle is greater than 0, rotate the feeding angle of the wine bottle to be analyzed counterclockwise based on the actual correction angle for correction, that is, rotate counterclockwise by angle_norm degrees.

Images