A paper stacking control method and system based on machine vision

Abstract

The present invention discloses a paper folding control method and system based on machine vision, which relate to the technical field of paper product processing. The method comprises: collecting and preprocessing paper towel data to obtain preprocessed paper towel image data; constructing a paper towel feature extraction model to obtain paper towel image features; calculating paper towel geometric features according to the preprocessed paper towel image data, and checking whether there is a deviation in the paper towel rotation angle; if there is a deviation in the paper towel rotation angle, calculating the deviation angle, and outputting a PID rotation adjustment parameter to a PID control system to correct the deviation angle; constructing a crease position detection model to calculate the crease position, and comparing it with the expected crease position to check whether there is a crease deviation; if there is a crease deviation, correcting the crease deviation through a PID control system; the present invention aims to solve the problem of poor folding effect caused by inaccurate control of the steering angle and crease position in traditional paper towel folding methods.

Description

Technical Field

[0001] The present invention relates to the technical field of paper product processing, and in particular to a paper stacking control method and system based on machine vision. Background Art

[0002] Intelligent origami control realizes automatic folding of paper or tissue paper through electronic control system, camera and origami mechanical structure; intelligent origami control can not only automatically complete complex folding actions according to preset programs, but also monitor the state of paper during folding, such as position and angle, in real time through machine vision system, and dynamically adjust the folding strategy according to this information to ensure the folding effect of tissue paper; in this process, the machine vision system is responsible for capturing paper images in real time, extracting key information, and feeding this information back to the control system to realize closed-loop control.

[0003] Compared with the intelligent origami method, the traditional origami method has the following shortcomings in automation and precision control: First, as the number of folds increases, the thickness of the paper towel gradually increases, and direct folding may become difficult and inaccurate, so pre-folding is required before folding, and the traditional method is difficult to accurately control the position of the pre-folded crease; second, after folding four times, the paper towel needs to turn to complete the subsequent folding action. The traditional method often relies on a simple mechanical structure to achieve steering, but this method has shortcomings in accuracy and stability. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the deficiencies in the prior art, the present invention provides a paper folding control method and system based on machine vision, which aims to solve the problem of poor folding effect caused by inaccurate control of steering angle and crease position in traditional paper towel folding methods.

[0006] (II) Technical solution

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0008] A paper stacking control method based on machine vision, comprising:

[0009] S1, collecting and preprocessing paper towel data to obtain a grayscale image and a binary image in sequence, and then storing the paper towel data, grayscale image, and binary image as the first, second, and third channels respectively to obtain preprocessed paper towel image data;

[0010] S2. Calculate an edge image according to the binary image channel in the preprocessed tissue image data, and use the edge image as the fourth channel to perform channel superposition with the preprocessed tissue image data to obtain four-channel tissue image data; then, construct a tissue feature extraction model, input the four-channel tissue image data into the model, and obtain tissue image features;

[0011] The calculation method of the paper towel feature extraction model is:

[0012] ;

[0013] ;

[0014] ;

[0015] ;

[0016] in, is the key point of the tissue image, is the feature description vector of the key points of the tissue image, It is the SIFT feature point extraction operation. They are the data of the first, third, and fourth channels in the four-channel tissue image data. is the convolution feature of the tissue image, is the residual convolutional network operation, is the data of the second channel in the four-channel tissue image data, Rotate the features of the tissue image, is the sigmoid function, is the rotation weight matrix, is the rotation bias vector, is the tissue image feature, For convolutional neural network operations, This is a vector concatenation operation.

[0017] S3. Use the YOLOv8 model to perform target recognition on the preprocessed tissue image data, and calculate the tissue geometric features according to the edge contour of the tissue recognition frame; after splicing the tissue image features with the tissue geometric features, input them into the VGGNet network, calculate the feature comparison vector, and output whether there is a rotation deviation of the tissue rotation angle according to the feature comparison vector;

[0018] S4. If there is a deviation in the rotation angle of the paper towel, the deviation angle is calculated, and the PID rotation adjustment parameter is output to the PID control system to correct the deviation angle;

[0019] S5. Construct a crease position detection model, calculate the crease position based on the tissue image features, and compare it with the expected crease position to check whether there is a crease deviation; if there is a crease deviation, correct it through the PID control system.

[0020] Furthermore, paper towel data is collected and preprocessed to obtain a grayscale image and a binary image in sequence, and the paper towel data, grayscale image, and binary image are stored as the first, second, and third channels respectively to obtain preprocessed paper towel image data, including:

[0021] Collecting paper towel data, wherein the paper towel data is a color image obtained by photographing the paper towels on a conveyor belt at a vertical viewing angle using a high-resolution camera;

[0022] The paper towel data is grayed out using the weighted average method and Gaussian filtering to convert the paper towel data into a grayscale image.

[0023] The grayscale image is subjected to Otsu's method to calculate the inter-class variance of the grayscale image histogram and output a segmentation threshold, and then the grayscale image is converted into a binary image according to the segmentation threshold; the conversion rule is: pixels whose grayscale values ​​are greater than or equal to the segmentation threshold are set to white, and pixels whose grayscale values ​​are less than the threshold are set to black;

[0024] The paper towel data, the grayscale image and the binary image are stored as the first, second and third channels respectively to obtain the preprocessed paper towel image data;

[0025] This step utilizes the multi-source information of the image by storing the original color image, grayscale image and binary image of the paper towel as three different channels; the color image retains the original color information of the paper towel, which helps to more accurately identify the material and texture characteristics of the paper towel in subsequent steps; the grayscale image simplifies the image information, reduces the processing complexity, and retains the image structure information; the binary image highlights the edge and shape features of the paper towel through threshold segmentation, providing a clear boundary for subsequent feature extraction and geometric feature calculation.

[0026] Furthermore, in step S2, the edge image is calculated according to the binary image channel in the preprocessed tissue image data, including:

[0027] The Canny edge detection algorithm is used to perform edge detection on the binary image channel in the preprocessed tissue image data, and then the edge image is obtained through non-maximum suppression and double threshold processing.

[0028] Traditional methods often rely on only a single algorithm or model when processing feature extraction of paper towel images, and it is difficult to simultaneously meet the requirements of rotation, scale invariance and computational efficiency. The present invention constructs a paper towel feature extraction model, combines the advantages of multiple algorithms such as SIFT (Scale Invariant Feature Transform) and ResNet, and realizes feature extraction of four-channel paper towel image data. Since the paper towel needs to be rotated after being folded four times before it can be folded again, and the SIFT algorithm performs well in extracting key points and feature description vectors of paper towel images with its high rotation and scale invariance, SIFT feature point extraction is used to retain the features of the paper towel image before and after rotation. ResNet effectively alleviates the problems of gradient vanishing and gradient exploding in deep network training by introducing residual connections. The strategy of multi-algorithm fusion enables the paper towel feature extraction model to capture local details of the image and understand the overall structure and rotation characteristics of the image. It provides rich and accurate information support for subsequent steps such as paper towel geometric feature calculation, rotation angle deviation inspection, and crease position detection.

[0029] Furthermore, the YOLOv8 model is used to perform target recognition on the preprocessed tissue image data, and the tissue geometric features are calculated according to the edge contour of the tissue recognition frame, including:

[0030] The YOLOv8 model is used to perform target recognition on the preprocessed tissue image data, and the center point coordinates and direction angles of the tissue target are calculated according to the edge contour of the tissue recognition frame as the tissue geometric features. The calculation method is:

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] in, They are The horizontal and vertical coordinates of the upper left and lower right vertices of the tissue recognition box detected by the model. For YOLOv8 model operation, is the preprocessed tissue image data, are the horizontal and vertical coordinates of the center point of the paper towel target, is the direction angle of the tissue target, is the fully connected layer operation, It is the Harris corner detection operation;

[0036] Traditional geometric methods for calculating direction angles usually rely on the geometric features of the image, such as image edges and straight lines, which is more effective when the image quality is high. However, in practical applications, tissue images may be affected by multiple factors such as lighting changes, shadows, and complex textures, making it difficult for traditional geometric methods to accurately extract and identify the necessary geometric features. Inputting geometric features such as corner points and center points into the deep learning model and then calculating the image direction angle in a model-predicted manner can better adapt to these complex scenes with the help of complex features in a large amount of sample data, thereby improving the calculation effect.

[0037] After the tissue image features and tissue geometric features are spliced, they are input into the VGGNet network, feature comparison vectors are calculated, and whether there is a rotation deviation in the tissue rotation angle is output according to the feature comparison vectors, including:

[0038] The tissue image features and tissue geometric features are spliced ​​and input into the VGGNet network. The features are compared with the preset rotated tissue image to obtain the feature comparison vector. The feature comparison vector is used to output whether there is a rotation deviation in the tissue rotation angle. The calculation method is as follows:

[0039] ;

[0040] ;

[0041] ;

[0042] ;

[0043] ;

[0044] in, Overlay features on the tissue image. For VGGNet network operation, Overlay features on the preset rotated tissue image. is the orientation angle of the preset rotated tissue target, are the horizontal and vertical coordinates of the center point of the preset rotated tissue target, is the tissue image feature, is the preset rotated tissue image feature, is the feature comparison vector, is the rotation deviation probability, is the rotation deviation parameter matrix, is the rotation deviation bias vector; is the result of whether there is rotation deviation, For the existence of rotational deviation, To avoid rotation bias, the VGGNet network adopts a weight sharing mechanism.

[0045] The full name of VGG network is Visual Geometry Group Network, which is a deep convolutional neural network architecture developed by the Visual Geometry Group (VGG) research team at the University of Oxford;

[0046] The tissue image features are superimposed with the tissue geometric features and input into the weight-shared VGGNet convolutional neural network. The weight-shared network is equivalent to giving the model prior knowledge. When learning the superimposed features of tissue images, it already has preset features as the basis for learning and comparison, which can better capture the subtle differences in tissue images and provide feature vectors for the subsequent rotational deviation adjustment mechanism.

[0047] Furthermore, if there is a deviation in the rotation angle of the paper towel, the deviation angle is calculated, and the PID rotation adjustment parameter is output to the PID control system to correct the deviation angle, including:

[0048] If there is a deviation in the rotation angle of the paper towel, the deviation angle is calculated according to the feature comparison vector. The calculation method is:

[0049] ;

[0050] in, is the deviation angle;

[0051] Compared with the traditional method of directly subtracting angles, the present invention combines the feature comparison vector after angle subtraction and outputs the angle deviation in the form of full connection layer prediction, which is more robust and avoids the angle calculation error caused by inaccurate corner point feature extraction as much as possible;

[0052] Construct a PID parameter adjustment model, calculate the PID rotation adjustment feature according to the deviation angle and feature comparison vector, and then calculate the PID rotation adjustment parameter. The calculation method is:

[0053] ;

[0054] ;

[0055] in, is the PID rotation regulation characteristic, is the hyperbolic tangent function, is the PID proportional gain, is the proportional gain weight, is the proportional gain bias term, is the PID integral gain, is the integral gain weight, is the integral gain bias term, is the PID differential gain, is the differential gain weight, is the differential gain bias term; the PID proportional gain, PID integral gain and PID differential gain together constitute the PID rotation adjustment parameter;

[0056] By taking the deviation angle feature comparison vector as input, the PID parameter adjustment model can dynamically adjust the parameters of the PID controller according to the actual state of the current system (i.e., the deviation angle of the paper towel and the image feature comparison result); this dynamic adaptability enables the control system to respond to various changes more flexibly and improve the stability and response speed of the system; and the weights in the PID parameter adjustment model and the bias term It can be optimized through training; therefore, as the system runs and data accumulates, the parameters of the PID controller can be gradually adjusted to the optimal state to further improve control performance;

[0057] The calculation process of the PID parameter adjustment model is uniformly expressed as:

[0058] ;

[0059] in, is the PID rotation adjustment parameter, Adjust model operation for PID parameters;

[0060] The PID rotation adjustment parameters are transmitted to the PID control system to correct the deviation angle.

[0061] Furthermore, a crease position detection model is constructed, and the crease position is calculated based on the characteristics of the tissue image, and compared with the expected crease position to check whether there is a crease deviation; if there is a crease deviation, the crease deviation is corrected through the PID control system, including:

[0062] Construct a crease position detection model, input the tissue image features into the crease position detection model to calculate the crease position, and output the probability that the crease position of the tissue is deviated. The calculation method is:

[0063] ;

[0064] ;

[0065] ;

[0066] in, is the crease position image calculated by the Unet network, For UNet network operations, is the estimated fold position image, is the probability that the fold position of the paper towel is biased;

[0067] The present invention adopts the UNet network for image segmentation. The UNet network was proposed by Olaf Ronneberger et al. in 2015 and is described in detail in the paper "U-Net: Convolutional Networks for Biomedical Image Segmentation". The UNet network, with its unique U-shaped structure and jump connection, can accurately capture the detailed information in the image, especially the subtle features such as creases, thereby achieving high-precision positioning of the crease position. After high-precision positioning of the crease position, the present invention performs feature fusion and intelligent judgment on the actual crease position image and the expected crease position image through a fully connected layer activated by a sigmoid function, and outputs the probability of deviation of the crease position.

[0068] If the probability of deviation of the fold position of the paper towel is > 0.3, the fold deviation characteristics are calculated and input into the PID parameter adjustment model, the PID fold adjustment parameters are output and transmitted to the PID control system for parameter adjustment to correct the fold deviation. The calculation method is: ;

[0069] ;

[0070] in, is the crease deviation feature, For channel superposition operation, It is the PID crease adjustment parameter;

[0071] If the probability that there is a deviation in the fold position of the paper towel is ≤ 0.3, it is considered that there is no fold position deviation and the paper towel folding continues.

[0072] A paper stacking control system based on machine vision, comprising:

[0073] A paper towel data preprocessing unit: collects and preprocesses paper towel data to obtain a grayscale image and a binary image in sequence, and then stores the paper towel data, the grayscale image, and the binary image as the first, second, and third channels respectively to obtain preprocessed paper towel image data;

[0074] Paper towel feature extraction unit:

[0075] According to the binary image channel in the preprocessed tissue image data, the edge image is calculated, and the edge image is used as the fourth channel to perform channel superposition with the preprocessed tissue image data to obtain four-channel tissue image data; then, a tissue feature extraction model is constructed, and the four-channel tissue image data is input into the model to obtain tissue image features;

[0076] Rotation angle detection unit: The YOLOv8 model is used to perform target recognition on the preprocessed tissue image data, and the geometric features of the tissue are calculated based on the edge contour of the tissue recognition frame; the tissue image features and the tissue geometric features are spliced ​​and input into the VGGNet network to calculate the feature comparison vector, and the output is based on the feature comparison vector to determine whether there is a rotation deviation in the tissue rotation angle;

[0077] Rotation deviation correction unit: If there is a deviation in the rotation angle of the paper towel, the deviation angle is calculated and the PID rotation adjustment parameter is output to the PID control system to correct the deviation angle;

[0078] Crease position detection and correction unit: Build a crease position detection model, calculate the crease position based on the tissue image features, and compare it with the expected crease position to check whether there is a crease deviation; if there is a crease deviation, correct it through the PID control system.

[0079] (III) Beneficial effects

[0080] The present invention provides a paper stacking control method and system based on machine vision, which has the following beneficial effects:

[0081] 1. The present invention introduces machine vision and deep learning algorithms to achieve accurate detection and control of the steering angle and crease position during the folding process of paper towels. Compared with the traditional method that relies on manual experience and mechanical devices, the present invention can capture paper towel images in real time, accurately identify the rotation angle and crease position, and generate adjustment parameters to feed back to the PID control system, thereby achieving timely deviation correction and improving the folding effect of paper towels.

[0082] 2. Based on tissue image data, the present invention integrates the advantages of algorithms such as the rotation scale invariance of SIFT feature extraction and the network stability of ResNet, and utilizes the data characteristics of different channels to construct a tissue feature extraction model, thereby realizing feature extraction of four-channel tissue image data. This enables the tissue feature extraction model to capture local details of the image and understand the overall structure and rotation characteristics of the image, thus solving the problem of angle deviation caused by the rotation of the tissue after being folded four times.

[0083] 3. The present invention calculates the feature contrast vector through the shared weight VGGNet, and fuses the angle difference with the feature contrast vector to calculate the rotation deviation angle of the paper towel; compared with the traditional angle calculation method based on geometry, the present invention inputs geometric characteristics into the neural network, and can still maintain the stability of the deviation angle calculation when the image quality is poor. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 It is a flow chart of the paper stacking control method based on machine vision of the present invention;

[0085] Figure 2 It is a schematic diagram of the algorithm flow of paper towel image feature extraction of the present invention;

[0086] Figure 3 It is a schematic diagram of the algorithm flow of feature comparison vector extraction of the present invention;

[0087] Figure 4 This is a schematic diagram of the paper towel folding and rotating process of the present invention;

[0088] Figure 5 This is a schematic diagram of the pre-folding of a paper towel of the present invention. DETAILED DESCRIPTION

[0089] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0090] Embodiment 1: A paper stacking control method based on machine vision provided by the present invention, such as Figure 1 As shown, including:

[0091] Step 1, collecting and preprocessing paper towel data to obtain a grayscale image and a binary image in sequence, and then storing the paper towel data, grayscale image, and binary image as the first, second, and third channels respectively to obtain preprocessed paper towel image data;

[0092] The step 1 includes the following contents:

[0093] Step 101, collecting paper towel data, wherein the paper towel data is a color image obtained by photographing the paper towels on the conveyor belt with a high-resolution camera at a vertical viewing angle;

[0094] Step 102: grayscale the paper towel data using a weighted average method and perform Gaussian filtering to convert the paper towel data into a grayscale image;

[0095] Step 103: using Otsu's method to calculate the inter-class variance of the grayscale image histogram and output the segmentation threshold, and then converting the grayscale image into a binary image according to the segmentation threshold; the conversion rule is: pixels with grayscale values ​​greater than or equal to the segmentation threshold are set to white, and pixels with grayscale values ​​less than the threshold are set to black;

[0096] Step 104 , storing the tissue data, the grayscale image and the binary image as the first, second and third channels respectively, to obtain pre-processed tissue image data.

[0097] This step utilizes the multi-source information of the image by storing the original color image, grayscale image and binary image of the paper towel as three different channels; the color image retains the original color information of the paper towel, which helps to more accurately identify the material and texture characteristics of the paper towel in subsequent steps; the grayscale image simplifies the image information, reduces the processing complexity, and retains the image structure information; the binary image highlights the edge and shape features of the paper towel through threshold segmentation, providing a clear boundary for subsequent feature extraction and geometric feature calculation.

[0098] Step 2: Calculate the edge image based on the binary image channel in the preprocessed tissue image data, and use the edge image as the fourth channel to perform channel superposition with the preprocessed tissue image data to obtain four-channel tissue image data; then, construct a tissue feature extraction model, input the four-channel tissue image data into the model, and obtain tissue image features, such as Figure 2 As shown;

[0099] The step 2 includes the following contents:

[0100] Step 201, using the Canny edge detection algorithm to perform edge detection on the binary image channel in the preprocessed tissue image data, and then performing non-maximum suppression and double threshold processing to obtain an edge image;

[0101] Step 202: use the edge image as the fourth channel and perform channel superposition with the preprocessed tissue image data to obtain four-channel tissue image data;

[0102] Step 203: construct a paper towel feature extraction model, and input the four-channel paper towel image data into the model to obtain paper towel image features; the calculation method of the paper towel feature extraction model is:

[0103] ;

[0104] ;

[0105] ;

[0106] ;

[0107] in, is the key point of the tissue image, is the feature description vector of the key points of the tissue image, It is the SIFT feature point extraction operation. They are the data of the first, third, and fourth channels in the four-channel tissue image data. is the convolution feature of the tissue image, is the residual convolutional network operation, is the data of the second channel in the four-channel tissue image data, Rotate the features of the tissue image, is the sigmoid function, is the rotation weight matrix, is the rotation bias vector, is the tissue image feature, For convolutional neural network operations, This is a vector concatenation operation.

[0108] Traditional methods often rely on a single algorithm or model when processing tissue image feature extraction, which makes it difficult to simultaneously meet the requirements of rotation, scale invariance, and computational efficiency. The present invention constructs a tissue feature extraction model, combining the advantages of multiple algorithms such as SIFT (Scale Invariant Feature Transform) and ResNet, and realizes feature extraction of four-channel tissue image data. Since a tissue needs to be rotated before it can be folded again after being folded four times, Figure 4 As shown in the figure, the SIFT algorithm, with its high rotation and scale invariance, performs well in extracting key points and feature description vectors of tissue images. Therefore, SIFT feature point extraction is used to retain the features of tissue images before and after rotation. ResNet effectively alleviates the problems of gradient vanishing and gradient explosion in deep network training by introducing residual connections. The multi-algorithm fusion strategy enables the tissue feature extraction model to capture the local details of the image and understand the overall structure and rotation characteristics of the image. It provides rich and accurate information support for subsequent tissue geometric feature calculation, rotation angle deviation inspection, crease position detection and other steps.

[0109] Step 3: Use the YOLOv8 model to perform target recognition on the preprocessed tissue image data, and calculate the tissue geometric features according to the edge contour of the tissue recognition frame; after splicing the tissue image features with the tissue geometric features, input them into the VGGNet network, calculate the feature comparison vector, and output whether there is a rotation deviation in the tissue rotation angle according to the feature comparison vector;

[0110] The step three includes the following contents:

[0111] Step 301: Use the YOLOv8 model to perform target recognition on the preprocessed tissue image data, and calculate the center point coordinates and direction angle of the tissue target according to the edge contour of the tissue recognition frame as the tissue geometric features. The calculation method is:

[0112] ;

[0113] ;

[0114] ;

[0115] ;

[0116] in, They are The horizontal and vertical coordinates of the upper left and lower right vertices of the tissue recognition box detected by the model. For YOLOv8 model operation, is the preprocessed tissue image data, are the horizontal and vertical coordinates of the center point of the paper towel target, is the direction angle of the tissue target, is the fully connected layer operation, Harris corner detection operation;

[0117] Traditional geometric methods for calculating direction angles usually rely on the geometric features of the image, such as image edges and straight lines, which is more effective when the image quality is high. However, in practical applications, tissue images may be affected by multiple factors such as lighting changes, shadows, and complex textures, making it difficult for traditional geometric methods to accurately extract and identify the necessary geometric features. Inputting geometric features such as corner points and center points into the deep learning model and then calculating the image direction angle in a model-predicted manner can better adapt to these complex scenes with the help of complex features in a large amount of sample data, thereby improving the calculation effect.

[0118] For example, some tissues may have wrinkles, pulp blocks or stains during the manufacturing or transportation process, which are manifested as irregular texture changes in the image, resulting in incorrect corner point recognition. The orientation angle of the tissue image calculated using traditional geometric methods is 35°. However, through deep learning, it is found that the distance between the bounding box and the center point is too close, so this corner point is not the corner point of the tissue boundary, and this data point cannot be used to calculate the orientation angle of the tissue image. Combined with the calculation of the fully connected layer, it is found that this tissue image is consistent with other tissue images except for the corner points, so the orientation angle is correctly calculated to be 45°, there is no rotation deviation, and there is no need to adjust the PID rotation adjustment parameters.

[0119] In addition, during the tissue manufacturing process, due to machine failure or raw material problems, some tissue edges may have irregular wrinkles or bulges, which appear as significant changes in the edge contour, especially the curvature, in the image. Traditional direction angle calculation methods may not be able to accurately capture these changes, resulting in deviations in direction angle calculation, which in turn affects subsequent processes.

[0120] Therefore, when the accuracy requirement is very high, it is necessary to sample the points on the edge of the paper towel based on the sampling strategy of edge features to obtain the number of all points of the edge contour, and then correct the direction angle of the paper towel target calculated in step 301. The calculation method is:

[0121] ;

[0122] ;

[0123] ;

[0124] in, is the curvature feature, is the curvature of the i-th point on the edge contour, is the curvature weight of the ith point, is the number of all points of the edge contour, i is the curvature point index, is the Euclidean distance from the i-th point to the closer end of the edge contour, is the Euclidean distance from the i-th point to the farther end of the edge contour, β is a positive real number used to adjust the sensitivity of the weight to the change of distance. is the orientation angle of the corrected tissue target.

[0125] Step 302: splice the tissue image features and tissue geometric features, input them into the VGGNet network, perform feature comparison with the preset rotated tissue image, obtain a feature comparison vector, and output whether there is a rotation deviation in the tissue rotation angle according to the feature comparison vector. The calculation method is: ;

[0126] ;

[0127] ;

[0128] ;

[0129] ;

[0130] in, Overlay features on the tissue image. For VGGNet network operation, Overlay features on the preset rotated tissue image. is the orientation angle of the preset rotated tissue target, are the horizontal and vertical coordinates of the center point of the preset rotated tissue target, is the preset rotated tissue image feature, is the feature comparison vector, is the rotation deviation probability, is the rotation deviation parameter matrix, is the rotation deviation bias vector; is the result of whether there is rotation deviation, For the existence of rotational deviation, There is no rotation bias; the VGGNet network adopts a weight sharing mechanism;

[0131] The full name of VGG network is Visual Geometry Group Network, which is a deep convolutional neural network architecture developed by the Visual Geometry Group (VGG) research team at the University of Oxford;

[0132] The tissue image features are superimposed with the tissue geometric features and input into the weight-shared VGGNet convolutional neural network. The weight-shared network is equivalent to giving the model prior knowledge. When learning the superimposed features of tissue images, it already has preset features as the basis for learning and comparison, which can better capture the subtle differences in tissue images and provide feature vectors for the subsequent rotational deviation adjustment mechanism.

[0133] Step 4: If there is a deviation in the rotation angle of the paper towel, the deviation angle is calculated, and the PID rotation adjustment parameter is output to the PID control system to correct the deviation angle;

[0134] The step 4 includes the following contents:

[0135] Step 401: If there is a deviation in the rotation angle of the paper towel, the deviation angle is calculated according to the feature comparison vector. The calculation method is:

[0136] ;

[0137] in, is the deviation angle;

[0138] Compared with the traditional method of directly subtracting angles, the present invention combines the feature comparison vector after angle subtraction and outputs the angle deviation in the form of full connection layer prediction, which is more robust and avoids the angle calculation error caused by inaccurate corner point feature extraction as much as possible;

[0139] Step 402: construct a PID parameter adjustment model, calculate the PID rotation adjustment feature according to the deviation angle and the feature comparison vector, and then calculate the PID rotation adjustment parameter. The calculation method is:

[0140] ;

[0141] ;

[0142] in, is the PID rotation regulation characteristic, is the hyperbolic tangent function, is the PID proportional gain, is the proportional gain weight, is the proportional gain bias term, is the PID integral gain, is the integral gain weight, is the integral gain bias term, is the PID differential gain, is the differential gain weight, is the differential gain bias term; the PID proportional gain, PID integral gain and PID differential gain together constitute the PID rotation adjustment parameter;

[0143] By taking the deviation angle feature comparison vector as input, the PID parameter adjustment model can dynamically adjust the parameters of the PID controller according to the actual state of the current system (i.e., the deviation angle of the paper towel and the image feature comparison result); this dynamic adaptability enables the control system to respond to various changes more flexibly and improve the stability and response speed of the system; and the weights in the PID parameter adjustment model and the bias term It can be optimized through training; therefore, as the system runs and data accumulates, the parameters of the PID controller can be gradually adjusted to the optimal state to further improve control performance;

[0144] The calculation process of the PID parameter adjustment model is uniformly expressed as:

[0145] ;

[0146] in, is the PID rotation adjustment parameter, Adjust model operation for PID parameters;

[0147] Step 403: Transmit the PID rotation adjustment parameter to the PID control system to correct the deviation angle.

[0148] Step 5: Construct a crease position detection model, calculate the crease position based on the tissue image features, and compare it with the expected crease position to check whether there is a crease deviation; if there is a crease deviation, correct it through the PID control system;

[0149] The step five includes the following contents:

[0150] Step 501: construct a fold position detection model, input the tissue image features into the model to calculate the fold position, and output the probability that the fold position of the tissue has a deviation, and the calculation method is:

[0151] ;

[0152] ;

[0153] ;

[0154] in, is the crease position image calculated by the Unet network, For UNet network operations, is the estimated fold position image, is the probability that the fold position of the paper towel is biased;

[0155] As the number of folds increases, the thickness of the paper towel gradually increases, and direct folding may become difficult and inaccurate, so it is necessary to perform pre-folding before folding, as shown, and it is difficult to accurately control the crease position of the pre-folding with the traditional method;

[0156] The present invention adopts UNet network for image segmentation. UNet network, with its unique U-shaped structure and jump connection, can accurately capture the detailed information in the image, especially the subtle features such as creases, so as to achieve high-precision positioning of the crease position. After high-precision positioning of the crease position, the present invention performs feature fusion and intelligent judgment on the actual crease position image and the expected crease position image through the fully connected layer activated by the sigmoid function, and outputs the probability of deviation of the crease position.

[0157] Step 502: If the probability of deviation of the fold position of the paper towel is greater than 0.3, the fold deviation feature is calculated, and the fold deviation feature is input into the PID parameter adjustment model, and the PID fold adjustment parameter is output and transmitted to the PID control system for parameter adjustment to correct the fold deviation. The calculation method is: ;

[0158] ;

[0159] in, is the crease deviation feature, For channel superposition operation, It is the PID crease adjustment parameter;

[0160] Step 503: If the probability that there is a deviation in the fold position of the paper towel is ≤ 0.3, it is considered that there is no fold position deviation, and the paper towel folding continues.

[0161] For example, after the paper towel has been folded four times and then rotated, the thickness of the paper towel has increased significantly. If the paper towel is folded directly, it may deviate during the folding process, resulting in an unsatisfactory folding effect. To avoid this, a pre-crease mechanism with a V-shaped folding knife is used, so that the paper towel can be accurately folded into the expected crease by the V-shaped folding knife after being folded and conveyed by the roller, thereby enhancing the stability and accuracy of the paper towel in the subsequent folding process. Figure 5 As shown;

[0162] However, due to mechanical vibration, uneven conveyor belt speed or human error, the position of the paper towel is offset. In order to detect and correct this offset in time, the system performs real-time detection of the paper towel that is about to enter the pre-crease mechanism based on the constructed crease position detection model.

[0163] Specifically, the crease position detection model uses the advantages of the UNet network in image segmentation and feature extraction to compare the features of the current crease position image of the paper towel with the expected crease position image, and identifies the difference between the actual crease position on the paper towel and the theoretical crease position; after being processed by the fully connected layer activated by the sigmoid function, the system finally outputs the probability of the crease position deviation as 0.7, which is significantly larger than the threshold value of 0.3 set by the system;

[0164] Therefore, the system determines that there is a significant deviation in the current crease position of the paper towel, which needs to be corrected immediately; next, the system calculates the corresponding PID crease adjustment parameters through the PID parameter adjustment model based on the crease deviation characteristics, and transmits them to the PID control system to adjust the relevant parameters in the pre-crease mechanism, such as the position, angle, and pressure of the V-shaped folding knife, to ensure that in the subsequent folding process, the paper towel can be accurately folded according to the expected crease position.

[0165] Embodiment 2: A paper stacking control system based on machine vision provided by the present invention comprises:

[0166] A paper towel data preprocessing unit: collects and preprocesses paper towel data to obtain a grayscale image and a binary image in sequence, and then stores the paper towel data, the grayscale image, and the binary image as the first, second, and third channels respectively to obtain preprocessed paper towel image data;

[0167] Paper towel feature extraction unit:

[0168] According to the binary image channel in the preprocessed tissue image data, the edge image is calculated, and the edge image is used as the fourth channel to perform channel superposition with the preprocessed tissue image data to obtain four-channel tissue image data; then, a tissue feature extraction model is constructed, and the four-channel tissue image data is input into the model to obtain tissue image features;

[0169] Rotation angle detection unit: The YOLOv8 model is used to perform target recognition on the preprocessed tissue image data, and the geometric features of the tissue are calculated based on the edge contour of the tissue recognition frame; the tissue image features and the tissue geometric features are spliced ​​and input into the VGGNet network to calculate the feature comparison vector, and the output is based on the feature comparison vector to determine whether there is a rotation deviation in the tissue rotation angle;

[0170] Rotation deviation correction unit: If there is a deviation in the rotation angle of the paper towel, the deviation angle is calculated and the PID rotation adjustment parameter is output to the PID control system to correct the deviation angle;

[0171] Crease position detection and correction unit: Build a crease position detection model, calculate the crease position based on the tissue image features, and compare it with the expected crease position to check whether there is a crease deviation; if there is a crease deviation, correct it through the PID control system.

[0172] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired means (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available media can be magnetic media (for example, floppy disks, hard disks, tapes), optical media (for example, DVD ), or semiconductor media. The semiconductor media may be a solid state drive.

[0173] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0174] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0175] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only some logical function divisions. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0176] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0177] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0178] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U Disk, mobile hard disk, read-only memory ( read - onlymemory , ROM ), Random Access Memory ( RandomAccessMemory , RAM ), magnetic disks, optical disks, and other media that can store program codes.

[0179] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0180] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A paper stacking control method based on machine vision, characterized in that: include: S1, collecting and preprocessing paper towel data to obtain a grayscale image and a binary image in sequence, and then storing the paper towel data, grayscale image, and binary image as the first, second, and third channels respectively to obtain preprocessed paper towel image data; S2. Calculate an edge image according to the binary image channel in the preprocessed tissue image data, and use the edge image as the fourth channel to perform channel superposition with the preprocessed tissue image data to obtain four-channel tissue image data; then, construct a tissue feature extraction model, input the four-channel tissue image data into the model, and obtain tissue image features; The calculation method of the paper towel feature extraction model is: ； ； ； ； in, is the key point of the tissue image, is the feature description vector of the key points of the tissue image, It is the SIFT feature point extraction operation. They are the data of the first, third, and fourth channels in the four-channel tissue image data. is the convolution feature of the tissue image, is the residual convolutional network operation, is the data of the second channel in the four-channel tissue image data, Rotate the features of the tissue image, is the sigmoid function, is the rotation weight matrix, is the rotation bias vector, is the tissue image feature, For convolutional neural network operations, It is a vector concatenation operation; S3. Use the YOLOv8 model to perform target recognition on the preprocessed tissue image data, and calculate the tissue geometric features according to the edge contour of the tissue recognition frame; after splicing the tissue image features with the tissue geometric features, input them into the VGGNet network, calculate the feature comparison vector, and output whether there is a rotation deviation of the tissue rotation angle according to the feature comparison vector; S4. If there is a deviation in the rotation angle of the paper towel, the deviation angle is calculated, and the PID rotation adjustment parameter is output to the PID control system to correct the deviation angle; S5. Construct a crease position detection model, calculate the crease position based on the characteristics of the tissue image, and compare it with the expected crease position to check whether there is a crease deviation; if there is a crease deviation, correct the crease deviation through the PID control system; The step S5 comprises: Construct a crease position detection model, input the tissue image features into the crease position detection model to calculate the crease position, and output the probability that the crease position of the tissue is deviated. The calculation method is: ； ； ； in, is the crease position image calculated by the Unet network, For UNet network operations, is the estimated fold position image, is the probability that the fold position of the paper towel is biased; is the tissue image feature, is the preset rotated tissue image feature, It is a fully connected layer operation; If the probability of deviation of the fold position of the paper towel is > 0.3, the fold deviation characteristics are calculated and input into the PID parameter adjustment model, the PID fold adjustment parameters are output and transmitted to the PID control system for parameter adjustment to correct the fold deviation. The calculation method is: ; ； in, is the crease deviation feature, For channel superposition operation, is the PID crease adjustment parameter, To adjust the model operation for PID parameters; If the probability that there is a deviation in the fold position of the paper towel is ≤ 0.3, it is considered that there is no fold position deviation and the paper towel folding continues.

2. The paper stacking control method based on machine vision according to claim 1, characterized in that: The step S1 comprises: Collecting paper towel data, wherein the paper towel data is a color image obtained by photographing the paper towels on a conveyor belt at a vertical viewing angle using a high-resolution camera; The paper towel data is grayed out using the weighted average method and Gaussian filtering to convert the paper towel data into a grayscale image. The grayscale image is subjected to Otsu's method to calculate the inter-class variance of the grayscale image histogram and output the segmentation threshold, and then the grayscale image is converted into a binary image according to the segmentation threshold; the conversion rule is: pixels with grayscale values ​​greater than or equal to the segmentation threshold are set to white, and pixels with grayscale values ​​less than the threshold are set to black; The paper towel data, grayscale image and binary image are stored as the first, second and third channels respectively to obtain the preprocessed paper towel image data.

3. The paper stacking control method based on machine vision according to claim 2, characterized in that: In the step S2, the edge image is calculated according to the binary image channel in the preprocessed tissue image data, including: The Canny edge detection algorithm is used to perform edge detection on the binary image channel in the preprocessed tissue image data, and then the edge image is obtained through non-maximum suppression and double threshold processing.

4. The paper stacking control method based on machine vision according to claim 3, characterized in that: The YOLOv8 model is used to perform target recognition on the preprocessed tissue image data, and the tissue geometric features are calculated according to the edge contour of the tissue recognition frame, including: The YOLOv8 model is used to perform target recognition on the preprocessed tissue image data, and the center point coordinates and direction angles of the tissue target are calculated according to the edge contour of the tissue recognition frame as the tissue geometric features. The calculation method is: ； ； ； ； in, They are The horizontal and vertical coordinates of the upper left and lower right vertices of the tissue recognition box detected by the model. For YOLOv8 model operation, is the preprocessed tissue image data, are the horizontal and vertical coordinates of the center point of the paper towel target, is the direction angle of the tissue target, is the fully connected layer operation, It is the Harris corner detection operation; After the tissue image features and tissue geometric features are spliced, they are input into the VGGNet network, feature comparison vectors are calculated, and whether there is a rotation deviation in the tissue rotation angle is output according to the feature comparison vectors, including: The tissue image features and tissue geometric features are spliced ​​and input into the VGGNet network. The features are compared with the preset rotated tissue image to obtain the feature comparison vector. The feature comparison vector is used to output whether there is a rotation deviation in the tissue rotation angle. The calculation method is as follows: ； ； ； ； ； in, Overlay features on the tissue image. For VGGNet network operation, Overlay features on the preset rotated tissue image. is the orientation angle of the preset rotated tissue target, are the horizontal and vertical coordinates of the center point of the preset rotated tissue target, is the tissue image feature, is the preset rotated tissue image feature, is the feature comparison vector, is the rotation deviation probability, is the rotation deviation parameter matrix, is the rotation deviation bias vector; is the result of whether there is rotation deviation, For the existence of rotational deviation, To avoid rotation bias, the VGGNet network adopts a weight sharing mechanism.

5. The paper stacking control method based on machine vision according to claim 4, characterized in that: The step S4 comprises: If there is a deviation in the rotation angle of the paper towel, the deviation angle is calculated according to the feature comparison vector. The calculation method is: ； in, is the deviation angle; Construct a PID parameter adjustment model, calculate the PID rotation adjustment feature according to the deviation angle and feature comparison vector, and then calculate the PID rotation adjustment parameter. The calculation method is: ； ； in, is the PID rotation regulation characteristic, is the hyperbolic tangent function, is the PID proportional gain, is the proportional gain weight, is the proportional gain bias term, is the PID integral gain, is the integral gain weight, is the integral gain bias term, is the PID differential gain, is the differential gain weight, is the differential gain bias term; the PID proportional gain, PID integral gain and PID differential gain together constitute the PID rotation adjustment parameter; The calculation process of the PID parameter adjustment model is uniformly expressed as: ； in, is the PID rotation adjustment parameter, To adjust the model operation for PID parameters; The PID rotation adjustment parameters are transmitted to the PID control system to correct the deviation angle.

6. A paper stacking control system based on machine vision, characterized in that: include: A paper towel data preprocessing unit: collects and preprocesses paper towel data to obtain a grayscale image and a binary image in sequence, and then stores the paper towel data, the grayscale image, and the binary image as the first, second, and third channels respectively to obtain preprocessed paper towel image data; Paper towel feature extraction unit: An edge image is calculated according to the binary image channel in the preprocessed tissue image data, and the edge image is used as the fourth channel to perform channel superposition with the preprocessed tissue image data to obtain four-channel tissue image data; Then, a paper towel feature extraction model is constructed, and the four-channel paper towel image data is input into the model to obtain the paper towel image features; Rotation angle detection unit: calculates geometric features of the tissue paper based on the preprocessed tissue paper image data; And combined with the image features and geometric features of the paper towel, check whether there is deviation in the rotation angle of the paper towel; Rotation deviation correction unit: The YOLOv8 model is used to perform target recognition on the preprocessed tissue image data, and the tissue geometric features are calculated according to the edge contour of the tissue recognition frame; the tissue image features and tissue geometric features are spliced ​​and input into the VGGNet network to calculate the feature comparison vector, and the output of the tissue rotation angle is based on the feature comparison vector to determine whether there is a rotation deviation; Crease position detection and correction unit: Build a crease position detection model, calculate the crease position based on the characteristics of the tissue image, and compare it with the expected crease position to check whether there is a crease deviation; if there is a crease deviation, correct it through the PID control system; To realize the paper stacking control method based on machine vision as described in any one of claims 1-5.

Images