Tissue package AI detection method based on image analysis

Abstract

The invention discloses a tissue package AI detection method based on image analysis, and relates to the technical field of tissue package detection, and the method comprises the following steps: S1, detecting tissue surface defect points, S2, confirming the packaging state of a packaging bag, S3, dividing a carton contour surface, S4, constructing a detection model, S5, outputting a carton damage position, and S6, carrying out auxiliary early warning prompt. According to the method, after the image data of the carton storing the paper towels in the stacked state is extracted, all carton contour surface position information contained in each piece of image data is accurately marked, the position detection model trained by the data has better performance, and once the carton image data is received, the paper towels can be accurately detected. According to the method, the contour surface corresponding to each carton can be analyzed in the first time, the corresponding image data can be conveniently extracted according to the position of the contour surface subsequently, the package damage condition of each carton in the image can be rapidly analyzed by using the damage detection model, and the multiple cartons are detected at the same time, so that the time is saved, and the method is convenient and rapid.

Description

Technical Field

[0001] The present invention relates to the technical field of tissue packaging detection, and specifically to an AI detection method for tissue packaging based on image analysis. Background Art

[0002] The AI detection system for tissue packaging can analyze the subtle changes on the surface of tissues, packaging bags, and cartons in real time, quickly identify various defects, automatically trigger a warning mechanism, and provide support for the immediate adjustment of the production line. In the invention patent with the application number 202110838883.3, "A method for detecting the packaging of tissue products based on deep learning" is disclosed, including the following steps: using the pictures of five faces of tissue boxes with various different packages to train the network model corresponding to each face respectively, to obtain the final network model corresponding to each face; when actually conducting the packaging production detection of a certain type of tissue box, input the five-face images of the standard tissue box sample and the actual tissue box into the corresponding network model of each face respectively, to obtain the standard feature parameters and the actual feature parameters, calculate the Euclidean distance β between the actual feature parameters and the standard feature parameters of the same face. If β is less than the preset threshold α, it is determined that the packaging of the tissue box on this face is intact; if β is greater than the preset threshold α, it is determined that the packaging of the tissue box on this face is damaged. The present invention has the advantages of high recognition efficiency, adjustable precision, convenient secondary deployment, etc., can detect various tissue box packages, and has portability and wide applicability.

[0003] The above-mentioned prior art solves problems such as completely relying on manual labor for tissue packaging detection. However, in the process of execution, this method needs to detect each tissue packaging box one by one, which not only wastes a lot of time, but also the re-placement during the handling process will affect the production efficiency. At the same time, this method completely relies on manual labor for information annotation when constructing the sample set, which greatly increases the labor cost, and also cannot detect the tissues themselves, resulting in the inability of operators to screen out unqualified tissue products in a short time. Summary of the Invention

[0004] The purpose of the present invention is to provide an AI detection method for tissue packaging based on image analysis to solve the problems raised in the above background art.

[0005] To achieve the above purpose, the present invention provides the following technical solution: An AI detection method for tissue packaging based on image analysis, including the following steps:

[0006] S1. Detect surface defect points of the tissue paper: After obtaining the image data of the surface of the tissue paper to be measured, divide it into multiple sub-regions, determine the binary image data and the convolved image data of each sub-region, screen out all pixel points that meet the gray value conditions in the sub-region according to these two types of image data, divide the target region and the background region in the convolved image data according to the position coordinates of these pixel points, connect the pixel points with the same pixel value and adjacent positions in the target region, and use the connected region as the detection region. Set the pattern threshold, scan the detection region row by row, count all pixel points with gray values greater than the pattern threshold, determine the pattern region and the non-pattern region, delete the printed parts in each pattern region to obtain the non-printed region, count the gray values of all pixel points in the non-printed region and the non-pattern region, mark the pixel points with gray values lower than the mean value, screen out the defect points among the marked points, and output the position information of each defect point;

[0007] S2. Confirm the packaging status of the packaging bag: Collect thermal imaging images of the sealing parts of multiple tissue paper packaging bags, label the corresponding packaging status according to each thermal imaging image, construct a training set and a test set using the image data and the status information. After setting the decision tree parameters, use different parts of the data in the training set to complete the training of different decision trees. Transmit the data in the test set to each decision tree in turn, and update the prediction confidence corresponding to each status information using multiple output results. Train the SVM model using the data in the training set, analyze the prediction confidence corresponding to each status information according to the data in the test set. After extracting the thermal imaging image dataset of the sealing part of the tissue paper packaging bag, analyze it using the decision tree and the SVM model respectively. After determining the status information of all data, use the data in the dataset to complete the training of the CNN model. Transmit the current thermal imaging image of the sealing part of the tissue paper packaging bag to the CNN model for analysis, and output the packaging status information corresponding to the current image data;

[0008] S3. Divide the contour surfaces of the carton: Extract the image data of multiple cartons storing tissue paper in a stacked state, number and label the endpoints on the contour line of each carton in the image, determine the pixel coordinates of each endpoint, and then construct a directed weighted graph according to the labeling order of each endpoint. Use the selected endpoint as the starting point to search in the directed weighted graph, determine the shortest path back to the current starting point, and analyze the position information of all contour surfaces corresponding to the carton according to the shortest path;

[0009] S4. Build a detection model: After determining the position information of the contour surfaces corresponding to all cartons in each image data, the image data and the position information are transmitted to the ResNet network model. The target position is preliminarily predicted using a prediction analysis algorithm, the depth information of each image data is imported, and analysis is performed based on the preliminarily predicted target position and the depth information. The model parameters are updated through a loss function, and the operation is repeated until the model training is completed. The trained model is used as the position detection model. After obtaining the carton damage image dataset, these image data are used to train the ChostNet model, thereby building a damage detection model;

[0010] S5. Output the damaged position of the carton: After using the camera to take pictures of the stacked cartons within the specified position, the obtained image data and depth information are transmitted to the position detection model to obtain the position information of all the contour surfaces of each carton in the image. The corresponding image data is extracted according to the center point position of the contour surface and transmitted to the damage detection model to determine the damaged position and the center point position of the contour surface, and then output through a visualization interface;

[0011] S6. Auxiliary warning prompt: If the position information of the defect points on the surface of the tissue to be tested is received, the intelligent voice device is used to send a prompt message indicating that there are contaminants on the tissue. If the packaging status information of the tissue packaging bag is semi-packaged and sandwich-packaged, the voice device is used to send a prompt message indicating that the tissue is not fully packaged. If the relative position information of the damaged part of the carton is received, the voice device is used to send a prompt message indicating that the packaging is damaged. The intelligent voice device is specifically the Shouli voice prompt.

[0012] Preferably, the S1 includes the following steps:

[0013] S101. After obtaining the image data of the surface of the tissue to be tested, set a pixel threshold, perform a binarization operation on the image data according to the pixel threshold, and then use a one-dimensional Gaussian filter to perform convolution processing on each pixel value in the image to obtain the processed image data;

[0014] S102. Divide the overall area of the image into multiple sub-regions according to the brightness value, and set the gray threshold g according to the average brightness value within each sub-region to determine the binarized image data f i and the convolved image data g r,c of each sub-region. After calculating the difference between different pixel points at the same position row by row, set the brightness threshold b. If r,c then it is determined that the current sub-region belongs to the high-brightness region, and all pixel points that meet f -g r,c -g r,c ≥g i in this region are selected as target pixel points. If Then it is determined that the current sub-region belongs to the low-brightness region, and all pixel points within this region that meet g r,c -f r,c ≥g i are selected, and these pixel points are used as target pixel points;

[0015] S103. After statistically analyzing the position coordinates of all target pixel points in the image data, the target region and the background region in the convolved image data are selected according to the position coordinates of the target pixel points, so as to obtain the segmented image data.

[0016] Preferably, S1 further includes the following steps:

[0017] S104. After performing a series of operations on the segmented image data, such as connectivity, erosion, filling, and dilation, all pixel points in the target region of the image are scanned, and pixel points with the same pixel value and adjacent positions are connected. The connected region is used as the detection region. Feature parameters are selected, the feature value corresponding to each detection region is calculated, the feature range and the pattern threshold are set, and all detection regions whose feature values meet the range are output. Each detection region is scanned row by row, and all pixel points with a gray value greater than the pattern threshold are statistically analyzed. According to the regions where these pixel points are located, the pattern region and the non-pattern region are analyzed;

[0018] S105. After extracting the complete printing data from the database, the printed parts in each pattern region are deleted according to the printing data to obtain the non-printed region;

[0019] S106. Set the neighborhood size to 3×3, determine the gray values of all pixel points in the non-printed region and the non-pattern region, mark the pixel points with gray values lower than the average value, and statistically analyze the number of marked points in the neighborhood around each marked pixel point. If the number is greater than or equal to 3, then this point is determined as a defective point. After analyzing all marked points, the position information of each defective point is output.

[0020] Preferably, S2 specifically includes the following steps:

[0021] S201. Collect multiple thermal imaging images of the sealing part of the paper towel packaging bag, and label the corresponding packaging states according to each thermal imaging image. The packaging states include normal, semi-packaging, and sandwich packaging. 60 image data of each packaging state are stored in the sample set;

[0022] S202. After sequentially extracting the color features corresponding to each image data in the sample set, the color features are combined with the state information, and 40 are stored in the training set according to the state information, and the remaining 20 are stored in the test set;

[0023] After setting the number, depth, and coefficient of the decision tree, randomly sample in the training set to construct multiple different subsets. Use different subsets to complete the training of different decision trees. Transmit the data in the test set to each decision tree in turn. After obtaining multiple output results, use the variable analysis algorithm to calculate the final result based on different output results. Update the prediction confidence corresponding to each state information using the deviation value between the final result and the true state information. The variable analysis algorithm is specifically as follows:

[0024]

[0025] Among them, C′(x) represents the final output result, k represents the total number of decision trees, and C i (x) represents the output result of a single decision tree, y represents the output variable, I(*) represents the sex function, and i represents the parameter.

[0026] Preferably, the S2 specifically further includes the following steps:

[0027] S204. Use the data in the training set to train the SVM model. After determining the parameters and weights of the model, analyze the prediction confidence corresponding to each state information according to the data in the test set;

[0028] S205. After extracting the thermal imaging map dataset at the sealing position of the tissue paper packaging bag, use the decision tree and the SVM model for analysis respectively. If the state information output by the two is the same, automatically label the corresponding packaging state for the current image data. Otherwise, compare the sizes of the corresponding prediction confidences of the two, and take the result of the model with the higher confidence as the packaging state corresponding to the current image data for labeling. Repeat the operation until all the data in the dataset are labeled;

[0029] S206. After training the CNN model using the image data in the dataset, use a thermal imager to take a picture of the sealing position of the current tissue paper packaging bag. After obtaining the thermal imaging map, transmit it to the CNN model for analysis, and output the packaging state information corresponding to the current image data.

[0030] Preferably, the S3 specifically includes the following steps:

[0031] S301. Extract the image data of multiple stacked cartons containing tissue paper. Use the LableMe annotation tool to number and annotate the endpoints on the contour line of each carton in the image according to preset rules. After determining the pixel coordinates of each endpoint, store them in the endpoint set;

[0032] S302. Set the scale radius, select any one endpoint as the target point, calculate the distance values between the current target point and other endpoints in the set. If there exists a distance value less than or equal to the scale radius, determine the endpoint corresponding to the distance value as a similar point; otherwise, determine it as a non-similar point.

[0033] S303. After using the center point analysis algorithm to filter out all similar points around the current target point, calculate the center point position coordinates based on the position coordinates of the target point and the similar points, and replace the original target point and similar point numbers with the center point and store them in the set. The specific center point analysis algorithm is as follows:

[0034]

[0035]

[0036] Among them, d(p c , p i ) represents the distance value, p c represents the target point, p i represents the similar point, (x c , y c ) represents the pixel coordinates of the target point p c , (x i , y i ) represents the pixel coordinates of the similar point p i , δ represents the scale radius, p a represents the center point, (x′, y′) represents the pixel coordinates of the center point, n represents the total number of similar points, represents the accumulated value of the abscissas of all similar points, represents the accumulated value of the ordinates of all similar points;

[0037] S304. Construct a corresponding relationship matrix according to the labeling order of each endpoint by the LableMe labeling tool, draw a directed weighted graph using the relationship matrix, count the number of endpoints that pass through two contour lines in the set, use these endpoints as starting points, search in the directed weighted graph to determine the shortest path back to the current starting point, and extract the position information of all contour surfaces corresponding to each carton according to the endpoint numbers included in each shortest path.

[0038] Preferably, the S4 specifically includes the following steps:

[0039] S401. After determining the position information of all contour surfaces corresponding to each carton in each image data, perform an association operation on this information with the image data in the form of a label and transmit it to the ResNet network model. Obtain the feature maps with sizes of 13×13, 26×26, and 52×52 corresponding to each image data through multiple convolutional operations.

[0040] S402. After determining the corresponding prediction box size according to the size of the feature map, use the prediction analysis algorithm to make a preliminary prediction of the target position, import the depth information of each image data, perform a concat fusion operation on the depth information and the feature image, and then analyze according to the preliminary predicted target position and the fusion result to obtain the prediction result. After determining the position coordinates of the center points of each contour surface of the carton using the label information corresponding to the image data, update the model parameters according to the deviation value between the prediction result and the true position coordinates through the loss function, and repeat the operation until the model training is completed. Use the trained model as the position detection model;

[0041] S403. After obtaining the carton damage image dataset, take 80% of the image data as the training set, and the remaining image data as the validation set. Use the training set and the validation set to train the ChostNet model, update the model parameters and weights, so as to construct a damage detection model.

[0042] Preferably, the S5 specifically includes the following steps:

[0043] S501. After using the camera to take pictures of the stacked cartons within the specified position, transmit the obtained image data and depth information to the database;

[0044] S502. Use the position detection model to analyze the positions of all contour surfaces of each carton in the current image according to the image data and the depth information, extract the corresponding image data according to the position of the center point of the contour surface, and transmit it to the damage detection model;

[0045] S503. Judge whether the current carton is damaged according to the model output result. If it is determined that there is no damage, extract the image data of other contour surfaces of the carton for analysis. If it is determined that there is damage, determine the relative position of the damaged part of the carton according to the damaged position and the position of the center point of the contour surface, and output it through the visualization interface. Repeat the operation until all the image data of all the cartons in the image data are processed.

[0046] Compared with the prior art, the beneficial effects of the present invention are:

[0047] 1. The present invention analyzes the surface image of the tissue paper, performs image segmentation layer by layer to ensure that the obtained pattern area and non-pattern area are more complete and distinct. At the same time, after determining the pattern area and non-pattern area, the printed part in the pattern area is deleted to prevent the interference of printed data on defect recognition. During the detection process, once a defect point is found, it is immediately fed back to the user interface to facilitate the operator to screen out unqualified tissue paper products. Meanwhile, the thermal imaging images of the sealing parts of a small number of tissue paper packaging bags are manually labeled, and the decision tree and SVM models are trained using these small amounts of data, so that the data set containing a large number of thermal imaging images no longer requires manual addition of information. The trained decision tree and SVM models are directly used to automatically label these data, and the CNN model is trained through the labeled data set. This design can greatly reduce the labor cost on the one hand and give full play to the respective advantages of different models on the other hand, making the result of the sealing analysis more accurate.

[0048] 2. After extracting the image data of the cartons storing tissue paper in a stacked state, the present invention accurately labels the position information of all the carton contour surfaces contained in each image data. The position detection model trained using these data has better performance. Once receiving the carton image data, it can analyze the corresponding contour surface of each carton in the first time, which is convenient for subsequent extraction of the corresponding image data according to the position of the contour surface. The damage detection model can quickly analyze the packaging damage situation of each carton in the image. Since multiple cartons storing tissue paper are often placed in the same place during the packing process and detected simultaneously, it can not only save time but also be more convenient and fast. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flowchart of the overall method provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0051] Please refer to Figure 1 , the present invention provides a technical solution: an AI detection method for tissue paper packaging based on image analysis, including the following steps:

[0052] S1. Detect surface defect points of the tissue paper: After obtaining the image data of the surface of the tissue paper to be tested, divide it into multiple sub-regions, determine the binary image data and the convolved image data of each sub-region, screen out all pixel points that meet the gray value conditions in the sub-region according to these two types of image data, divide the target region and the background region in the convolved image data according to the position coordinates of these pixel points, connect the pixel points with the same pixel value and adjacent positions in the target region, and use the connected region as the detection region. Set the pattern threshold, scan the detection region row by row, count all pixel points with gray values greater than the pattern threshold, determine the pattern region and the non-pattern region, delete the printed parts in each pattern region to obtain the non-printed region, count the gray values of all pixel points in the non-printed region and the non-pattern region, mark the pixel points with gray values lower than the mean value, screen out the defect points among the marked points, and output the position information of each defect point;

[0053] S2. Confirm the packaging status of the packaging bag: Collect thermal images of the sealed parts of multiple tissue paper packaging bags, label the corresponding packaging status according to each thermal image, use the image data and status information to construct a training set and a test set. After setting the decision tree parameters, use different parts of the data in the training set to complete the training of different decision trees. Transmit the data in the test set to each decision tree in turn, and use multiple output results to update the prediction confidence corresponding to each status information. Train the SVM model with the data in the training set, analyze the prediction confidence corresponding to each status information according to the data in the test set. After extracting the thermal image dataset of the sealed part of the tissue paper packaging bag, use the decision tree and the SVM model for analysis respectively. After determining the status information of all data, use the data in the dataset to complete the training of the CNN model. Transmit the current thermal image of the sealed part of the tissue paper packaging bag to the CNN model for analysis, and output the packaging status information corresponding to the current image data;

[0054] S3. Divide the contour surfaces of the carton: Extract the image data of multiple cartons storing tissue paper in a stacked state, number and label the endpoints on the contour line of each carton in the image, determine the pixel coordinates of each endpoint, and then construct a directed weighted graph according to the labeling order of each endpoint. Use the selected endpoint as the starting point to search in the directed weighted graph, determine the shortest path back to the current starting point, and analyze the position information of all contour surfaces corresponding to the carton according to the shortest path;

[0055] S4. Build a detection model: After determining the position information of the contour surfaces corresponding to all cartons in each image data, transmit the image data and the position information to the ResNet network model. Use the prediction analysis algorithm to make a preliminary prediction of the target position, import the depth information of each image data, analyze based on the preliminarily predicted target position and the depth information, update the model parameters through the loss function, and repeat the operation until the model training is completed. Use the trained model as the position detection model. After obtaining the carton damage image data set, use these image data to train the ChostNet model, thereby building a damage detection model;

[0056] S5. Output the damaged position of the carton: After using the camera to take pictures of the stacked cartons within the specified position, transmit the obtained image data and depth information to the position detection model to obtain the position information of all the contour surfaces of each carton in the image. Extract the corresponding image data according to the center point position of the contour surface and transmit it to the damage detection model to determine the damaged position and the center point position of the contour surface, and output it through the visualization interface;

[0057] S6. Auxiliary warning prompt: If the position information of the defect points on the surface of the to-be-tested tissue is received, use the intelligent voice device to send a prompt message indicating that there are pollutants on the tissue. If the packaging status information of the tissue packaging bag is semi-packaging and sandwich packaging, use the voice device to send a prompt message indicating that the tissue is not fully packaged. If the relative position information of the damaged part of the carton is received, use the voice device to send a prompt message indicating that the packaging is damaged. The intelligent voice device is specifically the Shouli voice prompt.

[0058] S1 includes the following steps:

[0059] S101. After obtaining the image data of the surface of the to-be-tested tissue, set the pixel threshold, perform binary operation on the image data according to the pixel threshold, and then use a one-dimensional Gaussian filter to perform convolution processing on each pixel value in the image to obtain the processed image data;

[0060] S102. Divide the overall area of the image into multiple sub-regions according to the brightness value, and set the gray threshold g according to the average brightness value in each sub-region Determine the binary image data f i and the convolved image data g r,c of each sub-region after binary operation. Calculate the difference between different pixel points at the same position row by row, and then set the brightness threshold b. If r,c then judge that the current sub-region belongs to the high-brightness region, and screen out all pixel points in this region that meet f r,c -g r,c ≥g i ​​Then it is determined that the current sub-region belongs to the low-brightness region, and all pixel points within this region that meet g r,c -f r,c ≥g i are selected, and these pixel points are used as target pixel points;

[0061] S103. After statistically obtaining the position coordinates of all target pixel points in the image data, the target region and the background region in the convolved image data are selected according to the position coordinates of the target pixel points, so as to obtain the segmented image data;

[0062] S1 further includes the following steps:

[0063] S104. After performing a series of operations on the segmented image data, such as connectivity, erosion, filling, and dilation, all pixel points in the target region of the image are scanned, and pixel points with the same pixel value and adjacent positions are connected. The connected region is used as the detection region. Feature parameters are selected, the corresponding feature values of each detection region are calculated, the feature range and the pattern threshold are set, and all detection regions whose feature values meet the range are output. The detection regions are scanned row by row, and all pixel points with a gray value greater than the pattern threshold are statistically counted. According to the regions where these pixel points are located, the pattern region and the non-pattern region are analyzed;

[0064] S105. After extracting the complete printing data from the database, the printed parts in each pattern region are deleted according to the printing data to obtain the non-printed region;

[0065] S106. Set the neighborhood size to 3×3, determine the gray values of all pixel points in the non-printed region and the non-pattern region, mark the pixel points with gray values lower than the mean value, and count the number of marked points in the neighborhood around each marked pixel point. If the number is greater than or equal to 3, then this point is determined as a defective point. After analyzing all the marked points, the position information of each defective point is output;

[0066] S2 specifically includes the following steps:

[0067] S201. Collect thermal imaging maps of the sealed parts of multiple paper towel packaging bags, and label the corresponding packaging states according to each thermal imaging map. The packaging states include normal, semi-sealed, and interlayer-sealed. 60 image data of each packaging state are stored in the sample set;

[0068] S202. After sequentially extracting the corresponding color features of each image data in the sample set, the color features are combined with the state information, and 40 are stored in the training set according to the state information, and the remaining 20 are stored in the test set;

[0069] S203. After setting the number, depth, and coefficient of the decision tree, randomly sample in the training set to construct multiple different subsets. Use different subsets to complete the training of different decision trees. Transmit the data in the test set to each decision tree in turn. After obtaining multiple output results, use the variable analysis algorithm to calculate the final result according to different output results. Update the prediction confidence corresponding to each state information using the deviation value between the final result and the true state information. The variable analysis algorithm is specifically as follows:

[0070]

[0071] Among them, C′(x) represents the final output result, k represents the total number of decision trees, C i (x) represents the output result of a single decision tree, y represents the output variable, I(*) represents the identity function, and i represents the parameter;

[0072] S2 specifically further includes the following steps:

[0073] S204. Use the data in the training set to train the SVM model. After determining the parameters and weights of the model, analyze the prediction confidence corresponding to each state information according to the data in the test set;

[0074] S205. After extracting the thermal imaging map dataset at the sealing part of the tissue paper packaging bag, use the decision tree and the SVM model for analysis respectively. If the state information output by both is the same, automatically label the corresponding packaging state for the current image data. Otherwise, compare the sizes of the corresponding prediction confidences of the two, and take the result of the model with the higher confidence as the packaging state corresponding to the current image data for labeling. Repeat the operation until all the data in the dataset are labeled;

[0075] S206. After training the CNN model using the image data in the dataset, use a thermal imager to take a picture of the sealing part of the current tissue paper packaging bag. After obtaining the thermal imaging map, transmit it to the CNN model for analysis and output the packaging state information corresponding to the current image data;

[0076] S3 specifically includes the following steps:

[0077] S301. Extract the image data of multiple stacked cartons containing tissue paper. Use the LableMe annotation tool to number and label the endpoints on the contour line of each carton in the image according to the preset rules. After determining the pixel coordinates of each endpoint, store them in the endpoint set;

[0078] S302. Set the scale radius, select any one endpoint as the target point, calculate the distance value between the current target point and other endpoints in the set. If there is a distance value less than or equal to the scale radius, determine that the endpoint corresponding to the distance value is a similar point. Otherwise, determine it as a non-similar point;

[0079] S303. After screening out all the similar points around the current target point using the center point analysis algorithm, calculate the position coordinates of the center point based on the position coordinates of the target point and the similar points, and store the center point as the new end point to replace the original target point and the similar point numbers in the set. The center point analysis algorithm is specifically as follows:

[0080]

[0081]

[0082] Among them, d(p c , p i ) represents the distance value, p c represents the target point, p i represents the similar point, (x c , y c ) represents the pixel coordinates of the target point p c , (x i , y i ) represents the pixel coordinates of the similar point p i , δ represents the scale radius, p a represents the center point, (x′, y′) represents the pixel coordinates of the center point, n represents the total number of similar points, represents the accumulated value of the abscissas of all similar points, represents the accumulated value of the ordinates of all similar points;

[0083] S304. Construct a corresponding relationship matrix according to the annotation order of each end point by the LableMe annotation tool, draw a directed weighted graph using the relationship matrix, count the number of end points that pass through two contour lines in the set, use these end points as the starting points, search in the directed weighted graph, determine the shortest path back to the current starting point, and extract the position information of all contour surfaces corresponding to each carton according to the end point numbers included in each shortest path;

[0084] S4 specifically includes the following steps:

[0085] S401. After determining the position information of all contour surfaces corresponding to each carton in each image data, perform an association operation on this information with the image data in the form of labels, and transmit it to the ResNet network model. Obtain the feature maps with sizes of 13×13, 26×26, and 52×52 corresponding to each image data through multiple convolutional operations;

[0086] S402. After determining the corresponding prediction box size according to the size of the feature map, use the prediction analysis algorithm to make a preliminary prediction of the target position. Import the depth information of each image data, perform a concat fusion operation on the depth information and the feature image, and then analyze according to the preliminary predicted target position and the fusion result to obtain the prediction result. After determining the position coordinates of the center points of each contour surface of the carton using the label information corresponding to the image data, update the model parameters through the loss function according to the deviation value between the prediction result and the true position coordinates, and repeat the operation until the model training is completed. Use the trained model as the position detection model. The prediction analysis algorithm is specifically as follows:

[0087]

[0088] Pr(Ob)*IOU(B,Ob)=σ(t0)

[0089] Among them, B x ,B y ,B w ,B h represents the prediction box parameter value, t x ,t y represents the offset of the prediction box relative to the center point of the true carton, t w ,t h represents the scale change amount of the prediction box relative to the width and height of the true carton contour, C x ,C y represents the grid offset, Pr(Ob) represents the boxed target, IOU(B,Ob) represents the IOU value between the prediction box and the true carton contour, B represents the prediction box, Ob represents the true carton contour, σ(t x ) represents the sigmoid function corresponding to the x-axis offset of the prediction box relative to the center point of the true carton, σ(t y ) represents the sigmoid function corresponding to the y-axis offset of the prediction box relative to the center point of the true carton, σ(t0) represents the sigmoid function corresponding to the confidence level output by the model, t0 represents the confidence level output by the model, p w represents the set width of the prediction box, p h represents the set length of the prediction box, and e represents the natural constant;

[0090] S403. After obtaining the carton damage image dataset, take 80% of the image data as the training set, and the remaining image data as the validation set. Use the training set and the validation set to train the ChostNet model, update the model parameters and weights, and thus construct a damage detection model;

[0091] S5 specifically includes the following steps:

[0092] S501. After photographing the stacked cartons within the specified position using a camera, transmit the obtained image data and depth information to the database;

[0093] S502. Use the position detection model to analyze the positions of all the contour surfaces of each carton in the current image based on the image data and depth information, extract the corresponding image data according to the positions of the contour surface center points, and transmit it to the damage detection model;

[0094] S503. Determine whether the current carton is damaged according to the model output result. If it is determined that there is no damage, extract the image data of other contour surfaces of the carton for analysis. If it is determined that there is damage, determine the relative position of the damaged part of the carton according to the damage position and the position of the contour surface center point, and output it through the visualization interface. Repeat the operation until all the image data of the cartons in the image data have been processed.

[0095] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device.

[0096] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. The AI ​​detection method for tissue packaging based on image analysis is characterized by: The method comprises the following steps: S1. Detecting defect points on the surface of paper towels: After obtaining image data of the surface of the paper towel to be tested, divide it into multiple sub-areas, determine the binarized image data and the convolved image data of each sub-area, screen out all pixel points that meet the gray value conditions in the sub-area according to the two image data, divide the target area and the background area in the convolved image data according to the position coordinates of these pixel points, connect the pixel points with the same pixel value and adjacent position in the target area, and use the connected area as the detection area, set the pattern threshold, scan the detection area line by line, count all pixel points with gray values ​​greater than the pattern threshold, determine the pattern area and the non-pattern area, delete the printed part in each pattern area to obtain the non-print area, count the gray values ​​of all pixel points in the non-print area and the non-pattern area, mark the pixel points with gray values ​​lower than the mean, screen out defect points from the marked points, and output the position information of each defect point; S2. Confirm the packaging status of the packaging bag: collect multiple thermal images of the sealing of the tissue packaging bag, mark the corresponding packaging status according to each thermal image, use the image data and status information to build a training set and a test set, set the decision tree parameters, use the data of different parts of the training set to complete the training of different decision trees, transfer the data of the test set to each decision tree in turn, use multiple output results to update the prediction confidence corresponding to each state information, train the SVM model with the data in the training set, analyze the prediction confidence corresponding to each state information according to the data in the test set, extract the thermal image data set of the sealing of the tissue packaging bag, respectively use the decision tree and SVM model to analyze, determine the status information of all the data, use the data in the data set to complete the training of the CNN model, transfer the thermal image of the sealing of the current tissue packaging bag to the CNN model for analysis, and output the packaging status information corresponding to the current image data; S3, dividing the carton contour surface: extracting image data of multiple cartons of tissue paper in a stacked state, numbering and marking the endpoints on the contour line of each carton in the image, determining the pixel coordinates of each endpoint, constructing a directed weight graph according to the marking order of each endpoint, searching in the directed weight graph with the selected endpoint as the starting point, determining the shortest path back to the current starting point, and analyzing all contour surface position information corresponding to the carton based on the shortest path; S4. Build a detection model: After determining the contour surface position information corresponding to all cartons in each image data, transfer the image data and position information to the ResNet network model, use the prediction analysis algorithm to make a preliminary prediction of the target position, import the depth information of each image data, analyze the target position and depth information according to the preliminary prediction, update the model parameters through the loss function, repeat the operation until the model training is completed, use the trained model as the position detection model, obtain the carton damage image data set, and use these image data to train the ChostNet model, so as to build a damage detection model; S5. Outputting the damaged position of the carton: After using the camera to shoot the stacked cartons in the specified position, the obtained image data and depth information are transmitted to the position detection model to obtain all the contour surface positions of each carton in the image, and the corresponding image data is extracted according to the position of the center point of the contour surface, and it is transmitted to the damage detection model to determine the damaged position and the position of the center point of the contour surface, and output through the visual interface; S6. Auxiliary early warning prompt: If the position information of the defect point on the surface of the paper towel to be tested is received, the intelligent voice device will be used to issue a prompt message that the paper towel has pollutants. If the packaging status information of the paper towel packaging bag is received as half-packaging and sandwich packaging, the voice device will be used to issue a prompt message that the paper towel is not fully packaged. If the relative position information of the damaged part of the carton is received, the voice device will be used to issue a prompt message that the packaging is damaged.

2. The AI ​​detection method for tissue packaging based on image analysis according to claim 1 is characterized in that: The S1 comprises the following steps: S101, after acquiring image data of the surface of the paper towel to be tested, setting a pixel threshold, performing a binarization operation on the image data according to the pixel threshold, and performing convolution processing on each pixel value in the image using a one-dimensional Gaussian filter to obtain processed image data; S102: Divide the entire area of ​​the image into multiple sub-areas according to the brightness value, and calculate the brightness of each sub-area according to the average brightness b. i Set the grayscale threshold g i , determine the image data f after binarization of each sub-region r,c And the convolved image data g r,c , after calculating the difference of different pixels at the same position row by row, set the brightness threshold b. If b i ≥b, the current sub-region is judged to be a high-brightness region, and the sub-region that meets f is selected. r,c -g r,c ≥g i All pixels of b are taken as target pixels. i <b, the current sub-region is determined to be a low-brightness region, and the sub-region that meets g is selected. r,c -f r,c ≥g i All pixels of the image are taken as target pixels; S103, after counting the position coordinates of all target pixels in the image data, the target area and the background area in the convolved image data are filtered out according to the position coordinates of the target pixels, so as to obtain the segmented image data.

3. The AI ​​detection method for tissue packaging based on image analysis according to claim 2 is characterized in that: The S1 further comprises the following steps: S104, after a series of operations such as connection, corrosion, filling and expansion are performed on the segmented image data, all pixels in the target area of ​​the image are scanned, pixels with the same pixel value and adjacent positions are connected, the connected area is used as the detection area, feature parameters are selected, feature values ​​corresponding to each detection area are calculated, feature range and pattern threshold are set, all detection areas whose feature values ​​meet the range are output, the detection area is scanned line by line, all pixels with gray values ​​greater than the pattern threshold are counted, and pattern areas and non-pattern areas are analyzed according to the areas where these pixels are located; S105, after extracting the complete printing data from the database, deleting the printing part in each pattern area according to the printing data to obtain a non-printing area; S106. Set the neighborhood size to 3×3, determine the grayscale values ​​of all pixels in the non-printed area and the non-patterned area, mark the pixels with grayscale values ​​lower than the mean, and count the number of marked points in the neighborhood around each marked pixel. If the number is greater than or equal to 3, the point is determined to be a defective point. After analyzing all the marked points, the location information of each defective point is output.

4. The AI ​​detection method for tissue packaging based on image analysis according to claim 1 is characterized in that: The S2 specifically includes the following steps: S201, collecting multiple thermal images of the sealing portion of the tissue packaging bag, marking the corresponding packaging state according to each thermal image, wherein the packaging state includes normal, semi-encapsulated and sandwich packaging, and collecting 60 image data for each packaging state and storing them in a sample set; S202, after sequentially extracting the color features corresponding to each image data in the sample set, combine the color features with the state information, take 40 images according to the state information and store them in the training set, and store the remaining 20 images in the test set; S203. After setting the number, depth and coefficient of decision trees, random sampling is performed in the training set to construct multiple different subsets, and different decision trees are trained using different subsets. The data in the test set is transmitted to each decision tree in turn. After obtaining multiple output results, the final result is calculated based on the different output results using a variable analysis algorithm, and the prediction confidence corresponding to each state information is updated using the deviation value between the final result and the actual state information.

5. The AI ​​detection method for tissue packaging based on image analysis according to claim 4 is characterized in that: The S2 specifically further comprises the following steps: S204, using the data in the training set to train the SVM model, after determining the parameters and weights of the model, analyzing the prediction confidence corresponding to each state information according to the data in the test set; S205, after extracting the thermal imaging data set of the sealing part of the tissue packaging bag, respectively use the decision tree and SVM models to analyze, if the state information output by the two is consistent, then the current image data is automatically labeled with the corresponding packaging state, otherwise the corresponding prediction confidences of the two are compared, and the model result with the larger confidence is taken as the packaging state corresponding to the current image data for labeling, and the operation is repeated until all the data in the data set are labeled; S206. After training the CNN model using the image data in the data set, use a thermal imager to photograph the seal of the current tissue packaging bag. After obtaining the thermal image, transmit it to the CNN model for analysis, and output the packaging status information corresponding to the current image data.

6. The AI ​​detection method for tissue packaging based on image analysis according to claim 1 is characterized in that: The S3 specifically includes the following steps: S301, extracting image data of a plurality of cartons storing tissue paper in a stacked state, using the LabelMe annotation tool to number and annotate the endpoints on the contour line of each carton in the image according to a preset rule, and after determining the pixel coordinates of each endpoint, storing them in an endpoint set; S302, setting a scale radius, selecting any endpoint as a target point, calculating the distance between the current target point and other endpoints in the set, if there is a distance value less than or equal to the scale radius, then the endpoint corresponding to the distance value is determined to be a similar point, otherwise it is determined to be a different point; S303, after screening out all similar points around the current target point using the center point analysis algorithm, calculate the center point position coordinates according to the position coordinates of the target point and similar points, and store the center point as a new endpoint to replace the original target point and similar point numbers in the set; S304. Construct a corresponding relationship matrix according to the labeling order of each endpoint using the LableMe labeling tool, draw a directed weight graph using the relationship matrix, count the number of endpoints in the set that pass through two contour lines at the same time, use these endpoints as starting points, search in the directed weight graph, determine the shortest path back to the current starting point, and extract all contour surface position information corresponding to each carton based on the endpoint numbers contained in each shortest path.

7. The AI ​​detection method for tissue packaging based on image analysis according to claim 1 is characterized in that: The S4 specifically comprises the following steps: S401, after determining the contour surface position information corresponding to all cartons in each image data, associate this information with the image data in the form of labels and transmit it to the ResNet network model, and obtain the 13×13 size feature map, 26×26 size feature map and 52×52 size feature map corresponding to each image data through multi-layer convolution operations; S402, after determining the corresponding prediction box size according to the size of the feature map, use the prediction analysis algorithm to make a preliminary prediction of the target position, import the depth information of each image data, concat the depth information with the feature image, and analyze it according to the preliminary predicted target position and the fusion result to obtain the prediction result, and use the label information corresponding to the image data to determine the position coordinates of the center points of each contour surface of the carton, and then use the loss function to update the model parameters according to the deviation value between the prediction result and the actual position coordinates. Repeat the operation until the model training is completed, and use the trained model as the position detection model; S403. After obtaining the carton damage image dataset, take 80% of the image data as the training set and the remaining image data as the validation set. Use the training set and the validation set to train the ChostNet model, update the model parameters and weights, and thus build a damage detection model.

8. The AI ​​detection method for tissue packaging based on image analysis according to claim 1, characterized in that: The S5 specifically includes the following steps: S501, after photographing the stacked cartons in the designated position using a camera, the obtained image data and depth information are transmitted to a database; S502, using the position detection model to analyze all contour surface positions of each carton in the current image according to the image data and depth information, extracting the corresponding image data according to the position of the center point of the contour surface, and transmitting it to the damage detection model; S503. Determine whether the current carton is damaged based on the model output results. If it is determined that there is no damage, extract the image data of other contour surfaces of the carton for analysis. If it is determined that there is damage, determine the relative position of the damaged part of the carton based on the damaged position and the position of the center point of the contour surface, and output it through a visual interface. Repeat the operation until the image data of all cartons in the image data are processed.

Images