A deep learning-based text printing defect detection method for handmade cigarette boxes

Abstract

The present invention relates to a text printing defect detection method based on deep learning applied to a handmade cigarette box, and the text printing defect detection method comprises: inputting a printed photograph and a preset image into an image processing model, performing cross-attention calculation on a query image vector set and a standard vector set, and obtaining a calculated feature vector; merging the feature vectors to obtain a reshaped long vector; converting the reshaped long vector into a three-dimensional tensor, and inputting the three-dimensional tensor into a CNN model to generate a defect heat map aligned with the printed photograph; identifying defects in the defect heat map and determining a target area where the defects occur; performing recognition processing on an image in the target area and determining a shape change amount when the text is printed; determining a defect detection result of the target area according to the obtained shape change amount; obtaining a defect position of the target area according to the defect detection result of the target area, and determining a comprehensive evaluation value of the target area.

Description

Technical Field

[0001] The present invention relates to the technical field of text printing defect detection, and in particular to a text printing defect detection method based on deep learning applied to handmade cigarette boxes. Background Art

[0002] In modern industrial production, text printing quality control is one of the important links to ensure the aesthetics and readability of products. Traditional text printing defect detection mainly relies on manual visual inspection or the use of simple machine vision systems. However, these methods are inefficient, costly and prone to errors. With the development of artificial intelligence technology, especially deep learning, methods based on convolutional neural networks (CNNs) have gradually become an effective means to improve the accuracy and efficiency of printed product quality inspection.

[0003] However, in the prior art, when identifying printed materials, it is necessary to comprehensively consider the deformation brought by the printed material itself and the different display conditions in different areas under the corresponding deformation to evaluate the printed work and prevent the problem of some areas being ignored during print identification. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] The purpose of the present invention is to provide a text printing defect detection method based on deep learning applied to handmade cigarette boxes, the text printing defect detection method comprising:

[0006] S1, obtaining a printed photographic image and a preset image of the text on the surface of a handmade cigarette box, converting the printed photographic image into a query image vector set, and converting the preset image into a standard vector set;

[0007] S2, input the printed photographic image and the preset image into the image processing model, perform cross-attention calculation on the query image vector set and the standard vector set, and obtain the calculated feature vector;

[0008] S3, merge the feature vectors to obtain a reshaped long vector;

[0009] S4, converting the reshaped long vector into a three-dimensional tensor, and inputting the three-dimensional tensor into a CNN model to generate a defect heat map aligned with the printed photograph;

[0010] S5, identifying defects existing in the defect thermal map and determining the target area where the defects occur;

[0011] S6, performing recognition processing on the image in the target area to determine the shape change amount when the text is printed;

[0012] S7, determining the defect detection result of the target area according to the obtained shape change amount;

[0013] S8, obtaining the defect position of the target area according to the defect detection result of the target area, and determining the comprehensive evaluation value of the target area.

[0014] Beneficial effects of the present invention: The present invention generates a defect heat map by using an image processing model and a CNN model, thereby being able to accurately identify and locate defective areas in printed products; combining the cross-calculation of feature vectors and the processing of three-dimensional tensors, being able to extract feature information of images from multiple dimensions, thereby improving the accuracy of defect detection; determining the matching recognition degree by calculating the complexity of text, enabling the system to adaptively adjust the recognition strategy according to the complexity of different areas; calculating the defect variation at the boundary defect position and the center defect position, as well as the overall defect variation in the target area, being able to comprehensively analyze the change in defects; improving the accuracy of defect detection, and providing detailed defect analysis and comprehensive evaluation, which is helpful to further improve the printing quality control process. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0016] Figure 1 A flowchart of a method for detecting text printing defects based on deep learning applied to handmade cigarette boxes provided in one embodiment of the present application;

[0017] Figure 2 A logic diagram of a method for detecting text printing defects based on deep learning applied to handmade cigarette boxes provided in one embodiment of the present application;

[0018] Figure 3 A flowchart of a method for determining a comprehensive evaluation value of a target area provided in one embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0020] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0021] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0022] Reference Figure 1 , a text printing defect detection method based on deep learning applied to handmade cigarette boxes, comprising:

[0023] S1, obtain the printed photographic image and the preset image of the text on the surface of the handmade cigarette box, convert the printed photographic image into a query image vector set, and convert the preset image into a standard vector set.

[0024] S2, input the printed photographic image and the preset image into the image processing model, perform cross-attention calculation on the query image vector set and the standard vector set, and obtain the calculated feature vector.

[0025] like Figure 2 As shown, in the present invention, the GRU and CNN convolution models are used as image processing models, and the printed photographs and preset images of the text are input into the image processing model. The model finds the locations where defects exist in the image based on the input images; Figure 2 The corresponding notations and English are explained in the following way:

[0026] Printing photo: the printing result taken, mainly the photo of printed text; preset image: the original electronic image corresponding to the expected printing; resize: use the interpolation method to scale the image to the target size; reshape: convert a tensor of one dimension to a tensor of another dimension by changing the dimension of the tensor; Q: represents a token in the query image vector set, used as query data; K and V: represent a token in the standard vector set, this value is used for the queried content, representing a key value K and a value V; concat: connects two or more vectors together according to a certain dimension to form a new vector.

[0027] The printed photograph used above represents the photographed printing result, which is mainly a photograph of printed text; the preset image represents the original electronic image corresponding to the expected printing; the GRU network can be used to convert the input image into sequence data, and the corresponding sequence data will eventually be output. At this time, the sequence data is in the form of tokens, which is a vector representation of the numerical value converted from the image, that is, the printed photograph is converted into a query image vector set, and the preset image is converted into a standard image vector set; at this time, the query vector set and the standard image vector set are cross-attention calculated and then fully connected, and then the image is input into the image processing model again by changing the tensor dimension to obtain a defect heat map, thereby completing the evaluation of the printing effect.

[0028] For multiple tokens in the query image vector set, they are used as the A domain of the query; each token is in the form of a one-dimensional tensor of 256 elements, denoted as t i A , i represents the i-th token; multiple tokens in the standard vector set are used as the retrieved B domain. Each token in the standard vector set is also a one-dimensional tensor with 256 elements, denoted by t j B , a token in the standard vector set can be considered to have a key value and a value at the same time. This value is used for the queried content and represents a key value K and a value V. The number of tokens in the two domains A and B can be set to the same number, for example, 25.

[0029] The query image vector set and the standard vector set are cross-calculated to obtain the calculated feature vector, and the calculated content at this time is marked as the result token, recorded as t R The number of generated tokens is consistent with the number of tokens in domains A and B. The calculation formula for the feature vector is as follows:

[0030] ;

[0031] The SUM() bracket contains a 25*256 tensor. The function adds the tensors directly according to the last dimension to obtain a 256-dimensional vector. Represents the value of the nth token in the feature vector, Represents an element in the standard vector set; Represents the value of the nth token in the query image vector set; express Perform inner product operations with the vectors inside [ ] respectively, and we get Result form of the same form; [ ] [ ] means that the inner product of the two vectors inside [ ] is taken, and the sum is obtained. The result is of the same form; so the final result token is represented as According to the token represented by the feature vector at this time, the content in the printed photo can be marked, and after multi-dimensional processing, it can help find the location where defects and complexities exist.

[0032] S3, merge the feature vectors to obtain a reshaped long vector.

[0033] S4, converts the reshaped long vector into a three-dimensional tensor, and inputs the three-dimensional tensor into the CNN model to generate a defect heat map aligned with the printed photograph.

[0034] Combine the result tokens into a long token according to the lowest dimension, that is, Converted to a reshaped long vector of length 25*256=6400 . Use the reshape method to convert this long vector into a 16*16*25 three-dimensional tensor , represents a 16*16 feature map.

[0035] ;

[0036] in, represents a three-dimensional tensor, Reshape a long vector.

[0037] The three-dimensional vector is input into the CNN network to obtain the output feature map; the output feature map can be expressed as shown below.

[0038] ;

[0039] in, Represents the output feature map; at this time, the output feature map can be obtained according to the CNN model, and the output feature map is upsampled to obtain a heat map.

[0040] ;

[0041] in, Represents a heat map; the obtained heat map is normalized, and the pixel value range of the normalized heat map is between 0 and 1.

[0042] ;

[0043] in, represents the normalized heat map, Represents a heat map, represents the minimum value of the heat map, Indicates the maximum value of the heat map.

[0044] Compare the normalized heat map with the printed photo to obtain a defect heat map aligned with the input printed photo.

[0045] ;

[0046] in, Defect heatmap aligned with the input print photo, Represents the image alignment operation, which is used to align the normalized heat map with the printed photo. Indicates the size of the printed photograph.

[0047] After upsampling through the CNN network, a 25*25*1 heat map is generated. The range of each heat map pixel value is between 0 and 1. The value closer to 1 indicates that the probability of existence at this position is also greater. Finally, the size is aligned with the input photo by resizing, so as to determine whether there is a corresponding problem in the current image.

[0048] S5, identifying the defects in the defect thermal map and determining the target area where the defects occur.

[0049] The method for determining the target area at this time is to compare the pixels in the defect heat map one by one. When the pixel value in the heat map is different from the preset map, the corresponding pixel is marked as 1, and when it is not, it is marked as 0. Therefore, the pixel with a value of 1 is the location where the defect exists. The ratio of the number of all pixel values ​​1 to the total number of pixels is recorded, and the heat map is divided into grid areas of equal size. The grid area size is set to 4×4. When there is an area with a ratio greater than 0.2 in the grid area in the heat map, the corresponding grid area is marked as the target area.

[0050] S6, performing recognition processing on the image in the target area to determine the shape change amount when the text is printed;

[0051] S7, determining the defect detection result of the target area according to the obtained shape change amount;

[0052] S8, obtaining the defect position of the target area according to the defect detection result of the target area, and determining the comprehensive evaluation value of the target area.

[0053] When the shape change amount in step S6 is expressed as comparing the image in the target area with the preset image, the pixels existing in the target area at this time are identified, and the relative difference values ​​of the corresponding positions of the pixels are used as the shape change amount at this time.

[0054] In one embodiment, step S7 includes: calculating the average deformation amount of the shape change in the target area, and when the average deformation amount is greater than a preset threshold, it is considered that there are obvious defects in the current target area, and the corresponding target area is output as a defect detection result.

[0055] In one embodiment, the preset threshold is set to 0.5 pixel value, which is used to distinguish the target area with problems.

[0056] like Figure 3 As shown, in step S8, the processing method of the comprehensive evaluation value of the target area includes the following steps:

[0057] S81, obtaining a preset allocation rule for the target area, sorting the target areas according to the preset allocation rule, and determining the matching recognition degree of the target area.

[0058] S82, performing defect recognition on the target area according to the matching recognition degree of the target area, and determining the defect positions of the target area in turn, wherein the defect positions include boundary defect positions and center defect positions.

[0059] S83, based on the acquired boundary defect position and center defect position, calculate the defect variation corresponding to the adjacent target area.

[0060] S84, taking the defect variation as a constraint condition, determining the comprehensive recognition function corresponding to the target area when the defect position in the target area changes in matching recognition degree.

[0061] S85, calculating the loss function of the target area according to the pixel values ​​of the defective positions in the target area, and obtaining a comprehensive evaluation value of the target area based on the loss function and the comprehensive recognition function.

[0062] The above-mentioned preset allocation rule represents the position where the printed text image is set when the current text is printed, and is sorted according to the ratio of the number of pixels corresponding to the text in the target area to the total number of pixels in the target area, as the preset allocation rule at this time, and this ratio is recorded as the text complexity; when the ratios are the same, the text is sorted according to the strokes of the text in the target area, and the target area with more strokes and a larger ratio of the number of pixels corresponding to the text to the total number of pixels in the target area is used as the data content for priority processing.

[0063] At this time, the matching recognition degree in step S81 is implemented as follows: the text complexity corresponding to the target area is obtained, and the matching recognition degree is obtained according to the text complexity.

[0064] ;

[0065] in, Indicates the matching recognition degree, Indicates the text complexity of the current target area. Indicates the maximum value of text complexity, Indicates the minimum value of text complexity.

[0066] After the matching recognition degree is obtained, the target area is sorted according to the matching recognition degree, and the defect positions identified in the target area are marked, and the boundary defect positions and the center defect positions are marked respectively.

[0067] The defect variation in step S83 is calculated by dividing the defect position into a boundary defect position and a center defect position, calculating the difference in the corresponding defect forms at the two defect positions, and integrating the defects at the two positions, so as to obtain the defect variation of the target area to quantify the difference in defect distribution that may exist in different target areas.

[0068] At this time, the defect variation at the boundary defect position and the defect variation at the center defect position are calculated respectively to obtain the defect variation in the target area.

[0069] The defect variation of the boundary defect position is as follows: the average distance between all adjacent boundary defect positions is calculated as the defect variation between the boundary defect positions.

[0070] The defect variation of the central defect position is as follows: the average distance between all central defect positions and adjacent boundary defect positions is calculated as the defect variation of the central defect position.

[0071] At this time, the defect variation of the target area is realized as follows: the defect variation of the boundary defect position and the defect variation of the center defect position are normalized to obtain the defect variation of the target area:

[0072] ;

[0073] in, represents the defect variation in the target area, represents the defect variation at the boundary defect location, represents the defect variation at the center defect location, represents the exponential constant, , Indicates the adjustment coefficient. At this time, the value of the adjustment coefficient can be set to ;in, Indicates the average distance between the boundary defect position and the reference point. At this time, the reference point is set to the point closest to all boundary defect positions in the current target area. The setting method is Same as above, except that the average distance between the boundary defect position and the reference point is replaced by the average distance between the center defect position and the reference point; Represents a small constant, with a value of 0.01.

[0074] The comprehensive recognition function corresponding to the target area can be expressed as shown below.

[0075] ;

[0076] in, represents the comprehensive recognition function corresponding to the target area, represents the weight of the j-th target area, Indicates the matching recognition degree corresponding to the j-th target area; It represents the difference between the defect variation of the jth target area and the adjacent j+1th target area. represents the number of target regions, It represents the regularization parameter, which is used to measure the relationship between the matching recognition degree and the defect variation. At this time, the value of the regularization parameter is 0.1 or 0.2, and the value range of j is 1 to m.

[0077] The weight of the target area set above is expressed as the ratio of the area of ​​the current target area to the area of ​​the maximum target area.

[0078] The loss function of the target area can be calculated as follows.

[0079] ;

[0080] in, represents the loss function, Indicates the b-th pixel value of the preset image. Indicates the b-th pixel value of the current target area. Indicates the number of pixels, the value range of b is 1 to N.

[0081] After the loss function and the comprehensive recognition function of the target area are calculated, the comprehensive evaluation value is the comprehensive value of the two. The comprehensive evaluation value of the target area can be expressed as follows:

[0082] ;

[0083] in, Represents the comprehensive evaluation value of the target area, represents the exponential constant, represents the loss function, It represents the comprehensive recognition function corresponding to the target area. Thus, the recognition of the current image is completed to verify whether the text printing effect meets the expectations.

[0084] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A text printing defect detection method based on deep learning applied to handmade cigarette boxes, characterized in that: include: S1, obtaining a printed photograph and a preset image of the text on the surface of a handmade cigarette box, using a first GRU network to convert the printed photograph into a query image vector set, and using a second GRU network to convert the preset image into a standard vector set; S2, input the printed photographic image and the preset image into the image processing model, perform cross-attention calculation on the query image vector set and the standard vector set, and obtain the calculated feature vector; S3, merge the feature vectors to obtain a reshaped long vector; S4, converting the reshaped long vector into a three-dimensional tensor, and inputting the three-dimensional tensor into a CNN model to generate a defect heat map aligned with the printed photograph; S5, identifying defects existing in the defect thermal map and determining the target area where the defects occur; S6, performing recognition processing on the image in the target area to determine the shape change amount when the text is printed; S7, determining the defect detection result of the target area according to the obtained shape change amount; S8, obtaining the defect position of the target area according to the defect detection result of the target area, and determining the comprehensive evaluation value of the target area; The processing method of the comprehensive evaluation value of the target area in step S8 includes the following steps: S81, obtaining a preset allocation rule for the target area, sorting the target areas according to the preset allocation rule, and determining the matching recognition degree of the target area; S82, performing defect recognition on the target area according to the matching recognition degree of the target area, and determining the defect positions of the target area in sequence, wherein the defect positions include boundary defect positions and center defect positions; S83, calculating defect variation corresponding to adjacent target areas based on the acquired boundary defect positions and center defect positions; S84, taking the defect variation as a constraint condition, determining a comprehensive recognition function corresponding to the target area when the defect position in the target area changes in matching recognition degree; S85, calculating the loss function of the target area according to the pixel values ​​of the defective positions in the target area, and obtaining a comprehensive evaluation value of the target area based on the loss function and the comprehensive recognition function.

2. A method for detecting text printing defects based on deep learning applied to handmade cigarette boxes as claimed in claim 1, characterized in that: The calculation formula of the eigenvector is as follows: ； in, Represents the value of the nth token in the feature vector, Represents an element in the standard vector set; Indicates the value of the nth token in the query image vector set; [ ] [ ] means taking the inner product of the two vectors inside [ ].

3. The method for detecting text printing defects based on deep learning applied to handmade cigarette boxes according to claim 1, characterized in that: Step S7 includes: calculating an average deformation amount of the shape change amount in the target area, and when the average deformation amount is greater than a preset threshold, outputting the corresponding target area as a defect detection result.

4. The method for detecting text printing defects based on deep learning applied to handmade cigarette boxes according to claim 1, characterized in that: The matching recognition degree is implemented as follows: the text complexity corresponding to the target area is obtained, and the matching recognition degree is obtained according to the text complexity; ； in, Indicates the matching recognition degree, Indicates the text complexity of the current target area. Indicates the maximum value of text complexity, Indicates the minimum value of text complexity.

5. The method for detecting text printing defects based on deep learning applied to handmade cigarette boxes according to claim 1, characterized in that: Step S83 includes: The defect variation at the boundary defect position and the defect variation at the center defect position are calculated respectively to obtain the defect variation in the target area; The defect variation of the boundary defect position is as follows: the average distance between all adjacent boundary defect positions is calculated as the defect variation between boundary defect positions; The defect variation of the central defect position is as follows: the average distance between all central defect positions and adjacent boundary defect positions is calculated as the defect variation of the central defect position.

6. A method for detecting text printing defects based on deep learning applied to handmade cigarette boxes as claimed in claim 5, characterized in that: The defect variation of the target area is implemented as follows: the defect variation of the boundary defect position and the defect variation of the center defect position are normalized to obtain the defect variation of the target area: ； represents the defect variation in the target area, represents the defect variation at the boundary defect location, represents the defect variation at the center defect location, represents the exponential constant, , Represents the adjustment factor.

7. The method for detecting text printing defects based on deep learning applied to handmade cigarette boxes according to claim 1, characterized in that: The comprehensive recognition function corresponding to the target area is expressed as follows: ； in, represents the comprehensive recognition function corresponding to the target area, represents the weight of the j-th target area, Indicates the matching recognition degree corresponding to the j-th target area; It represents the difference between the defect variation of the jth target area and the adjacent j+1th target area. represents the number of target regions, represents the regularization parameter.

8. The method for detecting text printing defects based on deep learning applied to handmade cigarette boxes according to claim 1, characterized in that: The loss function of the target area is calculated as follows: ； in, represents the loss function, Indicates the b-th pixel value of the preset image. Indicates the b-th pixel value of the current target area. Indicates the number of pixels.

9. The method for detecting text printing defects based on deep learning applied to handmade cigarette boxes according to claim 1, characterized in that: The comprehensive evaluation value of the target area is expressed as follows: ； in, Represents the comprehensive evaluation value of the target area, represents the exponential constant, represents the loss function, Represents the comprehensive recognition function corresponding to the target area.

Images