Variable data printing defect detection method and system based on YOLOv5s model

Abstract

The application is suitable for the technical field of industrial detection, and provides a variable data printing defect detection method and system based on a YOLOv5s model, which comprises the following steps: obtaining a to-be-detected image of a to-be-detected printed matter based on a preset camera; inputting the to-be-detected image into a pre-trained improved YOLOv5s model to generate a feature detection image; and determining whether the to-be-detected printed matter has a printing defect according to the feature detection image. The application can efficiently and accurately determine possible printing defects in the to-be-detected printed matter, and has a detection efficiency much higher than manual detection, can adapt to detection of different types of printed matters and detection in different backgrounds, has better robustness and generalization, can timely find products with problems in printing quality, avoid inflow into the market to cause an impact on the enterprise image, protect the rights and interests of consumers, and improve the experience of consumers in purchasing or using products.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial detection, in particular to a variable data printing defect detection method and system based on a YOLOv5s model. BACKGROUND

[0002] Variable data printing, also known as variable information printing, personalized printing or customized printing, is a printing method that uses electronic files as carriers and transmits them to printing equipment through a network for direct printing. Variable data printing has characteristics such as one-off printing, no need for plate making, immediate availability, timely error correction, variable printing and on-demand printing, which traditional printing does not have. The variable data printing elements mainly include text, patterns and barcodes.

[0003] In actual printing production, various printing defects may occur due to uncertain factors such as production process, production equipment and industrial site production environment. These printing defects can be divided into three categories according to the defect morphology: points, lines and blocks. As the carrier of key information of commodities, printed matter is an important way for enterprises to track products and control quality. However, these printing defects can affect the readability of printed matter and the image of enterprise products, causing unavoidable losses to commodity manufacturers. Therefore, it is of great significance to detect and evaluate the quality of variable data printed matter.

[0004] Currently, printing defect detection mainly adopts manual detection method or machine vision detection method based on digital image processing technology. The manual detection method refers to observing with the aid of tools (such as magnifying glass) or directly observing with the naked eye. However, in the detection of large quantities of printed matter, the naked eye is prone to fatigue. The machine vision detection method based on digital image processing technology needs to finely adjust the light source, shooting distance and shooting angle before detection, which has the problem of low detection efficiency and needs to be further improved. SUMMARY

[0005] Therefore, the embodiments of the present application provide a variable data printing defect detection method and system based on a YOLOv5s model to solve the problem of low detection efficiency in the prior art.

[0006] In a first aspect, the embodiments of the present application provide a variable data printing defect detection method based on a YOLOv5s model, which comprises:

[0007] Based on a preset camera, an image to be detected of a printed matter to be detected is obtained;

[0008] The image to be detected is input into an improved YOLOv5s model trained in advance to generate a feature detection image;

[0009] determine whether the to-be-detected printed matter has a printing defect according to the feature detection image.

[0010] Compared with the prior art, the beneficial effects are that the variable data printing defect detection method based on the YOLOv5s model provided in the embodiments of the present application can first acquire a to-be-detected image of a to-be-detected printed matter based on a preset camera, then input the to-be-detected image into the improved YOLOv5s model trained in advance to obtain a feature detection image, and when the to-be-detected printed matter has a printing defect, the feature detection image output by the improved YOLOv5s model will mark the printing defect, and then determine whether the to-be-detected printed matter has a printing defect according to the feature detection image, thereby facilitating timely and accurate discovery of the printing defect of the to-be-detected printed matter, greatly improving the detection efficiency, and the detection efficiency is much higher than manual detection and can adapt to detection of different types of printed matters and detection in different backgrounds, and to some extent, the problem of low detection efficiency is solved.

[0011] In a second aspect, the embodiments of the present application provide a variable data printing defect detection system based on a YOLOv5s model, which comprises:

[0012] a to-be-detected image acquisition module configured to acquire a to-be-detected image of a to-be-detected printed matter based on a preset camera;

[0013] a feature detection image output module configured to input the to-be-detected image into an improved YOLOv5s model trained in advance to generate a feature detection image;

[0014] a printing defect determination module configured to determine whether the to-be-detected printed matter has a printing defect according to the feature detection image.

[0015] In a third aspect, the embodiments of the present application provide a terminal device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method of the first aspect when executing the computer program.

[0016] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the steps of the method of the first aspect.

[0017] It can be understood that the beneficial effects of the second aspect to the fourth aspect can be understood with reference to the related description in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows.

[0019] Figure 1 is a flowchart of a variable data printing defect detection method provided by an embodiment of the present application;

[0020] Figure 2 is a flowchart of step S201 in the variable data printing defect detection method provided by an embodiment of the present application;

[0021] Figure 3 is a network structure diagram of an improved YOLOv5s model provided by an embodiment of the present application;

[0022] Figure 4 is a first flowchart of step S204 in the variable data printing defect detection method provided by an embodiment of the present application;

[0023] Figure 5 is a network structure diagram of a C3 module provided by an embodiment of the present application;

[0024] Figure 6 is a flowchart of step S2042 in the variable data printing defect detection method provided by an embodiment of the present application;

[0025] Figure 7 is a second flowchart of step S204 in the variable data printing defect detection method provided by an embodiment of the present application;

[0026] Figure 8 is a schematic diagram of a bidirectional multi-scale feature fusion network structure provided by an embodiment of the present application;

[0027] Figure 9 is a third flowchart of step S204 in the variable data printing defect detection method provided by an embodiment of the present application;

[0028] Figure 10 is a network structure diagram of a CA module provided by an embodiment of the present application;

[0029] Figure 11 is a flowchart before step S201 in the variable data printing defect detection method provided by an embodiment of the present application;

[0030] Figure 12 is a schematic diagram of a printing defect provided by an embodiment of the present application, wherein, Figure 12 (a) of is a first schematic diagram of a to-be-detected image with a printing defect, Figure 12 (b) of is a first schematic diagram of a feature detection image output by the improved YOLOv5s model;

[0031] Figure 13 is a schematic diagram of a printing defect provided by an embodiment of the present application, wherein, Figure 13 (a) of is a second schematic diagram of an image to be detected with a printing defect, Figure 13 (b) of is a second schematic diagram of a feature detection image output by the improved YOLOv5s model;

[0032] Figure 14 is a schematic diagram of a printing defect provided by an embodiment of the present application, wherein, Figure 14 (a) of is a third schematic diagram of an image to be detected with a printing defect, Figure 14 (b) of is a third schematic diagram of a feature detection image output by the improved YOLOv5s model;

[0033] Figure 15 is a block diagram of a variable data printing defect detection system provided by an embodiment of the present application;

[0034] Figure 16 is a schematic diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0035] In the following description, specific details are set forth in order to provide a thorough understanding of embodiments of the application. However, persons of ordinary skill in the art will readily recognize that embodiments of the application can be practiced without these specific details. In other instances, well-known structures, devices, circuits, and processes have not been described in detail so as not to unnecessarily obscure aspects of the application.

[0036] In the description of the present application and the appended claims, the terms "first", "second", "third", etc. are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0037] In the present application, the reference "one embodiment" or "some embodiments" and the like means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in other some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0038] In order to illustrate the technical solutions described in the present application, the following will be described by specific embodiments.

[0039] Referring to Figure 1 , Figure 1 is a flowchart of a variable data printing defect detection method based on a YOLOv5s model provided in an embodiment of the present application. In this embodiment, the execution subject of the variable data printing defect detection method is a terminal device. It can be understood that the types of the terminal device include but are not limited to a mobile phone, a tablet computer, a notebook computer, an Ultra-Mobile Personal Computer (UMPC), a netbook, a Personal Digital Assistant (PDA), and the like, and the specific type of the terminal device is not limited in the embodiment of the present application.

[0040] Referring to Figure 1 , the variable data printing defect detection method provided in the embodiment of the present application includes but is not limited to the following steps:

[0041] In S100, based on a preset camera, a to-be-detected image of a to-be-detected printed matter is acquired.

[0042] Without loss of generality, the camera can be a video image acquisition device capable of being stably and efficiently applied to an industrial site, i.e., an industrial camera; the to-be-detected printed matter is used to describe a printed matter to be detected whether there is a printing defect; above the conveying belt of the printing device, an industrial camera can be pre-installed, which can continuously shoot a video about the to-be-detected printed matter.

[0043] Specifically, the terminal device can first acquire the video about the to-be-detected printed matter in real time through the industrial camera, and then the terminal device can determine each frame of image in the video as the to-be-detected image of the to-be-detected printed matter.

[0044] In some possible implementation manners, in order to facilitate subsequent accurate and efficient detection of the printing defect, after the terminal device acquires the to-be-detected image, the terminal device can pre-process the to-be-detected image based on a preset computer vision library (such as OpenCv), segment the to-be-detected printed matter and the background in the to-be-detected image, and / or segment multiple to-be-detected printed matters in the to-be-detected image, and then perform mean filter denoising processing to obtain a denoised to-be-detected image.

[0045] In S200, the to-be-detected image is input into a pre-trained improved YOLOv5s model to generate a feature detection image.

[0046] Specifically, after the terminal device obtains the to-be-detected image, the terminal device can input the to-be-detected image into the improved YOLOv5s model pre-trained to generate a feature detection image, where the feature detection image is used to describe an image output by the improved YOLOv5s model; when the to-be-detected printed matter has a printing defect, the printing defect in the feature detection image is labeled.

[0047] In some possible implementation manners, in order to accurately and efficiently determine the printing defect, before step S200, the method further includes but is not limited to the following steps:

[0048] In S201, an improved YOLOv5s model is constructed based on an initial YOLOv5s model, a bidirectional multi-scale feature fusion network structure, and a CA attention mechanism.

[0049] Specifically, the initial YOLOv5s model is used to describe an initial YOLOv5s model, and the initial YOLOv5s model includes a backbone feature extraction network and an enhanced feature extraction network, where the backbone feature extraction network is a backbone network, and the enhanced feature extraction network is a neck network; the improved YOLOv5s model is used to describe an improved YOLOv5s model, and the improved YOLOv5s model can accurately and efficiently determine the printing defect. The terminal device can construct the improved YOLOv5s model based on the initial YOLOv5s model, in combination with the bidirectional multi-scale feature fusion network structure and the CA attention mechanism.

[0050] In some possible implementation manners, in order to construct the improved YOLOv5s model, refer to Figure 2 , step S201 includes but is not limited to the following steps:

[0051] In S202, an initial YOLOv5s model is obtained.

[0052] Specifically, the terminal device can obtain the initial YOLOv5s model.

[0053] In S203, for the enhanced feature extraction network in the initial YOLOv5s model: a bidirectional multi-scale feature fusion network structure is used to replace a feature pyramid structure of the enhanced feature extraction network, to generate a first optimized YOLOv5s model.

[0054] Specifically, after the terminal device obtains the initial YOLOv5s model, the terminal device can perform the following processing on the enhanced feature extraction network in the initial YOLOv5s model: a bidirectional multi-scale feature fusion network structure is used to replace a feature pyramid structure in the original enhanced feature extraction network, to generate a first optimized YOLOv5s model.

[0055] In S204, the CA attention mechanism is embedded in the backbone feature extraction network corresponding to the first optimized YOLOv5s model to generate an improved YOLOv5s model.

[0056] Specifically, after the terminal device generates the first optimized YOLOv5s model, the terminal device can embed the CA attention mechanism in the backbone feature extraction network corresponding to the first optimized YOLOv5s model to generate the improved YOLOv5s model, wherein, please refer to Figure 3 , Figure 3 “Backbone” in the improved YOLOv5s model, Figure 3 “BiFusion Neck” in the improved YOLOv5s model, Figure 3 “Head” in the improved YOLOv5s model, the backbone feature extraction network corresponding to the improved YOLOv5s model is sequentially connected with the CBS module, the CBS module, the C3 module, the CA module, the CBS module, the C3 module, the CA module, the CBS module, the C3 module, the CA module, the CBS module, the C3 module, the SPFF module and the CA module in the order from the shallow layer to the deep layer.

[0057] In some possible implementations, in order to improve the accuracy and stability of the improved YOLOv5s model, please refer to Figure 4 , step S204 includes but is not limited to the following steps:

[0058] In S2041, the input feature x is input into the CBS module in the main path and the CBS module in the branch, and the main path and the branch generate the output feature y1 after the convolution operation of the two CBS modules.

[0059] Without loss of generality, the CBS module is composed of a convolution layer (Conv), a batch normalization layer (BN) and an activation function layer (SiLU), wherein the SiLU activation function adopted by the activation function layer can be:

[0060]

[0061] In the formula, x is the image to be detected, Sigmoid() is a preset sigmoid growth curve function, and e is a preset constant.

[0062] Exemplarily, please refer to Figure 5, the C3 module is composed of a main branch and a sub-branch, the main branch includes a BottleNeck module and a CBS module, and the sub-branch includes the CBS module but does not include the BottleNeck module; the input feature x can be a to-be-detected image processed by the CBS module, for example, in relation to the C3 module of the third layer in the backbone feature extraction network, the to-be-detected image processed by and output from the second layer CBS module in the backbone feature extraction network is the input feature x. The terminal device can input the input feature x into the CBS module in the main branch and the CBS module in the sub-branch respectively, and after convolution operations are performed on both CBS modules, the main branch and the sub-branch both generate an output feature y1.

[0063] In S2042, the output feature y1 in the main branch is input into a preset BottleNeck module to generate an output feature y4.

[0064] Specifically, referring to Figure 5 , the terminal device can input the output feature y1 in the main branch into a preset BottleNeck module to generate an output feature y4.

[0065] In some possible implementation manners, in order to improve the efficiency of the output result and reduce the case of gradient disappearance or gradient explosion, referring to Figure 6 , the step S2042 includes but is not limited to the following steps:

[0066] In S20421, when the logical attribute of the residual bypass is True, the output feature y1 in the main branch is input into the CBS module to generate an output feature y2, and the output feature y1 is input into the residual bypass.

[0067] Exemplarily, referring to Figure 5 , the BottleNeck module includes a residual bypass and two CBS modules connected in series, the logical attribute of the residual bypass is True or False, and in one possible implementation manner, referring to Figure 3 , the logical attribute of the residual bypass corresponding to the C3 module in the backbone feature extraction network is True, and the logical attribute of the residual bypass corresponding to the C3 module in the enhanced feature extraction network is False.

[0068] Specifically, referring to Figure 5 , when the logical attribute of the residual bypass is True, the terminal device can input the output feature y1 in the main branch into the CBS module to generate an output feature y2, and input the output feature y1 into the residual bypass.

[0069] In S20422, the output feature y2 is input into the CBS module to generate an output feature y3.

[0070] Specifically, referring toFigure 5 After the terminal device generates the output feature y3, the terminal device can input the output feature y3 into the CBS module to generate the output feature y4.

[0071] In S20423, the output feature y3 is stacked with the output feature y1 in the residual bypass to generate the output feature y4.

[0072] Specifically, please refer to Figure 5 After the terminal device generates the output feature y3, the terminal device can input the output feature y3 into the CBS module to generate the output feature y4.

[0073] In S20424, when the logical attribute of the residual bypass is False, the output feature y1 in the main path is input into the CBS module to generate the output feature y2.

[0074] Specifically, please refer to Figure 5 When the logical attribute of the residual bypass is False, the terminal device can input the output feature y1 in the main path into the CBS module to generate the output feature y2.

[0075] In S20425, the output feature y2 is input into the CBS module to generate the output feature y3.

[0076] Specifically, please refer to Figure 5 After the terminal device generates the output feature y2, the terminal device can input the output feature y2 into the CBS module to generate the output feature y3.

[0077] In S20426, the output feature y3 is determined as the output feature y4.

[0078] Specifically, please refer to Figure 5 After the terminal device generates the output feature y3, the terminal device can determine the output feature y3 as the output feature y4, so that when the logical attribute of the residual bypass in the BottleNeck module is False, the operation of the BottleNeck module is only the two CBS modules in series in the main path.

[0079] In S2043, the output feature y1 generated by the branch is spliced with the output feature y4 to generate the output feature y5.

[0080] Specifically, the terminal device can splice the output feature y1 generated by the branch with the output feature y4 to generate the output feature y5.

[0081] In S2044, the output feature y5 is input into the CBS module, and after the convolution operation in the CBS module, the output feature y6 is generated.

[0082] Specifically, after the terminal device generates the output feature y5, the terminal device can input the output feature y5 to the CBS module, and generate the output feature y6 after the convolution operation in the CBS module.

[0083] In some possible implementation manners, in order to further enable the improved YOLOv5s model to accurately and efficiently detect the printing defects, please refer to Figure 7 , step S204 further includes but is not limited to the following steps:

[0084] In S2045, the feature map output by the ninth layer CA module in the backbone feature extraction network is subjected to 1×1 convolution dimension reduction processing to generate a first convolution dimension reduction feature map.

[0085] Specifically, please refer to Figure 3 and Figure 8 , the terminal device can perform 1×1 convolution dimension reduction processing on the feature map output by the ninth layer CA module in the backbone feature extraction network to generate a first convolution dimension reduction feature map, wherein the 1×1 convolution represents a convolution with a size of 1×1.

[0086] In S2046, the feature map output by the thirteenth layer CA module in the backbone feature extraction network is input into the enhanced feature extraction network to perform 2×2 transpose convolution upsampling processing to generate a convolution upsampling feature map.

[0087] Specifically, please refer to Figure 3 and Figure 8 , after the terminal device generates the first convolution dimension reduction feature map, the terminal device can input the feature map output by the thirteenth layer CA module in the backbone feature extraction network into the enhanced feature extraction network to perform 2×2 transpose convolution upsampling processing to generate a convolution upsampling feature map, wherein the 2×2 transpose convolution represents a transpose convolution with a size of 2×2.

[0088] In S2047, the feature map output by the sixth layer CA module in the backbone feature extraction network is input into the enhanced feature extraction network to perform 1×1 convolution dimension reduction processing to generate a second convolution dimension reduction feature map.

[0089] Specifically, please refer to Figure 3 and Figure 8 , the terminal device can input the feature map output by the sixth layer CA module in the backbone feature extraction network into the enhanced feature extraction network to perform 1×1 convolution dimension reduction processing to generate a second convolution dimension reduction feature map.

[0090] In S2048, the second convolution dimension reduction feature map is subjected to convolution downsampling processing with a size of 3×3 and a step of 2 in the enhanced feature extraction network to generate a third convolution dimension reduction feature map.

[0091] Specifically, referring to Figure 3 and Figure 8 , the terminal device can perform 3x3 convolution downsampling processing on the second convolution dimension reduction feature map in the enhanced feature extraction network, with a step size of 2, to generate a third convolution dimension reduction feature map.

[0092] In S2049, based on a preset Concat function, the first convolution dimension reduction feature map, the convolution upsampling feature map, and the third convolution dimension reduction feature map are spliced and fused to generate a spliced and fused feature map.

[0093] Specifically, referring to Figure 3 and Figure 8 , the terminal device can perform splicing and fusion processing on the first convolution dimension reduction feature map, the convolution upsampling feature map, and the third convolution dimension reduction feature map based on a preset Concat function to generate a spliced and fused feature map.

[0094] In S20491, the spliced and fused feature map is subjected to 1x1 convolution processing to generate a convolution fused feature map.

[0095] Specifically, referring to Figure 3 , after the terminal device generates the spliced and fused feature map, the terminal device can perform 1x1 convolution processing on the spliced and fused feature map to generate a convolution fused feature map, thereby realizing bidirectional multi-scale feature fusion from large scale downward and small scale upward. The features after fusing deep and shallow layers can have better semantic information and spatial information.

[0096] In some possible implementations, in order to more favorably improve the YOLOv5s model to accurately and efficiently determine the printing defects, referring to Figure 9 , step S204 further includes but is not limited to the following steps:

[0097] In S20492, based on the CA attention mechanism, the size of the input feature map is averaged pooled from the horizontal direction and the vertical direction respectively to obtain a horizontal one-dimensional vector and a vertical one-dimensional vector.

[0098] Without loss of generality, the size of the input feature map is CxHxW, the horizontal one-dimensional vector is CxHx1, and the vertical one-dimensional vector is Cx1xW, C is the number of channels, H is the height, and W is the width.

[0099] Specifically, referring to Figure 10 , the terminal device can first perform average pooling on the size of the input feature map from the horizontal direction based on the CA attention mechanism to obtain a horizontal one-dimensional vector, and then perform average pooling on the size of the input feature map from the vertical direction to obtain a vertical one-dimensional vector.

[0100] In S20493, the horizontal one-dimensional vector and the vertical one-dimensional vector are spliced in the spatial dimension to generate a spliced one-dimensional vector, and the channel number is compressed using 1x1 convolution to change the channel number to

[0101] Specifically, after the terminal device generates the horizontal one-dimensional vector and the vertical one-dimensional vector, the terminal device can splice the horizontal one-dimensional vector and the vertical one-dimensional vector in the spatial dimension to generate a spliced one-dimensional vector, and simultaneously compress the channel number using 1x1 convolution to change the channel number to Thus, the dimension is adjusted.

[0102] In S20494, the spatial information in the vertical direction and the spatial information in the horizontal direction in the spliced one-dimensional vector are encoded based on a batch normalization operation and a nonlinear transformation operation to generate a feature vector.

[0103] Specifically, the terminal device can encode the spatial information in the vertical direction and the spatial information in the horizontal direction in the spliced one-dimensional vector by a batch normalization (BatchNorm) operation and a nonlinear transformation (Non-linear) operation to generate a feature vector.

[0104] In S20495, the feature vector is separated to generate a new horizontal one-dimensional vector and a new vertical one-dimensional vector.

[0105] Specifically, the terminal device can separate (Split) the feature vector to generate a new horizontal one-dimensional vector and a new vertical one-dimensional vector, so as to redivide the feature vector into a horizontal vector and a vertical vector.

[0106] In S20496, 1x1 convolution processing is performed on the new horizontal one-dimensional vector and the new vertical one-dimensional vector respectively to determine the target channel number of the new horizontal one-dimensional vector and the new vertical one-dimensional vector.

[0107] Specifically, the target channel number of the new horizontal one-dimensional vector and the target channel number of the new vertical one-dimensional vector are the same as the channel number of the input feature map; the terminal device can first perform 1x1 convolution processing on the new horizontal one-dimensional vector to determine the target channel number of the new horizontal one-dimensional vector, and then perform 1x1 convolution processing on the new vertical one-dimensional vector to determine the target channel number of the new vertical one-dimensional vector.

[0108] In S20497, the new horizontal one-dimensional vector and the new vertical one-dimensional vector are respectively input to a preset activation function to obtain a horizontal output vector and a vertical output vector.

[0109] Specifically, the new horizontal one-dimensional vector and the new vertical one-dimensional vector are input into a preset activation function (Sigmold) respectively to obtain a horizontal output vector and a vertical output vector, where the activation function can be:

[0110]

[0111] In the formula, S() is a preset Sigmold activation function, e is a preset constant, and x is the image to be detected.

[0112] In S20498, the horizontal output vector and the vertical output vector are normalized and weighted to generate a target output vector.

[0113] Specifically, the terminal device can normalize and weight the horizontal output vector and the vertical output vector to generate a target output vector, which can be output by the CA module.

[0114] In some possible implementation manners, in order to construct a more optimized improved YOLOv5s model, please refer to Figure 11 Before step S201, the method further includes but is not limited to the following steps:

[0115] In S2011, based on a preset camera, a set of printed defect images is obtained.

[0116] Specifically, the set of printed defect images includes a plurality of printed defect images, and the printed defect images are used to describe images containing printed defects; the terminal device can obtain the set of printed defect images based on an industrial camera and a vision platform.

[0117] In S2012, for each printed defect image: the defect type and the defect position of the printed defect image are labeled to generate a labeled defect image.

[0118] Specifically, the terminal device can process each printed defect image as follows: based on LabelImg software, the defect type and the defect position of the printed defect image are labeled to generate a labeled defect image.

[0119] In one possible implementation manner, after the labeled defect image is generated, repeated data, missing value data and / or abnormal value data can be manually deleted, so as to ensure the accuracy, integrity, consistency and usability of the data, so that the subsequent data set can have high quality and credibility.

[0120] In S2013, a plurality of labeled defect images are divided into a training set and a validation set according to a preset ratio.

[0121] Specifically, the terminal device can divide a plurality of labeled defect images into a training set and a validation set according to a preset ratio, wherein the training set and the validation set are data sets, the number of labeled defect images corresponding to the training set is more than the number of labeled defect images corresponding to the validation set, and the ratio between the training set and the validation set can be 7:3, thereby facilitating training of the model.

[0122] In S2014, the training set and the validation set are subjected to data enhancement processing based on a preset mosaic data enhancement algorithm to generate an enhanced training set and an enhanced validation set.

[0123] Specifically, after the terminal device divides a plurality of labeled defect images into a training set and a validation set, the terminal device can first use the computer vision library Open CV to change the labeled defect images in the data set into images with lower resolution. In order to further improve the accuracy and stability of the improved YOLOv5s model and reduce overfitting, the terminal device can perform data enhancement processing on the training set and the validation set based on a preset mosaic data enhancement algorithm to generate an enhanced training set and an enhanced validation set.

[0124] Illustratively, the terminal device can first randomly select four different images, and then randomly select one image in the four images as a center image of a synthesized image. Next, the algorithm will randomly crop three adjacent images around the center image and splice them according to certain rules to form a new synthesized image. In the splicing process, the algorithm will use some random transformations such as rotation, scaling and horizontal flipping to increase the diversity of the synthesized image. Finally, the algorithm will use the label of the center region of the synthesized image as the label of the entire synthesized image to generate new training samples.

[0125] In some possible implementations, in order to efficiently and accurately detect printing defects, after step S201, the method further includes but is not limited to the following steps:

[0126] In S2015, the enhanced training set is input into the improved YOLOv5s model for training. During the training process, the hyperparameters of the improved YOLOv5s model are adjusted based on the enhanced validation set until the improved YOLOv5s model converges or reaches the maximum training round, thereby generating a trained improved YOLOv5s model.

[0127] Specifically, the terminal device can input the enhanced training set into the improved YOLOv5s model for training. During the training process, the hyperparameters of the improved YOLOv5s model are adjusted based on the enhanced validation set until the improved YOLOv5s model converges or reaches the maximum training round, thereby generating a trained improved YOLOv5s model.

[0128] In a possible implementation, the terminal device can draw a LOSS curve and a map curve according to data of the improved YOLOv5s model, and then evaluate the improved YOLOv5s model based on three indexes of precision, recall, and mean average precision.

[0129] Specifically, the precision can be obtained by the following formula:

[0130]

[0131] The recall can be obtained by the following formula:

[0132]

[0133] In the formula, Precision is the precision, Recall is the recall, TP is the number of correctly classified positive samples, that is, the number of samples corresponding to actual positive samples and classified as positive samples by the model, FP is the number of incorrectly classified positive samples, that is, the number of samples corresponding to actual negative samples but classified as positive samples by the model, and FN is the number of incorrectly classified negative samples, that is, the number of samples corresponding to actual positive samples but classified as negative samples by the model.

[0134] Without loss of generality, when determining the mean average precision, the average precision, that is, AP, can be determined according to the area under the Precision-Recall curve, and then the mean average precision can be determined according to the average of APs of various categories.

[0135] Correspondingly, step S200 includes but is not limited to the following steps:

[0136] In S210, the input image to be detected is input into the trained improved YOLOv5s model to determine a feature detection image.

[0137] Specifically, the terminal device can input the image to be detected into the trained improved YOLOv5s model to determine the feature detection image.

[0138] In S300, whether the printed matter to be detected has a printing defect is determined according to the feature detection image.

[0139] Specifically, referring to Figures 12-14 , the terminal device can determine whether the printed matter to be detected has a printing defect according to the feature detection image, for example, when a printing defect is detected in the feature detection image, it indicates that the printed matter to be detected has a printing defect; when no printing defect is detected in the feature detection image, it indicates that the printed matter to be detected has no printing defect.

[0140] It should be noted that, please refer to Table 1 below, Table 1 is obtained from a large number of experimental results; from Table 1, it can be seen that the improved YOLOv5s model provided by the embodiment of the present application is superior to other models, compared with the original YOLOv5s model, the accuracy is improved by 1.7%, the average precision value is improved by 2.9%, and the most advanced performance is realized with the advantages of processing efficiency and accuracy.

[0141] Table 1 is about the change of each index of the improved YOLOv5s model

[0142]

[0143] The implementation principle of the variable data printing defect detection method based on the YOLOv5s model in the embodiment of the present application is as follows: the terminal device can first acquire the to-be-detected image of the to-be-detected printed matter according to the preset industrial camera; then input the to-be-detected image into the improved YOLOv5s model obtained by training to output a feature detection image; and then determine whether the to-be-detected printed matter has a printing defect according to the feature detection image.

[0144] It should be noted that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0145] The embodiment of the present application also provides a variable data printing defect detection system based on a YOLOv5s model, for the convenience of description, only the part related to the present application is shown, as shown in Figure 15 The system 150 includes:

[0146] The to-be-detected image acquisition module 151 is configured to acquire the to-be-detected image of the to-be-detected printed matter based on the preset camera;

[0147] The feature detection image output module 152 is configured to input the to-be-detected image into the improved YOLOv5s model obtained by pre-training to generate a feature detection image;

[0148] The printing defect determination module 153 is configured to determine whether the to-be-detected printed matter has a printing defect according to the feature detection image.

[0149] Optionally, the system 150 further includes:

[0150] The improved YOLOv5s model construction module is configured to construct the improved YOLOv5s model based on the initial YOLOv5s model, the bidirectional multi-scale feature fusion network structure and the CA attention mechanism, wherein the initial YOLOv5s model includes a backbone feature extraction network and a strengthened feature extraction network;

[0151] Optionally, the improved YOLOv5s model construction module comprises:

[0152] An initial YOLOv5s model acquisition submodule is configured to acquire an initial YOLOv5s model.

[0153] A first optimized YOLOv5s model generation submodule is configured to replace a feature pyramid structure of the enhanced feature extraction network in the initial YOLOv5s model with a bidirectional multi-scale feature fusion network structure to generate a first optimized YOLOv5s model.

[0154] An improved YOLOv5s model generation submodule is configured to embed a CA attention mechanism in a backbone feature extraction network corresponding to the first optimized YOLOv5s model to generate an improved YOLOv5s model, wherein the backbone feature extraction network corresponding to the improved YOLOv5s model sequentially connects, in order from a shallow layer to a deep layer, a CBS module, a CBS module, a C3 module, a CA module, a CBS module, a C3 module, a CA module, a CBS module, a C3 module, a CA module, a CBS module, a C3 module, an SPFF module, and a CA module.

[0155] Optionally, the CBS module is composed of a convolution layer, a batch normalization layer, and an activation function layer; the C3 module is composed of a main path and a branch path, the main path includes a BottleNeck module, and the branch path does not include the BottleNeck module; and the improved YOLOv5s model generation submodule comprises:

[0156] A first output feature generation unit is configured to input an input feature x into the CBS module in the main path and the CBS module in the branch path, respectively, and generate an output feature y1 after the two CBS modules perform convolution operations.

[0157] A second output feature generation unit is configured to input the output feature y1 in the main path into a preset BottleNeck module to generate an output feature y4.

[0158] A third output feature generation unit is configured to splice the output feature y1 generated by the branch path and the output feature y4 to generate an output feature y5.

[0159] A fourth output feature generation unit is configured to input the output feature y5 into the CBS module to generate an output feature y6 after the CBS module performs a convolution operation.

[0160] The application also provides a terminal device, which comprises the improved YOLOv5s model construction module. Figure 16As shown, the terminal device 160 of this embodiment includes a processor 161, a memory 162, and a computer program 163 stored in the memory 162 and executable on the processor 161. The processor 161 implements the steps in the above traffic processing method embodiments when executing the computer program 163, for example Figure 1 the steps S100 to S300 shown above; or the processor 161 implements the functions of the modules in the above apparatus when executing the computer program 163, for example Figure 15 the functions of the modules 151 to 153 shown above.

[0161] The terminal device 160 can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The terminal device 160 includes but is not limited to the processor 161 and the memory 162. Those skilled in the art can understand that Figure 16 The terminal device 160 is only an example and does not constitute a limitation on the terminal device 160, which can include more or fewer components than shown, or combine some components, or include different components, for example, the terminal device 160 can also include an input / output device, a network access device, a bus, and the like.

[0162] The processor 161 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0163] The memory 162 can be an internal storage unit of the terminal device 160, for example, a hard disk or a memory of the terminal device 160. The memory 162 can also be an external storage device of the terminal device 160, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 160. Further, the memory 162 can include both the internal storage unit and the external storage device of the terminal device 160. The memory 162 can also store the computer program 163 and other programs and data required by the terminal device 160. The memory 162 can also be used to temporarily store data that has been output or will be output.

[0164] An embodiment of the present application further provides a computer readable storage medium, which stores a computer program. The computer program, when executed by a processor, can implement the steps of each method embodiment described above. The computer program includes computer program code, which can be in a form of source code, object code, executable file, or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, software distribution medium, and the like.

[0165] The above are preferred embodiments of the present application, which do not limit the protection scope of the present application. Any equivalent changes made according to the methods, principles and structures of the present application should be covered within the protection scope of the present application.

Claims

1. A variable data printing defect detection method based on a YOLOv5s model, characterized in that, The method comprises: Based on the preset camera, the image to be detected of the printed matter to be detected is acquired; input the image to be detected into the improved YOLOv5s model trained in advance to generate a feature detection image; determine whether the printed matter to be detected has a printing defect according to the feature detection image; wherein, before the input of the image to be detected into the improved YOLOv5s model trained in advance to generate a feature detection image, the method further comprises: Based on the initial YOLOv5s model, the bidirectional multi-scale feature fusion network structure and the CA attention mechanism, the improved YOLOv5s model is constructed, wherein the initial YOLOv5s model comprises a backbone feature extraction network and a strengthened feature extraction network; wherein, based on the initial YOLOv5s model, the bidirectional multi-scale feature fusion network structure and the CA attention mechanism, the improved YOLOv5s model is constructed, comprising: obtaining an initial YOLOv5s model; for the strengthened feature extraction network in the initial YOLOv5s model: using a bidirectional multi-scale feature fusion network structure to replace the feature pyramid structure of the strengthened feature extraction network, a first optimized YOLOv5s model is generated; embed the CA attention mechanism in the backbone feature extraction network corresponding to the first optimized YOLOv5s model to generate an improved YOLOv5s model, wherein the backbone feature extraction network corresponding to the improved YOLOv5s model is sequentially connected with CBS module, CBS module, C3 module, CA module, CBS module, C3 module, CA module, CBS module, C3 module, CA module, CBS module, C3 module, SPFF module and CA module in order from shallow to deep; wherein, the CBS module is composed of a convolution layer, a batch normalization layer and an activation function layer; the C3 module is composed of a main path and a branch path, the main path includes a BottleNeck module, the branch path does not include a BottleNeck module, and the SiLU activation function adopted by the activation function layer is: ， In the formula, is an image to be detected, is a preset S-shaped growth curve function, is a preset constant; the CA attention mechanism is embedded in the backbone feature extraction network corresponding to the first optimized YOLOv5s model to generate an improved YOLOv5s model, comprising: input the input feature x into the CBS module in the main path and the CBS module in the branch path, and after convolution operation is performed on both CBS modules, the main path and the branch path both generate output feature y1; input the output feature y1 in the main path into the preset BottleNeck module to generate output feature y4; splicing the output feature y1 generated by the branch path with the output feature y4 to generate output feature y5; input the output feature y5 into the CBS module, and generate output feature y6 after convolution operation is performed in the CBS module; wherein, the BottleNeck module includes a residual bypass, and the logical attribute of the residual bypass is True or False; the input of the output feature y1 in the main path into the preset BottleNeck module to generate the output feature y4 comprises: When the logical attribute of the residual bypass is True, input the output feature y1 in the main path to the CBS module, generate the output feature y2, and input the output feature y1 to the residual bypass; input the output feature y2 to the CBS module, generate the output feature y3; stack the output feature y3 and the output feature y1 in the residual bypass to generate the output feature y4; When the logical attribute of the residual bypass is False, input the output feature y1 in the main path to the CBS module, generate the output feature y2; input the output feature y2 to the CBS module, generate the output feature y3; determine the output feature y3 as the output feature y4.

2. The method of claim 1, wherein, The improved YOLOv5s model generated by embedding the CA attention mechanism in the backbone feature extraction network corresponding to the first optimized YOLOv5s model further comprises: perform 1x1 convolution dimension reduction processing on the feature map output by the ninth layer CA module in the backbone feature extraction network to generate a first convolution dimension reduction feature map; input the feature map output by the thirteenth layer CA module in the backbone feature extraction network to the enhanced feature extraction network to perform 2x2 transpose convolution upsampling processing to generate a convolution upsampling feature map; input the feature map output by the sixth layer CA module in the backbone feature extraction network to the enhanced feature extraction network to perform 1x1 convolution dimension reduction processing to generate a second convolution dimension reduction feature map; perform convolution downsampling processing with a size of 3x3 and a step of 2 on the second convolution dimension reduction feature map in the enhanced feature extraction network to generate a third convolution dimension reduction feature map; perform splicing and fusion processing on the first convolution dimension reduction feature map, the convolution upsampling feature map and the third convolution dimension reduction feature map based on a preset Concat function to generate a splicing and fusion feature map; perform 1x1 convolution processing on the splicing and fusion feature map to generate a convolution fusion feature map.

3. The method of claim 2, wherein, The improved YOLOv5s model generated by embedding the CA attention mechanism in the backbone feature extraction network corresponding to the first optimized YOLOv5s model further comprises: Based on the CA attention mechanism, the size of the input feature map is respectively averaged pooled from the horizontal direction and the vertical direction to obtain a horizontal one-dimensional vector and a vertical one-dimensional vector, wherein the size of the input feature map is , the horizontal one-dimensional vector is , and the vertical one-dimensional vector is , is the number of channels, is high, is wide; In the spatial dimension, the horizontal direction one-dimensional vector and the vertical direction one-dimensional vector are spliced to generate a spliced one-dimensional vector, and the channel number is compressed using 1x1 convolution to become ; based on the batch normalization operation and the nonlinear transformation operation, encode the spatial information in the vertical direction and the spatial information in the horizontal direction in the spliced one-dimensional vector to generate a feature vector; perform separation processing on the feature vector to generate a new horizontal direction one-dimensional vector and a new vertical direction one-dimensional vector; respectively perform 1x1 convolution processing on the new horizontal direction one-dimensional vector and the new vertical direction one-dimensional vector to determine the target channel number of the new horizontal direction one-dimensional vector and the new vertical direction one-dimensional vector, wherein the target channel number is the same as the channel number of the input feature map; input the new horizontal direction one-dimensional vector and the new vertical direction one-dimensional vector to a preset activation function respectively to obtain a horizontal output vector and a vertical output vector; perform normalization weighting processing on the horizontal output vector and the vertical output vector to generate a target output vector.

4. The method of claim 1, wherein, Before the constructing the improved YOLOv5s model based on the initial YOLOv5s model, the bidirectional multi-scale feature fusion network structure and the CA attention mechanism, the method further comprises: Based on a preset camera, a set of printed defect images is obtained, wherein the set of printed defect images includes multiple printed defect images; For each printed defect image: label the defect type and defect position of the printed defect image to generate a labeled defect image; According to a preset proportion, multiple labeled defect images are divided into a training set and a validation set; Based on a preset mosaic data enhancement algorithm, the training set and the validation set are subjected to data enhancement processing to generate an enhanced training set and an enhanced validation set.

5. The method of claim 4, wherein, After the constructing the improved YOLOv5s model based on the initial YOLOv5s model, the bidirectional multi-scale feature fusion network structure and the CA attention mechanism, the method further comprises: Input the enhanced training set into the improved YOLOv5s model for training, and in the training process, the hyperparameters of the improved YOLOv5s model are adjusted based on the enhanced validation set until the improved YOLOv5s model converges or reaches the maximum training round, generating a trained improved YOLOv5s model; Correspondingly, the inputting the to-be-detected image into the pre-trained improved YOLOv5s model to generate a feature detection image comprises: Inputting the to-be-detected image into the trained improved YOLOv5s model to generate a feature detection image.

6. A variable data printing defect detection system based on YOLOv5s model, characterized in that, The system comprises: A to-be-detected image acquisition module for acquiring a to-be-detected image of a printed product based on a preset camera; A feature detection image output module for inputting the to-be-detected image into a pre-trained improved YOLOv5s model to generate a feature detection image; A printed defect determination module for determining whether the to-be-detected printed product has a printed defect according to the feature detection image; Before the inputting the to-be-detected image into the pre-trained improved YOLOv5s model to generate a feature detection image, the system further comprises: Based on an initial YOLOv5s model, a bidirectional multi-scale feature fusion network structure and a CA attention mechanism, an improved YOLOv5s model is constructed, wherein the initial YOLOv5s model includes a backbone feature extraction network and a strengthened feature extraction network; The constructing the improved YOLOv5s model based on the initial YOLOv5s model, the bidirectional multi-scale feature fusion network structure and the CA attention mechanism comprises: An initial YOLOv5s model is obtained; For the strengthened feature extraction network in the initial YOLOv5s model: a bidirectional multi-scale feature fusion network structure is used to replace the feature pyramid structure of the strengthened feature extraction network to generate a first optimized YOLOv5s model; In the first optimization YOLOv5s model corresponding to the backbone feature extraction network embedded CA attention mechanism, an improved YOLOv5s model is generated, wherein the improved YOLOv5s model corresponding to the backbone feature extraction network is sequentially connected in the order from shallow to deep CBS module, CBS module, C3 module, CA module, CBS module, C3 module, CA module, CBS module, C3 module, CA module, CBS module, C3 module, SPFF module and CA module; Wherein, the CBS module is composed of convolution layer, batch normalization layer and activation function layer; the C3 module is composed of main path and branch, the main path includes BottleNeck module, the branch does not include BottleNeck module, and the SiLU activation function of the activation function layer is: ， In the formula, is an image to be detected, is a preset S-shaped growth curve function, is a preset constant; the CA attention mechanism is embedded in the backbone feature extraction network corresponding to the first optimized YOLOv5s model to generate an improved YOLOv5s model, comprising: The input feature x is input into the CBS module in the main path and the CBS module in the branch, and the output feature y1 is generated after the convolution operation of the two CBS modules; The output feature y1 in the main path is input into the preset BottleNeck module to generate output feature y4; The output feature y1 generated by the branch is spliced with the output feature y4 to generate output feature y5; The output feature y5 is input into the CBS module, and the output feature y6 is generated after the convolution operation of the CBS module; Wherein, the BottleNeck module includes residual bypass, and the logical attribute of the residual bypass is True or False; the output feature y1 in the main path is input into the preset BottleNeck module to generate output feature y4, including: When the logical attribute of the residual bypass is True, the output feature y1 in the main path is input into the CBS module to generate output feature y2, and the output feature y1 is input into the residual bypass; The output feature y2 is input into the CBS module to generate output feature y3; The output feature y3 is stacked with the output feature y1 in the residual bypass to generate output feature y4; When the logical attribute of the residual bypass is False, the output feature y1 in the main path is input into the CBS module to generate output feature y2; The output feature y2 is input into the CBS module to generate output feature y3; The output feature y3 is determined as output feature y4.

7. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the steps of the method of any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program is executed by the processor to realize the steps of the method of any one of claims 1 to 5.

Images