Visual inspection method for tiny appearance defects of composite polyester card product

Abstract

The invention relates to the technical field of image data processing, in particular to a composite polyester card product-oriented micro appearance defect visual detection method, which comprises the following steps of: acquiring image data, carrying out defect marking by using a roLabelimg tool, and generating a rotary marking data set; carrying out cutting processing by taking the flaw as the center through a cutting algorithm, and amplifying the image blocks by using a data amplification technology; an improved YOLOv8-OBB algorithm is adopted for training and detection, a detection head specially designed for a small target is introduced in the algorithm, and it is ensured that features of the small target are completely captured. An original SPFF module in a YOLOv8-OBB model is replaced with an SPFF-FSCA structure, feature extraction is further optimized through a space and channel attention mechanism, and then the detection precision is improved. The method provided by the invention can efficiently solve the problem of detecting the tiny target flaws in the image, and has relatively high practicability and accuracy.

Description

Technical Field

[0001] The invention relates to the technical field of image data processing, and in particular to a visual detection method for tiny appearance defects of composite polyester card products. Background Art

[0002] In the modern document production process, quality inspection is a crucial link. However, existing quality inspection methods mainly rely on manual work. Generally speaking, a production line often requires 2 to 6 quality inspectors to observe the surface of the document under high light for a long time to find tiny defects at the millimeter level. This method is not only labor-intensive and inefficient, but also limited by manual visual fatigue and subjective judgment, and is prone to missed inspections and false inspections. In addition, the stability and consistency of manual quality inspection are poor, and the inspection results often vary from person to person, resulting in uneven quality levels of quality inspection. At the same time, manual quality inspection has also greatly increased production costs, becoming a bottleneck problem restricting the automation and scale of document production.

[0003] With the development of industrial automation technology, automated inspection based on computer vision has gradually entered people's field of vision. However, traditional machine vision technology has many shortcomings when it comes to detecting tiny defects, especially when the defect area is small or the features are not obvious, conventional image processing methods are difficult to guarantee the accuracy of detection. In addition, machine vision systems are easily disturbed under complex backgrounds or changing lighting conditions, resulting in missed detections and misjudgments. Summary of the invention

[0004] In order to solve the technical problems of low efficiency and poor consistency of manual quality inspection in the production process of modern certificates, as well as poor accuracy of detection of minor defects, the present invention proposes a visual detection method for minor appearance defects of composite polyester card products. The target image data is collected in real time, and the annotation tool is used for manual annotation to generate a rotated annotation data set, and the defect center is sliced ​​by a cutting algorithm. Subsequently, a small target rotation box data set is generated in combination with data augmentation technology. The small target detection head and SPFF-FSCA structure are introduced to improve the YOLOv8-OBB algorithm, optimize feature extraction, and improve the accuracy and integrity of detection.

[0005] The present invention provides a visual detection method for small appearance defects of composite polyester card products, the method comprising:

[0006] Obtaining the appearance surface image corresponding to the composite polyester card product to be inspected;

[0007] According to the appearance surface image, the millimeter-level micro appearance defect detection is performed on the composite polyester card product to be detected through a pre-trained target defect recognition network, wherein the target defect recognition network is a YOLOv8-OBB model with a target detection head added and the original SPFF module replaced with an SPFF-FSCA structure; the target detection head uses the 160×160 feature map output by the Backbone network and combines it with the feature map after upsampling of the Neck layer to generate a detection head capable of detecting small targets at the 4×4 pixel level;

[0008] The training process of the target defect recognition network includes:

[0009] Collect image data of the target surface, and manually annotate the images containing defects using the roLabelimg image annotation tool to generate annotated data sets;

[0010] According to the labeled data set, a cutting algorithm is used to locate each defect in the image, and the defect is used as the center for cutting.

[0011] Perform data augmentation on the sliced ​​images to generate a target rotation box dataset;

[0012] A target defect recognition network is constructed, and the target rotation box dataset is used as the training set and the marked defects are used as training labels to train the constructed target defect recognition network.

[0013] Optionally, performing data augmentation on the sliced ​​image to generate a target rotation box dataset includes:

[0014] The sliced ​​images are rotated, translated, brightness adjusted, contrast adjusted, saturation adjusted, and mirror flipped to form a target rotation box dataset.

[0015] Optionally, the SPFF-FSCA structure is a structure formed by introducing an attention mechanism module FSCA and a channel attention mechanism module after the SPFF module in the YOLOv8-OBB model.

[0016] Optionally, FSCA generates feature attention maps from two independent dimensions, spatial and channel, and multiplies them element-by-element with the original feature map. The feature map F output by the SPFF module is input to FSCA for modeling, and the corresponding formula for performing average aggregation operations on the height and width dimensions is:

[0017]

[0018] in, and are the results of aggregation along the height and width directions, respectively, W and H are the height and width of the feature map, respectively; X(H,i) represents the i-th element of the feature map in the height H direction, and X(j,W) represents the j-th element in the width W direction. This process can capture the global information of the feature map in the spatial dimension;

[0019] The aggregated features and Concatenate and input it into the shared 1×1 convolutional layer F 1 The corresponding formula is:

[0020]

[0021] Among them, [,] represents the concatenation operation;

[0022] The features are adaptively adjusted through the activation function σ, and the formula corresponding to the channel weight coefficient is generated as follows:

[0023] W x ,W y =σ(Split(F))

[0024] Among them, W X and W y represents the two attention weights obtained by segmentation, σ represents the Sigmoid activation function; Split(F) represents the segmentation operation on the feature map F;

[0025] The corresponding formula for processing the feature map F using a 3×3 convolution is:

[0026] W z = relu(F 3×3 (X)

[0027] Among them, relu represents the ReLU activation function, F 3×3 (X) represents a 3×3 convolution operation;

[0028] Combining the channel and spatial attention weights, the feature map is weighted and the formula corresponding to the final optimized feature map is:

[0029] F'=F·W x ·W y ·W z

[0030] Among them, F' is the final optimized feature map, W x , W y , W z are the channel and spatial attention weights respectively.

[0031] The present invention has the following beneficial effects:

[0032] First, the present invention uses a cutting algorithm to cut the image into pieces with the defect as the center, avoiding feature loss or defect blurring caused by an overly large image. This not only reduces the computational complexity and improves the detection speed, but also ensures that each sub-image contains a complete defect area, enhances the algorithm's detection accuracy for small targets, and reduces missed detections and false detections.

[0033] Second, the present invention further introduces slice detection technology on the sub-images after cutting. By detecting multiple sub-images separately and combining the detection results using the non-maximum suppression (NMS) algorithm, the integrity of the target features is ensured, significantly improving the accuracy and consistency of the detection.

[0034] Third, the present invention generates a more diverse rotating frame dataset through multiple data augmentation methods. This augmentation technology not only enhances the generalization ability of the model, but also improves its detection performance under different lighting and complex background conditions.

[0035] Fourth, the present invention improves the YOLOv8-OBB algorithm by adding a small target detection head and replacing the original SPFF module with the SPFF-FSCA structure. This improvement optimizes the feature extraction process to ensure accuracy and efficiency in small target detection. The algorithm can quickly process complex image scenes and is particularly suitable for real-time detection tasks of small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 It is a flow chart of a visual detection method for minute appearance defects of composite polyester card products of the present invention;

[0038] Figure 2 A flowchart of a target defect recognition network training process of the present invention;

[0039] Figure 3 It is a schematic diagram of the structure of the FSCA of the present invention;

[0040] Figure 4 It is a schematic diagram of the framework of the target defect recognition network of the present invention. DETAILED DESCRIPTION

[0041] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the technical solutions proposed by the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0042] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0043] refer to Figure 1 , shows the process of some embodiments of a visual detection method for minor appearance defects of composite polyester card products according to the present invention. The visual detection method for minor appearance defects of composite polyester card products comprises the following steps:

[0044] Step S1, obtaining an appearance surface image corresponding to the composite polyester card product to be inspected.

[0045] The composite polyester card product to be inspected may be a composite polyester ID card to be inspected for defects. The appearance surface image may be a surface image of the composite polyester card product to be inspected.

[0046] In some embodiments, a surface image of the composite polyester card product to be inspected may be captured by a camera as an appearance surface image.

[0047] Step S2, based on the appearance surface image, the pre-trained target defect recognition network is used to perform millimeter-level micro appearance defect detection on the composite polyester card product to be inspected.

[0048] Among them, the target defect recognition network is a YOLOv8-OBB model with an added target detection head and the original SPFF module replaced with the SPFF-FSCA structure. The SPFF-FSCA structure is formed by introducing the attention mechanism module FSCA and the channel attention mechanism module after the SPFF module in the YOLOv8-OBB model. The target detection head uses the 160×160 feature map output by the Backbone network and combines it with the upsampled feature map of the Neck layer to generate a detection head that can detect small targets at the 4×4 pixel level.

[0049] In some embodiments, the appearance surface image can be input into a pre-trained target defect recognition network, and the target defect recognition network can be used to detect millimeter-level tiny appearance defects of the composite polyester card product to be inspected.

[0050] refer to Figure 2 , shows the process of the target defect recognition network training process, which may include the following steps:

[0051] Step 201, collect image data of the target surface, and manually annotate the image containing defects using the roLabelimg image annotation tool to generate an annotation data set.

[0052] As an example, the image data of the target surface can be collected in real time, and the image containing defects can be manually annotated using the roLabelimg image annotation tool to generate a rotated annotation dataset. The rotated annotation dataset is also an annotation dataset. The collected image data of the target surface can be image data obtained by using an industrial camera to collect data on the surface of the document. The manually annotated defect data can be used as the initial dataset.

[0053] Step 202: Based on the labeled data set, a cutting algorithm is used to locate each defect in the image, and a cutting process is performed with the defect as the center.

[0054] As an example, based on the annotated dataset, a cutting algorithm is used to locate each defect in the document image and cut the image into pieces with the defect as the center. The cutting algorithm calculates the center point based on the bounding box of the object and cuts out an image block with the center as the reference according to the set window size. After the cutting is completed, the corresponding cut image and new annotation file are generated.

[0055] Step 203: perform data augmentation on the sliced ​​image to generate a target rotation box data set.

[0056] Among them, the target rotation box dataset is also called the small target rotation box dataset.

[0057] As an example, in order to enhance the generalization ability of the detection model, the sliced ​​images are further subjected to data augmentation processing. Specifically, the sliced ​​images can be rotated, translated, brightness adjusted, contrast adjusted, saturation adjusted, and mirror flipped to form a target rotation box data set.

[0058] It should be noted that a variety of data augmentation methods, including rotation, translation, brightness adjustment, contrast adjustment, saturation adjustment, and mirror flipping, are used to increase the diversity of image data, and different shooting perspectives are simulated through affine transformation to form a new data set.

[0059] Step 204 , construct a target defect recognition network, and use the target rotation box data set as a training set and the marked defects as training labels to train the constructed target defect recognition network.

[0060] As an example, a YOLOv8-obb ​​model can be constructed, and the FSCA space and channel attention mechanism module can be introduced after the SPFF module in the backbone network to form an SPFF-FSCA structure, replace the SPFF module in the YOLOv8-obb ​​model, add a small target detection head, ensure the integrity of the small target object features, and use the resulting network as a target defect recognition network. The small target detection head is the above-mentioned target detection head. Among them, the method for obtaining the small target detection head is: based on the feature map of 160×160 resolution output by the Backbone network of the model, combined with the Neck layer upsampling operation, a detection head specifically for detecting small targets is integrated. This detection head can effectively process small targets of 4×4 pixels.

[0061] It should be noted that due to the complexity of the document pattern and the variety of colors, some small defects or defects with similar colors to the document pattern are difficult to identify. In order to improve the model's ability to extract defect features in complex backgrounds, the present invention introduces the FSCA (channel and spatial adaptive attention mechanism) module after the SPFF module in the backbone network of the YOLOv8-OBB model to generate the SPFF-FSCA structure and replace the original SPFF module. The structure of FSCA can be as follows Figure 3 The framework of the target defect recognition network can be shown as follows Figure 4 The introduction of the FSCA module aims to enhance the model's ability to extract features of small defects in complex backgrounds, which can be specifically done by the following steps:

[0062] In the first step, FSCA generates feature attention maps from two independent dimensions, space and channel, and multiplies them element by element with the original feature map to achieve adaptive optimization and enhance the expression ability of key feature signals. The feature map F output by the SPFF module is input to FSCA for modeling, and average aggregation operations are performed in the height and width dimensions. That is to say, the FSCA module performs global average pooling operations on the input feature map in the height and width directions respectively to capture the global information of the feature map in these two dimensions. Through this operation, the aggregation results of the feature map in the height and width directions can be obtained, and the corresponding formula is:

[0063]

[0064] in, and are the results of aggregation along the height and width directions, respectively, W and H are the height and width of the feature map, respectively. X(H,i) represents the i-th element of the feature map in the height H direction, and X(j,W) represents the j-th element in the width W direction. This process can capture the global information of the feature map in the spatial dimension.

[0065] In the second step, the aggregated features and The concatenation is combined into a feature map, which represents the combination of global information in two dimensions and is input into the shared 1×1 convolutional layer F 1 The corresponding formula is:

[0066]

[0067] Among them, [,] represents the concatenation operation.

[0068] In the third step, the features are adaptively adjusted through the activation function σ, and the formula corresponding to the channel weight coefficient is generated as follows:

[0069] W x ,W y =σ(Split(F))

[0070] Among them, W X and W y Represents the two attention weights obtained by segmentation or is called a weight tensor. σ represents the Sigmoid activation function. Split(F) represents the segmentation operation on the feature map F.

[0071] In the fourth step, in order to enhance the spatial expression ability of the feature map, a 3×3 convolution is used to process the feature map F. The corresponding formula is:

[0072] W z = relu(F 3×3 (X)

[0073] Where relu represents the ReLU activation function. 3×3 (X) represents a 3×3 convolution operation.

[0074] In the fifth step, the feature map is weighted by combining the channel and spatial attention weights to obtain the formula corresponding to the final optimized feature map:

[0075] F'=F·W x ·W y ·W z

[0076] Among them, F' is the final optimized feature map, W x , W y , W z are the channel and spatial attention weights respectively.

[0077] Through this series of operations, the FSCA module achieves adaptive optimization in channel and spatial dimensions, effectively improving the discrimination ability of feature maps and enabling the model to capture the information of the target area more accurately.

[0078] It should be noted that adding a small target detection head ensures the integrity of the small target object features. YOLOv8-obb ​​has multiple detection heads by default, corresponding to feature maps of different scales to detect targets of different sizes. When the input image size is 640x640, the output feature map sizes of the detection heads at each layer are 32×32, 16×16, and 8×8 pixels, respectively. These detection heads can effectively detect targets that are 0.0125 times the size of the minimum input image.

[0079] In order to improve the effect of small target detection, especially when the aspect ratio of the target is lower than 0.0125, the existing feature map resolution may not be sufficient to capture the features of small targets. To this end, the present invention proposes a new detection head, which uses the 160×160 feature map output by the Backbone network and combines it with the upsampled feature map of the Neck layer to generate a detection head capable of detecting small targets at the 4×4 pixel level, thereby enhancing the model's perception of small targets. The calculation formula is:

[0080]

[0081] Among them, T w and T h are the width and height of the feature map output by the detection head. pixel A represents the receptive field size of each pixel. input Represents the area of ​​the input image (i.e., the width multiplied by the height of the input image).

[0082] It should be noted that, during the detection process of the model, the detection module sets the segmentation overlap rate according to the preset sub-image size and target size, and divides the large image into multiple sub-images. The setting of the overlap rate ensures that each target can be fully presented in at least one sub-image, and ensures that the target features will not be lost by cutting. After each sub-image is detected separately, the present invention merges the detection results of multiple sub-images through the non-maximum suppression (NMS) algorithm, removes overlapping detection frames, and ensures the uniqueness and accuracy of the reasoning results. This operation avoids errors caused by multiple detections of the same target during the detection process.

[0083] Secondly, the detected document image is compared with the corresponding real image to analyze possible abnormal behaviors or defects. The detection module predicts and infers the processed image frames and finally outputs the detection results of abnormal behaviors. In the document production process, this process can be used to automatically screen defects, mark defective documents and process them, greatly improving the quality control level of the production line.

[0084] Based on the same inventive concept as the above method embodiment, the present invention provides a visual inspection method for small appearance defects of composite polyester card products, including: a data acquisition module, a data set production module, a data set preprocessing module, a model building module, a training module and a detection module. Among them, the detection module is used to set the segmentation overlap rate according to the preset sub-image size and target size, divide the image into multiple sub-images, and set the overlap rate to ensure that the target has complete features in at least one sub-image. Target detection is performed on each sub-image, and the detection frame of each sub-image is obtained. The non-maximum suppression (NMS) algorithm is used to merge the detection frames of multiple sub-images, eliminate duplicate detection results, and ensure the integrity of the target features and the accuracy of the reasoning results.

[0085] In summary, the present invention improves the YOLOv8-OBB algorithm, adds a small target detection head, and replaces the original SPFF module with the SPFF-FSCA structure. This improvement optimizes the feature extraction process to ensure accuracy and efficiency in small target detection. The algorithm can quickly process complex image scenes and is particularly suitable for real-time detection tasks of small targets.

[0086] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A visual inspection method for minor appearance defects of composite polyester card products, characterized in that: The following steps are involved: Obtaining the appearance surface image corresponding to the composite polyester card product to be inspected; According to the appearance surface image, the millimeter-level micro appearance defect detection is performed on the composite polyester card product to be detected through a pre-trained target defect recognition network, wherein the target defect recognition network is a YOLOv8-OBB model with a target detection head added and the original SPFF module replaced with an SPFF-FSCA structure; the target detection head uses the 160×160 feature map output by the Backbone network and combines it with the feature map after upsampling of the Neck layer to generate a detection head capable of detecting small targets at the 4×4 pixel level; The training process of the target defect recognition network includes: Collect image data of the target surface, and manually annotate the images containing defects using the roLabelimg image annotation tool to generate annotated data sets; According to the labeled data set, a cutting algorithm is used to locate each defect in the image, and the defect is used as the center for cutting. Perform data augmentation on the sliced ​​images to generate a target rotation box dataset; A target defect recognition network is constructed, and the target rotation box dataset is used as the training set and the marked defects are used as training labels to train the constructed target defect recognition network.

2. A visual inspection method for minor appearance defects of composite polyester card products according to claim 1, characterized in that: The data augmentation is performed on the sliced ​​image to generate a target rotation frame data set, including: The sliced ​​images are rotated, translated, brightness adjusted, contrast adjusted, saturation adjusted, and mirror flipped to form a target rotation box dataset.

3. A visual inspection method for minor appearance defects of composite polyester card products according to claim 1, characterized in that: The SPFF-FSCA structure is formed by introducing the attention mechanism module FSCA and the channel attention mechanism module after the SPFF module in the YOLOv8-OBB model.

4. A visual inspection method for minor appearance defects of composite polyester card products according to claim 3, characterized in that: FSCA generates feature attention maps from two independent dimensions, space and channel, and multiplies them element-by-element with the original feature map. The feature map F output by the SPFF module is input to FSCA for modeling. The corresponding formula for the average aggregation operation in the height and width dimensions is: in, and are the results of aggregation along the height and width directions, respectively. W and H are the height and width of the feature map, respectively. X(H,i) represents the i-th element of the feature map in the height H direction, and X(j,W) represents the j-th element in the width W direction. This process can capture the global information of the feature map in the spatial dimension. The aggregated features and Concatenate and input it into the shared 1×1 convolutional layer F 1 The corresponding formula is: Among them, [,] represents the concatenation operation; The features are adaptively adjusted through the activation function σ, and the formula corresponding to the channel weight coefficient is generated as follows: IN x ,IN y =σ(Split(F)) Among them, W X and W y represents the two attention weights obtained by segmentation, σ represents the Sigmoid activation function; Split(F) represents the segmentation operation on the feature map F; The corresponding formula for processing the feature map F using a 3×3 convolution is: IN z =relu(F 3×3 (X)) Among them, relu represents the ReLU activation function, F 3×3 (X) represents a 3×3 convolution operation; Combining the channel and spatial attention weights, the feature map is weighted and the formula corresponding to the final optimized feature map is: F'=F·W x ·W y ·W z Among them, F' is the final optimized feature map, W x , W y , W z are the channel and spatial attention weights respectively.

Images