Reflective film defect detection system and defect identification method based on deep learning
Through the deep learning-based reflective film defect detection system, combined with the YOLOV8-DEW algorithm and Efficient detection head, the problem of low detection efficiency and accuracy of surface defect detection of reflective film is solved, efficient and intelligent detection is achieved, and night driving safety is improved sex.
Patent Information
- Application Number
- CN202410936785.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-12-06
Smart Images

Figure CN119086554A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of industrial automation technology, and in particular to a reflective film defect detection system and a defect recognition method based on deep learning. Background Art
[0002] According to statistics from relevant departments, the use of reflective materials to set up eye-catching traffic signs and vehicle license plates under no lighting conditions or wearing clothes and hats decorated with reflective materials can reduce the traffic accident rate by 30% to 40%. According to comprehensive analysis, the traffic accident rate in major cities across the country is significantly higher at night than during the day, and the peak of accidents is concentrated between 8 and 10 p.m. One of the reasons is the low visibility on the road. The emergence of reflective materials has successfully solved the problem of "seeing" and "being seen" during night driving. The directional reflective properties of reflective materials determine that they have become a necessary consumable for ensuring people's livelihood safety, effectively reducing the accident rate, improving driving efficiency, and ultimately becoming a traffic safety essential consumable.
[0003] Reflective film is composed of a reflective layer, a base film, a pressure-sensitive adhesive and a release layer. Its main defects are generated during plastic forming, and the types of defects produced include delamination, scratches, breakage, stains, bubbles, insect prints, etc. The reflection mode of reflective film is retroreflective, and the irradiated light returns in the opposite direction at a close angle to the incident light. Therefore, it is necessary to avoid defects on the surface of the reflective film to avoid affecting the reflective performance. At present, manual quality inspection methods generally include two options: the first option is to stop production and adjust the machine in time when defects are found in the newly produced reflective film to improve the production quality of the reflective film. Although this method can stop and adjust the machine in time, the complex process makes the adjustment more cumbersome and the efficiency is reduced. The second option is to produce the reflective film by machine first, and then screen out the defective reflective film through manual quality inspection. The disadvantage is that the workers have a high labor intensity of using their eyes, and the accuracy begins to gradually decrease over time. This project aims to meet the needs of industrial automation production lines for large-scale, highly automated reflective film surface defect detection, and efficiently produce road traffic reflective film that meets GB / T 18833-2012. It is of great significance to ensure the safe, stable and efficient operation of my country's road traffic and the safety of people's property.
[0004] At present, the detection of surface defects of reflective film mainly relies on directly observing the surface of the reflective film with the naked eye or a magnifying glass to check whether it is flat, free of bubbles, delamination, scratches and other obvious defects. Reflective film with poor surface quality can be quickly identified. However, this detection method is often limited by the judgment of the human eye. As time goes by, the detection efficiency and accuracy will decrease, and it is impossible to achieve accurate grade classification and meet the needs of modern industrial production. Therefore, it is necessary to propose a new solution to the above problems. Summary of the invention
[0005] The purpose of the present invention is to provide a reflective film defect detection system and defect recognition method based on deep learning, aiming to use machine vision detection to achieve faster detection efficiency and higher accuracy, reduce labor costs, and meet the needs of large-scale, highly automated production lines. At the same time, according to the position information of the target detection, a more reasonable cutting solution is provided to solve the technical problems raised in the background technology.
[0006] To achieve the above-mentioned object, the present invention provides the following technical solutions: A reflective film defect detection system based on deep learning, comprising a reflective film guide structure, a detection control component and four sets of image acquisition modules;
[0007] The reflective film guiding structure comprises two conveying wheels and a conveying belt, the conveying belt is arranged outside the conveying wheels, and the reflective film guiding structure is used to drive the reflective film to move;
[0008] The detection control component includes an encoder, a PLC and a desktop host, and the desktop host is equipped with detection software;
[0009] The image acquisition module includes an array CCD camera and a matching industrial lens;
[0010] The recognition method of the reflective film defect detection system adopts the YOLOV8-DEW algorithm detection model.
[0011] The reflective film defect recognition method based on deep learning includes at least the following steps:
[0012] S1: preparing an image dataset, wherein the preparation of the image dataset at least includes image acquisition, data enhancement and dataset labeling;
[0013] S2: A deformable attention mechanism is constructed. Its innovation is that instead of calculating the offset independently for each query, the entire feature map shares a set of uniformly shifted keys and values. This design is particularly efficient when processing large feature maps, because all queries can perform self-attention operations using the same shifted key-value pairs, which significantly simplifies the calculation process.
[0014] S3: Build an Efficient detection head. Compared with the original detection head, the Efficient detection head reduces computational complexity and memory usage by optimizing the network structure, reducing redundant connections, and using more efficient convolution methods, thereby improving the detection speed. It has stronger feature extraction capabilities and can more accurately extract features useful for detection tasks from input images. Through the method of compound scaling, it can flexibly adjust the width, depth, and resolution of the model according to existing computing resources and needs, achieving the best balance between detection performance and efficiency. Therefore, the Efficient detection head adopts a more efficient and simpler network structure, has better generalization capabilities, and can perform well on the dataset of surface defects of reflective films;
[0015] S4: Design loss function;
[0016] S5: set parameters;
[0017] S6: Model comparison and detection effect, ablation experiment is carried out on the model to prove the effectiveness of the model.
[0018] Furthermore, the image acquisition includes at least the following steps:
[0019] There are 4 sets of image acquisition modules. When the reflective film moves to the right driven by the reflective film guide structure, the displacement signal of the reflective film is sent to the PLC through the installed encoder;
[0020] The PLC records the displacement signal output by the encoder. Whenever the reflective film moves over an integer multiple of 0.6 meters, the PLC sends a photo signal to the four image acquisition modules. The four array cameras record the reflective film surface image of the current area and send it to the desktop host. The size of the area captured by each image sent is 300mm (width) * 600mm (length).
[0021] At the same time, the PLC also sends the current position signal to the desktop host.
[0022] Furthermore, the data enhancement is used to increase the training sample data to achieve the purpose of expanding the data set, and the expanded data set is as close as possible to the actual distribution of the real data set, thereby improving the detection accuracy. In addition, data enhancement enables the model to learn more robust features, thereby improving the generalization ability of the model;
[0023] The data enhancement includes geometric transformation and pixel transformation. The geometric transformation is to change the shape of the image, but not the image itself, but only to redistribute part of the image or the pixels of the image.
[0024] The geometric transformation includes at least flipping, rotating, shifting, cropping, deforming and scaling;
[0025] The pixel transformation changes the internal information of the image itself, and the pixel transformation at least includes color, noise, blur, erase and fill.
[0026] Furthermore, the dataset annotation is to use Labelme to annotate the prepared image data and determine the target area position. Labelme is an image annotation tool that can realize various image annotation tasks, such as rectangular box annotation, polygon annotation, line segment annotation, key point annotation, etc. The related files generated by Labelme tagging can detect target detection, semantic segmentation, and instance segmentation tasks respectively. When processing target detection tasks, the rectangular box annotation method is often used.
[0027] Furthermore, the deformable attention mechanism is:
[0028] For the input feature map x∈R H×W×C , generating a point The uniform grid composed of the two components is used as a reference;
[0029] The grid size is downsampled from the input feature size by a factor r, The reference point values are linearly spaced 2D coordinates {(0,0),…,(H G -1,W G -1)}, and then according to the grid shape H G ×W G Normalize them to the range [-1, +1], where (-1, -1) represents the upper left corner and (+1, +1) represents the lower right corner;
[0030] To obtain the offset of each reference point, the feature map is linearly mapped to the query mark, using Query = xW q Represented, and then introduced into a lightweight network θ offset Take the feature query as input and generate the offset Δp=θ corresponding to all reference points offset (q). In order to stabilize the training process, the amplitude of Δp is scaled by a predefined factor s to prevent the offset from being too large, i.e., s→Δp,tanh(Δp);
[0031]
[0032] represents the keys and values embedded after deformation, θ offset (q) Lightweight network, φ(·;·) sampling function.
[0033] First, the position of the input feature map is sampled, then the moving points are bilinearly interpolated, and then sampled again as the key and value, and then the projection matrix is;
[0034]
[0035] where g(a,b) = max(0,1-|ab|) and (r x ,r y ) index z∈R H×W×C All positions on;
[0036] Since g is only close to (p x ,p y ) is non-zero at the four integral points, so the above formula is simplified to the weighted average of the four positions;
[0037] Combined with the existing method, multi-head attention is performed on q, k, v, and the relative position offset R is used. The output formula of the attention head is:
[0038]
[0039] in Represents the location embedding of the sampling points.
[0040] Furthermore, the network architecture of the deformable attention mechanism adopts Deformable Attention Transformer (DAT), which can be widely applied to visual tasks that require multi-scale feature maps.
[0041] Furthermore, the application of the Efficient detection head includes at least the following steps:
[0042] First, the 80×80 scale feature layer in Backbone and the upsampling feature layer in Neck are stacked upwards;
[0043] After C2f and upsampling, a deep semantic feature layer containing small target feature information is obtained, which is stacked with the shallow position feature layer in Backbone.
[0044] The 160×160 scale fusion feature layer is supplemented and improved to express the semantic features and location information of small targets, and finally sent to the Efficient detection head in the Head through C2f.
[0045] Furthermore, the loss function design at least includes the following steps:
[0046] The loss function is set to Wise-IoU v1, which introduces a distance attention mechanism. When the anchor box and the target box overlap well, the weight of the geometric metric is dynamically adjusted through distance attention, thereby weakening the impact on the model, that is, R WIoU∈[1,e), which can focus on the bounding box loss of common quality anchor boxes, see the following formula:
[0047]
[0048] Where: x, y are the center coordinates of the prediction box, x gt ,y gt is the center coordinate of the real box, W g , H g is the minimum bounding rectangle of the predicted box and the real box, R WIoU is the non-monotonic focusing coefficient;
[0049] The dynamic non-monotonic focusing mechanism uses "outlier degree" instead of IoU to evaluate the quality of the anchor frame. A small outlier degree β means that the anchor frame is of high quality and effective. The outlier degree is determined by the intersection-over-union ratio calculated for each batch and is a dynamic non-monotonic focusing value.
[0050] This paper introduces a gradient gain, which is used to finely control the shape of the focusing coefficient curve through two key hyperparameters, so as to dynamically adjust the loss weights for difficult or poor-quality samples.
[0051] Furthermore, the parameter setting at least includes the following steps:
[0052] The collected reflective film images were randomly divided into training set, validation set, and test set in a ratio of 7:1:2 to prevent the model from overfitting;
[0053] The input image is 640x640 pixels, the model is trained for 400 cycles, the initial learning rate is 0.001, the batch size is 16, and the weight decay is 0.0005;
[0054] The optimizer uses stochastic gradient descent (SGD) and adopts a cosine annealing learning strategy to dynamically adjust the learning rate to prevent model training from falling into local optimality.
[0055] Compared with the prior art, the present invention has the following beneficial effects:
[0056] 1. The present invention realizes the intelligent detection of reflective film surface defects, replaces the cumbersome manual operation, and can automatically identify and detect reflective film surface defects without manual judgment.
[0057] 2. The present invention has high working efficiency. Compared with traditional manual detection and identification, one person can operate multiple robots for automatic detection at the same time, which solves the problem of low efficiency of reflective film surface detection, meets the demand for reflective film for road traffic, and significantly improves the safety of night driving.
[0058] 3. The present invention realizes the real-time detection of the reflective film surface. By real-time processing of images and accurately judging the state of the reflective film, any potential problems, such as surface unevenness, bubbles, scratches or stains, can be immediately discovered and warned.
[0059] 4. The present invention can capture and analyze the defects on the surface of the reflective film in real time through the image recognition system. Defects such as bubbles, scratches, stains, etc. often reduce the reflective effect of the reflective film. Data processing is performed on the identified defect position information to automatically generate the optimal cutting plan. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0061] Figure 1 It is a schematic diagram of the detection system of the present invention;
[0062] Figure 2 This is the YOLOV8 network structure diagram of the present invention;
[0063] Figure 3 This is the YOLOV8-DEW network structure diagram of the present invention;
[0064] Figure 4 Schematic diagram of the deformable attention mechanism of the present invention;
[0065] Figure 5 Schematic diagram of the DeformableAttention Transformer of the present invention;
[0066] Figure 6 This is a schematic diagram of the structure of the Efficient detection head of the present invention;
[0067] Figure 7 It is a schematic diagram of the intersection-combination ratio of the present invention;
[0068] Figure 8 It is a relationship diagram between the outlier degree and the gradient gain r of the present invention;
[0069] Fig. 9 It is a schematic diagram of the detection effect of the present invention. DETAILED DESCRIPTION
[0070] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.
[0071] Embodiment 1:
[0072] See also Figure 1-Figure 3 , a reflective film defect detection system based on deep learning, including a reflective film guide structure, a detection control component and four sets of image acquisition modules;
[0073] The reflective film guiding structure includes two conveying wheels and a conveying belt. The conveying belt is arranged outside the conveying wheels. The reflective film guiding structure is used to drive the reflective film to move.
[0074] The detection control components include encoders, PLCs and desktop hosts, which are equipped with detection software;
[0075] The image acquisition module includes an array CCD camera and a matching industrial lens;
[0076] The recognition method of the reflective film defect detection system adopts the YOLOV8-DEW algorithm detection model.
[0077] After the starting mark position of the reflective film appears, the operator clicks the "Start Detection" button of the "Detection Software", so that the PLC starts counting the output pulses of the encoder, thereby ensuring that the encoder counting start position (0 meters) is aligned with the starting mark position of the reflective film.
[0078] Four groups of image acquisition modules (cameras and array lenses) are arranged in the width direction of the reflective film. When the reflective film moves to the right under the guidance of the reflective film guide structure, the displacement signal of the reflective film movement is sent to the PLC through the installed encoder. The PLC records the displacement signal output by the encoder. Whenever the reflective film moves an integer multiple of 0.6 meters, the PLC sends a photo signal to the four groups of image acquisition modules. The four array cameras record the surface image of the reflective film in the current area and send it to the desktop host. The size of the area captured by each image sent is 300mm (width) * 600mm (length). At the same time, the PLC also sends the current position signal to the desktop host.
[0079] The "image recognition module" processes the 4 images currently obtained to determine whether there are various defects on the reflective film. If there are defects in the reflective film, the "image recognition module" will further process it, including determining the type of defect (damage, stains, wrinkles, etc.) and size, extracting its coordinate position in the current image, and combining the position of the current image block generated by the encoder to jointly generate the absolute position of the defect in the entire roll of reflective film. Afterwards, the "reflective film defect online detection system software" saves the defect type, size, absolute position, and image to the database to provide data support for the subsequent cutting of reflective film. If there are no above defects, the current image is not saved. The reflective film does not need to pause during the entire process and runs continuously until the entire roll of reflective film has been inspected.
[0080] The recognition method of the reflective film defect detection system uses the YOLOV8-DEW algorithm detection model to accurately locate and identify the surface defects of the reflective film. This model is the latest version of the YOLO target detection and image segmentation model. It is a cutting-edge, state-of-the-art (SOTA) model. The complete structure of the original YOLOV8 is as follows Figure 2 As shown;
[0081] YOLOV8 is a single-stage target detection model that is effectively improved on the basis of YOLOv5. The YOLO8 model consists of three parts: backbone feature extraction network (Backbone), neck (Neck), and detection head (Head). The backbone feature extraction network uses Conv module, CSPDarknet53 and SPPF structure to extract features from the target area, and uses residual connection and bottleneck structure to reduce the size of the network and improve performance; the neck uses PAN-FPN network skeleton to fuse the effective feature layer extracted by the backbone network, so that the network can better adapt to target detection of different scales; the detection head mainly uses Anchor free decoupling head to predict the category probability, location information and confidence of the target area respectively, and finally merge the prediction results of three different scales, and finally output the prediction results. YOLOV8 adopts the idea of Anchor free detection head, gets rid of the constraints of prior box, and reduces the calculation amount of the model and time cost compared with Anchor based. The positive and negative sample allocation strategy in YOLOV8 uses Task-AlignedAssigner to alleviate the problem of imbalance between positive and negative samples. This is a dynamic allocation strategy that can dynamically adjust the weights according to the sample characteristics during training. The YOLOV8 model has high detection accuracy and fast detection speed, and has been widely used in various tasks. This paper selects the YOLOV8 model as the basic model for subsequent research on reflective film defect detection, and optimizes and improves the YOLOV8 model for accurate and real-time detection of defects generated by the reflective film surface.
[0082] In order to improve the detection accuracy of the model and achieve comprehensive and accurate detection, the backbone feature extraction network capability, loss function and target detection of different scales of the YOLOV8 model are optimized, so that the various detection results obtained by the model can reach the expected goals. The complete structure of the improved and optimized YOLOV8-DEW is as follows: Figure 3 as shown.
[0083] Embodiment 2:
[0084] The reflective film defect recognition method based on deep learning, based on the above-mentioned embodiment 1, at least includes the following steps:
[0085] S1: Produce an image dataset, which at least includes image acquisition, data enhancement, and dataset labeling;
[0086] Image acquisition includes at least the following steps:
[0087] There are 4 sets of image acquisition modules. When the reflective film moves to the right driven by the reflective film guide structure, the displacement signal of the reflective film is sent to the PLC through the installed encoder;
[0088] The PLC records the displacement signal output by the encoder. Whenever the reflective film moves over an integer multiple of 0.6 meters, the PLC sends a photo signal to the four image acquisition modules. The four array cameras record the reflective film surface image of the current area and send it to the desktop host. The size of the area captured by each image sent is 300mm (width) * 600mm (length).
[0089] At the same time, the PLC also sends the current position signal to the desktop host.
[0090] Data augmentation is used to increase the training sample data to expand the data set, and the expanded data set is as close as possible to the actual distribution of the real data set, thereby improving the detection accuracy. In addition, data augmentation enables the model to learn more robust features, thereby improving the generalization ability of the model;
[0091] Data enhancement includes geometric transformation and pixel transformation. Geometric transformation is to change the shape of the image, but not the image itself. It only redistributes part of the image or the pixels of the image.
[0092] Geometric transformations include at least flipping, rotation, shifting, cropping, deformation, and scaling;
[0093] Pixel transformation changes the internal information of the image itself, and pixel transformation includes at least color, noise, blur, erase and fill.
[0094] (1) Rotation
[0095] Image rotation is a commonly used data augmentation method. It rotates the original image at a random angle and crops the rotated image to the same size as the original image. The missing parts of the image are filled with pure colors. Rotation tasks are often used in tasks that are insensitive to direction.
[0096] (2) Scaling
[0097] Scaling is the operation of enlarging or reducing an image by a certain ratio. The basic principle is to calculate the pixel values of the target image according to certain rules based on the pixel values of the original image. When using scaling, although the data set can be enhanced, the processed image may be distorted after changing the image size, that is, the image will be jagged (reduction) or blurred (enlargement).
[0098] (3) Cutting
[0099] Cropping is to randomly crop a part of the original image containing the target area into a rectangle. The size of the cropped image usually changes. When the target area is small, random cropping will crop more of the image background area, making it easy to ignore the information of the image background area. However, it can also highlight the characteristic information of the target area.
[0100] (4) Noise
[0101] There are many ways to add noise to an image. Common noises include Gaussian noise and salt and pepper noise. Salt and pepper noise, also known as impulse noise, randomly adds white or black dots to the original image. The pixel value of salt and pepper noise is 0 or 255. Gaussian noise refers to a type of noise whose probability density function follows a Gaussian distribution. Adding noise to an image increases the interference to the image. The more noise is added, the more serious the image damage. However, adding noise in moderation can enhance the model's learning of regional features and enhance the model's generalization ability.
[0102] (5) Color change
[0103] Color transformation includes adjusting the color depth and brightness of the image. It changes the original content of the image by increasing or decreasing the color component in a certain color space of the image, or changing the order of the color channels. This changes the color and brightness of the original image and improves the learning ability of the model.
[0104] Data enhancement increases the number of images and alleviates the scenario of insufficient data when performing deep learning tasks. It has been widely used in the field of images. It can effectively reduce the phenomenon of overfitting during model training and improve the robustness of deep learning algorithms. The present invention collects 300 images through image acquisition. After data enhancement and expansion of the data set, the data obtained is 1500 reflective film images. It is divided into training set, verification set and test set according to 7:1:2, and the reliability of the algorithm is verified.
[0105] Dataset annotation is to use Labelme to annotate the prepared image data and determine the location of the target area. Labelme is an image annotation tool that can perform a variety of image annotation tasks, such as rectangular box annotation, polygon annotation, line segment annotation, key point annotation, etc. The related files generated by Labelme tagging can be used to detect target detection, semantic segmentation, and instance segmentation tasks respectively. When processing target detection tasks, the rectangular box annotation method is often used.
[0106] S2: Build a deformable attention mechanism. The advantage of the deformable attention mechanism is that it no longer calculates the offset for each query separately, but instead shares a set of shifted keys and values for each feature. For larger feature maps, the same shifted keys and values are used to perform the self-attention mechanism, which simplifies the calculation.
[0107] See also Figure 4 , the principle of the deformable attention mechanism is:
[0108] For the input feature map x∈R H×W×C , generating a point The uniform grid composed of the two components is used as a reference;
[0109] The grid size is downsampled from the input feature size by a factor r, The reference point values are linearly spaced 2D coordinates {(0,0),…,(H G -1,W G -1)}, and then according to the grid shape H G ×W G Normalize them to the range [-1, +1], where (-1, -1) represents the upper left corner and (+1, +1) represents the lower right corner;
[0110] To obtain the offset of each reference point, the feature map is linearly mapped to the query mark, using Query = xW q Represented, and then introduced into a lightweight network θ offset Take the feature query as input and generate the offset Δp=θ corresponding to all reference points offset (q). In order to stabilize the training process, the amplitude of Δp is scaled by a predefined factor s to prevent the offset from being too large, i.e., s→Δp,tanh(Δp);
[0111]
[0112] represents the keys and values embedded after deformation, θ offset (q) Lightweight network, φ(·;·) sampling function.
[0113] First, the position of the input feature map is sampled, then the moving points are bilinearly interpolated, and then sampled again as the key and value, and then the projection matrix is;
[0114]
[0115] where g(a,b) = max(0,1-|ab|) and (r x ,r y ) index z∈R H×W×C All positions on;
[0116] Since g is only close to (p x ,p y ) is non-zero at the four integral points, so the above formula is simplified to the weighted average of the four positions;
[0117] Combined with the existing method, multi-head attention is performed on q, k, v, and the relative position offset R is used. The output formula of the attention head is:
[0118]
[0119] in Represents the location embedding of the sampling points.
[0120] See also Figure 5 ,Network architecture of deformable attention mechanism;
[0121] In terms of network architecture, Deformable Attention Transformer (DAT) can be widely used in visual tasks that require multi-scale feature maps. Figure 5 As shown in Figure 2, the input image with shape H×W×3 is first obtained by a non-overlapping 4x4 convolution with a stride of 4 and then by a normalization layer. In order to construct a hierarchical feature pyramid, the backbone includes 4 stages of gradual increase. Between two consecutive stages, there is a non-overlapping 2x2 convolution with a stride of 2 to downsample the feature map, so that the spatial size is halved and the feature dimension is doubled. In the object detection task, the feature map output of the previous stage input is first normalized, and then the pooled feature map is used to predict the original value with a linear classifier.
[0122] DAT acts as a backbone network in the integrated visual model, extracting multi-scale features, adding a normalization layer to the features at each stage, and then feeding them into the next module. Continuous local attention blocks and sliding window attention blocks are introduced in the first and second stages of DAT, respectively, and continuous local attention blocks and deformable attention blocks are introduced in the third and fourth stages, respectively. The feature map is first processed by window-based local attention to locally aggregate information, and then the global relationship between local enhanced tags is modeled by the sliding window attention block. Since the first two stages mainly learn local features, the feature information of these early stages is incomplete. In addition, the keys and values of the first two stages have a considerable spatial size, which greatly increases the computational overhead of dot products and bilinear interpolation in deformable attention. Therefore, in order to achieve a trade-off between model capacity and computational burden, we only place deformable attention in the third and fourth stages and adopt the sliding window attention block in SwinTransformer for better performance in the early stages.
[0123] S3: Build an Efficient detection head. Compared with the original detection head, the Efficient detection head reduces computational complexity and memory usage by optimizing the network structure, reducing redundant connections, and using more efficient convolution methods, thereby improving the detection speed. It has stronger feature extraction capabilities and can more accurately extract features useful for detection tasks from input images. Through the method of compound scaling, it can flexibly adjust the width, depth, and resolution of the model according to existing computing resources and needs, achieving the best balance between detection performance and efficiency. Therefore, the Efficient detection head adopts a more efficient and simpler network structure, has better generalization capabilities, and can perform well on the dataset of surface defects of reflective films;
[0124] Small target detection has always been a challenge in the field of target detection, because small targets usually have lower resolution and less feature information, and are easily ignored or misclassified. In the original YOLOv8, due to the use of a larger downsampling ratio relative to the original image, there is a larger receptive field. Although some larger-scale targets can be predicted, smaller targets often cannot be fully noticed and accurately located. In view of the problems of large defect scale differences and complex background in online defect detection of reflective film, the present invention introduces a 160×160 scale small target detection layer added to the original network, including a supplementary fusion feature layer and an additional detection head, so as to enhance the semantic information and feature expression capabilities of small targets. First, the 80×80 scale feature layer in Backbone and the upsampled feature layer in Neck are stacked upward. After C2f and upsampling, a deep semantic feature layer containing small target feature information is obtained, which is further stacked with the shallow position feature layer in Backbone to supplement and improve the expression capabilities of the 160×160 scale fusion feature layer for small target semantic features and position information. Finally, it is sent to the Efficient detection head in Head through C2f. See Figure 6 .
[0125] S4: Design loss function, see Figure 7-Figure 8 ;
[0126] Yolov8 uses a composite loss strategy of classification loss and regression loss, in which the regression loss function uses Ciou. Compared with the ordinary Iou, which only meets the requirement of calculating the geometric similarity of two images, the Ciou loss function takes into account the shape and direction of the target box. The disadvantage is that the introduction of the calculation of the cross area and the correction factor will complicate the calculation and increase the amount of model training. Wise-Iou loss function constructs a non-monotonic focusing coefficient R WIoU , and the hyperparameters α and δ control the mapping of outlier degree β and gradient gain r. By selecting hyperparameters α = 1.6 and δ = 4, the optimal gradient gain allocation strategy is obtained, which can effectively improve the overall performance of the model.
[0127] IoU is used to measure the overlap between the predicted box and the real box in the target detection task and is defined as:
[0128]
[0129] Where: w, h are the width and height of the prediction box, w gt 、h gt is the width and height of the real box, W i H i is the intersection area between the predicted box and the true box, S u It is the union area of the predicted box and the true box.
[0130] In object detection tasks, the training dataset often inevitably contains low-quality anchor boxes, which may cause the model to suffer certain losses during optimization, especially when geometric metrics such as distance and aspect ratio are involved. Traditional loss functions often overemphasize the precise matching of these geometric metrics when calculating the difference between the anchor box and the target box (ground truth box), which may cause unnecessary losses on low-quality anchor boxes and thus damage the generalization ability of the model.
[0131] To solve this problem, this paper proposes a new loss function, Wise-IoU v1, which introduces the distance attention mechanism. The core idea is to dynamically adjust the weight of the geometric metric through distance attention when the anchor box and the target box overlap well, thereby weakening the impact on the model, that is, R WIoU ∈[1,e), which can focus on the bounding box loss of common quality anchor boxes.
[0132]
[0133] Where: x, y are the center coordinates of the prediction box, x gt ,y gt is the center coordinate of the real box, W g , H g is the minimum bounding rectangle of the predicted box and the real box, R WIoU is the non-monotonic focusing coefficient.
[0134] The dynamic non-monotonic focusing mechanism uses "outlier degree" instead of IoU to evaluate the quality of anchor frames. A small outlier degree β means that the anchor frame is of high quality and effective. The outlier degree is determined by the intersection-over-union ratio calculated for each batch and is a dynamic non-monotonic focusing value. This paper introduces a gradient gain, which is used to finely control the shape of the focusing coefficient curve through two key hyperparameters, so as to dynamically adjust the loss weights for difficult or poor-quality samples.
[0135] Experiments show that when α=1.6 and δ=4, the gradient gain r achieves the best effect.
[0136]
[0137] Where: α, δ are hyperparameter coefficients, β is the outlier degree, r is the gradient gain, L * IoU The intersection-over-union ratio calculated for the current batch, The average IoU value of the total batch.
[0138] S5: set parameters;
[0139] Parameter setting includes at least the following steps:
[0140] The collected reflective film images were randomly divided into training set, validation set, and test set in a ratio of 7:1:2 to prevent the model from overfitting;
[0141] The input image is 640x640 pixels, the model is trained for 400 cycles, the initial learning rate is 0.001, the batch size is 16, and the weight decay is 0.0005;
[0142] The optimizer uses stochastic gradient descent (SGD) and adopts a cosine annealing learning strategy to dynamically adjust the learning rate to prevent model training from falling into local optimality.
[0143] S6: Model comparison and detection effect, ablation experiment is carried out on the model to prove the effectiveness of the model.
[0144] Table 1 Ablation experiment
[0145]
[0146] mAP50 value for each defect type.
[0147] Table 2 Defect mAP50 values
[0148]
[0149] Its detection effect is as follows Fig. 9 shown.
[0150] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.
Claims
1. A reflective film defect detection system based on deep learning, characterized by: It includes a reflective film guiding structure, a detection control component and four sets of image acquisition modules; The reflective film guiding structure comprises two conveying wheels and a conveying belt, the conveying belt is arranged outside the conveying wheels, and the reflective film guiding structure is used to drive the reflective film to move; The detection control component includes an encoder, a PLC and a desktop host, and the desktop host is equipped with detection software; The image acquisition module includes an array CCD camera and a matching industrial lens; The recognition method of the reflective film defect detection system adopts the YOLOV8-DEW algorithm detection model.
2. A reflective film defect recognition method based on deep learning, used in the reflective film defect detection system based on deep learning as described in claim 1, characterized in that: At least the following steps are included: S1: preparing an image dataset, wherein the preparation of the image dataset at least includes image acquisition, data enhancement and dataset labeling; S2: A deformable attention mechanism is constructed. Instead of calculating the offset independently for each query, the entire feature map shares a set of uniformly shifted keys and values. This design is more efficient when processing large feature maps because all queries can perform self-attention operations using the same shifted key-value pairs, which significantly simplifies the calculation process. S3: Build an Efficient detection head. Compared with the original detection head, the Efficient detection head reduces computational complexity and memory usage by optimizing the network structure, reducing redundant connections, and using more efficient convolution methods, thereby improving the detection speed. It has stronger feature extraction capabilities and can more accurately extract features useful for detection tasks from input images. Through the method of compound scaling, it can flexibly adjust the width, depth, and resolution of the model according to existing computing resources and needs, achieving the best balance between detection performance and efficiency. Therefore, the Efficient detection head adopts a more efficient and simpler network structure, has better generalization capabilities, and can perform well on the dataset of surface defects of reflective films; S4: Design loss function; S5: set parameters; S6: Model comparison and detection effect, ablation experiment is carried out on the model to prove the effectiveness of the model.
3. The method for identifying reflective film defects based on deep learning according to claim 2, characterized in that: The image acquisition comprises at least the following steps: There are 4 sets of image acquisition modules. When the reflective film moves to the right driven by the reflective film guide structure, the displacement signal of the reflective film is sent to the PLC through the installed encoder; The PLC records the displacement signal output by the encoder. Whenever the reflective film moves over an integer multiple of 0.6 meters, the PLC sends a photo signal to the four image acquisition modules. The four array cameras record the reflective film surface image of the current area and send it to the desktop host. The size of the area captured by each image sent is 300mm (width) * 600mm (length). At the same time, the PLC also sends the current position signal to the desktop host.
4. The method for identifying reflective film defects based on deep learning according to claim 2, characterized in that: The data enhancement is used to increase the training sample data to achieve the purpose of expanding the data set, and the expanded data set is as close as possible to the actual distribution of the real data set, thereby improving the detection accuracy. In addition, data enhancement enables the model to learn more robust features, thereby improving the generalization ability of the model; The data enhancement includes geometric transformation and pixel transformation. The geometric transformation is to change the shape of the image, but not the image itself, but only to redistribute part of the image or the pixels of the image. The geometric transformation includes at least flipping, rotating, shifting, cropping, deforming and scaling; The pixel transformation changes the internal information of the image itself, and the pixel transformation at least includes color, noise, blur, erase and fill.
5. The method for identifying reflective film defects based on deep learning according to claim 2, characterized in that: The dataset annotation is to use Labelme to annotate the prepared image data, determine the target area position, and complete the generated related files through Labelme marking, which can detect target detection, semantic segmentation, and instance segmentation tasks respectively. When processing target detection tasks, the rectangular box annotation method is often used.
6. The method for identifying reflective film defects based on deep learning according to claim 1, characterized in that: The deformable attention mechanism is: For the input feature map x∈R H×W×C , generating a point The uniform grid composed of the two components is used as a reference; The grid size is downsampled from the input feature size by a factor r, The reference point values are linearly spaced 2D coordinates {(0,0),…,(H G -1,W G -1)}, and then according to the grid shape H G ×W G Normalize them to the range [-1, +1], where (-1, -1) represents the upper left corner and (+1, +1) represents the lower right corner; To obtain the offset of each reference point, the feature map is linearly mapped to the query mark, using Query = xW q Represented, and then introduced into a lightweight network θ offset Take the feature query as input and generate the offset Δp=θ corresponding to all reference points offset (q). In order to stabilize the training process, the amplitude of Δp is scaled by a predefined factor s to prevent the offset from being too large, i.e., s→Δp,tanh(Δp); represents the keys and values embedded after deformation, θ offset (q) Lightweight network, φ(·;·) sampling function. First, the position of the input feature map is sampled, then the moving points are bilinearly interpolated, and then sampled again as the key and value, and then the projection matrix is; where g(a,b) = max(0,1-|ab|) and (r x ,r y ) index z∈R H×W×C All positions on; Since g is only close to (p x ,p y ) is non-zero at the four integral points, so the above formula is simplified to the weighted average of the four positions; Combined with the existing method, multi-head attention is performed on q, k, v, and the relative position offset R is used. The output formula of the attention head is: in Represents the location embedding of the sampling points.
7. The method for identifying reflective film defects based on deep learning according to claim 2, characterized in that: The network architecture of the deformable attention mechanism adopts Deformable Attention Transformer, namely DAT, which can be widely used in visual tasks that require multi-scale feature maps.
8. The method for identifying reflective film defects based on deep learning according to claim 2, characterized in that: The application of the Efficient detection head comprises at least the following steps: First, the 80×80 scale feature layer in Backbone and the upsampling feature layer in Neck are stacked upwards; After C2f and upsampling, a deep semantic feature layer containing small target feature information is obtained, which is stacked with the shallow position feature layer in Backbone. The 160×160 scale fusion feature layer is supplemented and improved to express the semantic features and location information of small targets, and finally sent to the Efficient detection head in the Head through C2f.
9. The method for identifying reflective film defects based on deep learning according to claim 2, characterized in that: The loss function design at least comprises the following steps: The loss function is set to Wise-IoU v1, which introduces a distance attention mechanism. When the anchor box and the target box overlap well, the weight of the geometric metric is dynamically adjusted through distance attention, thereby weakening the impact on the model, that is, R WIoU ∈[1,e), which can focus on the bounding box loss of common quality anchor boxes, see the following formula: Where: x, y are the center coordinates of the prediction box, x gt ,y gt is the center coordinate of the real box, W g , H g is the minimum bounding rectangle of the predicted box and the real box, R WIoU is the non-monotonic focusing coefficient; The dynamic non-monotonic focusing mechanism uses "outlier degree" instead of IoU to evaluate the quality of the anchor frame. A small outlier degree β means that the anchor frame is of high quality and effective. The outlier degree is determined by the intersection-over-union ratio calculated for each batch and is a dynamic non-monotonic focusing value. A gradient gain is introduced, and two key hyperparameters are used to finely control the shape of the focusing coefficient curve, so as to dynamically adjust the loss weights for difficult or poor-quality samples.
10. The method for identifying reflective film defects based on deep learning according to claim 2, characterized in that: The parameter setting at least comprises the following steps: The collected reflective film images were randomly divided into training set, validation set, and test set in a ratio of 7:1:2 to prevent the model from overfitting; The input image is 640x640 pixels, the model is trained for 400 cycles, the initial learning rate is 0.001, the batch size is 16, and the weight decay is 0.0005; The optimizer uses the stochastic gradient descent method and adopts the cosine annealing learning strategy to dynamically adjust the learning rate to prevent the model training from falling into the local optimum.
Citation Information
Cited By
Parking lot visual target detection system and method based on YOLOV8
CN119672318A
