System and method for identifying surface defects in straw production based on machine vision

Abstract

The invention relates to the technical field of image analysis, in particular to a surface defect recognition system and method in straw production based on machine vision, and the method comprises the steps that an image collection module obtains an original RGB image of a straw; the preprocessing and feature enhancement unit processes the original image, including extracting a suction tube center line, performing attitude alignment and illumination artifact normalization based on the center line to generate a normalized image, and calculating a symmetry residual component of the normalized image and a template difference component based on a preset standard suction tube template to obtain a suction tube feature; and fusing to generate a defect enhanced residual heat map. The defect segmentation and classification unit generates an image defect mask by using a segmentation neural network containing a coordinate attention mechanism, and performs feature extraction and classification on a mask region to determine a defect type. And finally, the real-time output unit outputs the position and confidence information of the defect. The invention aims to realize accurate and automatic identification of the surface defects of the straw through multi-stage image analysis and processing.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to a system and method for identifying surface defects in straw production based on machine vision. Background Art

[0002] During the straw production process, ensuring surface quality is crucial to meeting hygiene and aesthetic standards. Surface defects such as scratches, bubbles, or stains can affect product quality, necessitating a surface inspection system. Machine vision technology, particularly in the field of image analysis, has become a core technology for addressing these issues due to its non-contact, high-speed, and high-precision capabilities.

[0003] However, in the practice of machine vision inspection of straw surface defects, the primary key challenge is how to accurately segment tiny or low-contrast defect areas from the complex straw background, especially when the straw material itself has reflective or transparent properties. This problem is more prominent. Therefore, image segmentation technology is crucial for distinguishing defective areas from normal surface textures. Traditional segmentation methods, such as threshold segmentation or edge detection, have difficulty dealing with image noise introduced by reflective surfaces or lighting changes in production environments. In recent years, deep learning-based segmentation models, such as U-Net or Mask R-CNN, have learned complex spatial patterns and contextual information from large amounts of data, demonstrating the potential for high-precision defect boundary segmentation in the presence of reflective or transparent artifacts caused by materials. However, these models usually have high requirements for the diversity and annotation quality of training data, and their computational complexity may pose challenges to certain high-speed and low-cost production line applications.

[0004] Another key challenge is how to accurately identify and classify different types of defects (for example, long and thin scratches, dot-shaped bubbles, irregular stains, etc.) based on the visual features of the defects. The diversity of straw colors, textures, and shapes requires the system to have robust feature extraction and pattern recognition capabilities. Traditional feature extraction methods, such as histograms of oriented gradients or scale-invariant feature transforms, have difficulty capturing the characteristics of tiny defects under complex production conditions. Modern machine vision methods increasingly use deep learning models, such as residual networks or Vision Transformer, to automatically learn high-level semantic features from data to achieve accurate classification of defects. However, the construction of such a comprehensive annotated dataset is expensive, and the computational requirements of deep models still face certain engineering optimization pressures for real-time applications in some high-speed production lines that pursue extreme production efficiency and low-latency response.

[0005] To this end, a surface defect recognition system and method in straw production based on machine vision is proposed. Summary of the Invention

[0006] The present invention aims to provide a system and method for surface defect identification in straw production based on machine vision. The system acquires the original RGB image of the straw through an image acquisition module. A preprocessing and feature enhancement unit processes the original image, including extracting the straw centerline, performing posture alignment based on the centerline, and normalizing illumination artifacts to generate a normalized image. The system then calculates the symmetry residual component of the normalized image and the template difference component based on a preset standard straw template to fuse them to generate a defect-enhanced residual heat map. A defect segmentation and classification unit uses the normalized image and the defect-enhanced residual heat map as multi-channel inputs, generates an image defect mask using a segmentation neural network that includes a coordinate attention mechanism, and performs feature extraction and classification on the masked area to determine the defect type. Finally, a real-time output unit outputs the defect location and confidence information. The present invention aims to achieve accurate and automated identification of straw surface defects through multi-stage image analysis and processing.

[0007] To achieve the above object, the present invention provides the following technical solutions: A machine vision-based surface defect recognition system for straw production, comprising: Image acquisition module, used to obtain the original RGB image of the straw; a preprocessing and feature enhancement unit, configured to extract the centerline of the straw from the original RGB image, and perform pose alignment and illumination artifact normalization on the original RGB image based on the centerline to generate a normalized image; calculate the symmetric residual components of the normalized image based on the centerline, and generate a defect enhancement residual heatmap by combining the template difference components between a preset standard straw template and the normalized image; a defect segmentation and classification unit, configured to fuse the normalized image and the defect-enhanced residual heatmap into a multi-channel input image; input the multi-channel input image into a segmentation neural network including a coordinate attention mechanism to generate an image defect mask; extract defect regions from the normalized image based on the image defect mask, and perform feature extraction and classification on the defect regions to determine the straw defect type; The real-time output unit is used to output the position and confidence information corresponding to the straw defect type.

[0008] Furthermore, the process of extracting the center line includes: Grayscale processing is performed on the original RGB image to generate a straw grayscale image; Applying an edge detection algorithm to the straw grayscale image to generate a binary edge map representing the straw boundary; Performing a straight line fitting operation on the binary edge image to determine a straight line of the straw edge; Based on the straw edge straight line, a skeletonization algorithm is used to extract and generate the center line with a single pixel width.

[0009] Furthermore, performing posture alignment and illumination artifact normalization processing on the original RGB image includes: Based on the coordinate points of the center line, a principal component analysis method is used to determine the current dominant direction of the straw in the original RGB image; Calculating a rotation angle between the current dominant direction and a preset target direction; using the center point of the center line as the rotation center, applying the rotation angle to perform a rotation transformation on the original RGB image to generate a direction-aligned image; Calculating a translation vector based on the centerline position after the rotation transformation and a preset target position, and performing a translation transformation on the direction-aligned image to generate a posture-aligned image; Performing logarithmic intensity transformation processing on the pose-aligned image; and applying adaptive histogram equalization processing to the processed pose-aligned image to generate the normalized image.

[0010] Furthermore, the process of generating the defect enhancement residual heat map includes: Based on the center line, performing a pixel mirroring operation on the normalized image and calculating the absolute difference between corresponding pixels of the normalized image before and after the operation to obtain the symmetric residual component; extracting a real-time grayscale projection of the normalized image along the center line, and obtaining a template difference component by comparing the real-time grayscale projection with the preset standard straw template; Combining the symmetric residual component and the template difference component using a preset weighted fusion strategy to generate a fused residual image; Gaussian smoothing filtering is applied to the fused residual image to generate the defect enhancement residual heat map.

[0011] Furthermore, the process of generating the preset standard straw template includes: obtaining a defect-free straw sample image; extracting the center line of each of the defect-free straw sample images; extracting grayscale projections along the center line of the defect-free straw sample image to obtain grayscale projection features; and performing statistical averaging on the grayscale projection features to generate the preset standard straw template.

[0012] Furthermore, the structure of the segmentation neural network specifically includes: A convolutional layer unit, comprising a convolutional layer and an activation function, configured to extract a hierarchical feature map from the multi-channel input image; A downsampling unit, configured to reduce the spatial dimension of the hierarchical feature map to generate low-level features; An upsampling unit, comprising an upsampling layer, a convolution layer, and an activation function, configured to restore spatial resolution and generate a restored feature map in combination with the low-level features; A skip connection unit, configured to perform feature fusion on the hierarchical feature map and the restored feature map; The coordinate attention mechanism unit is integrated into the network layer of the convolutional layer or the upsampling unit to enhance the representation of the slender structural features of the straw.

[0013] A method for identifying surface defects in straw production based on machine vision, comprising: Get the original RGB image of the straw; Extracting the centerline of the straw from the original RGB image, and performing pose alignment and illumination artifact normalization on the original RGB image based on the centerline to generate a normalized image; calculating the symmetric residual component of the normalized image based on the centerline, and combining the template difference component between a preset standard straw template and the normalized image to generate a defect enhancement residual heatmap; The normalized image and the defect-enhanced residual heatmap are fused into a multi-channel input image; the multi-channel input image is input into a segmentation neural network including a coordinate attention mechanism to generate an image defect mask; based on the image defect mask, defect regions are extracted from the normalized image, and feature extraction and classification processing are performed on the defect regions to determine the straw defect type; Output the position and confidence information corresponding to the straw defect type.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. This method processes and analyzes the original RGB image of the straw, extracting the centerline of its main structure. Using this centerline as a reference, it automatically aligns the image to a standardized coordinate system through rotation and translation. This geometric preprocessing eliminates visual discrepancies caused by random position and angular offsets of the straw during transport and imaging, providing stable image data with consistent posture for the subsequent image analysis module. This lays the foundation for establishing a unified defect detection model and achieving highly reliable identification.

[0015] 2. To address image quality degradation caused by factors such as the reflective and transparent properties of straw materials and lighting variations, this invention further normalizes lighting artifacts after completing pose alignment. It also combines residual analysis based on straw symmetry with difference analysis based on a preset standard template to generate a defect-enhanced residual heatmap. These feature enhancement methods effectively suppress artifacts and noise interference, highlighting the visual characteristics of true defects, thereby improving the image signal-to-noise ratio and defect recognizability, enabling accurate and automated identification of straw surface defects.

[0016] 3. This method combines a normalized image with a defect-enhanced residual heatmap through multi-channel fusion and feeds this into a segmentation neural network incorporating a coordinate attention mechanism, achieving pixel-level localization of defective areas on the straw surface. This multi-channel input strategy leverages the texture information of the original image and the defect indicators from the residual heatmap. Combined with the coordinate attention mechanism's focus on slender structures, it effectively improves the segmentation accuracy of small, irregular, and low-contrast defects. This, combined with subsequent defect classification using a lightweight neural network, ensures accurate identification of surface defects on straws. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The present invention provides a structural schematic diagram of a surface defect recognition system in straw production based on machine vision; Figure 2 A schematic diagram of the process of generating a defect enhancement residual heat map according to the present invention; Figure 3 The present invention provides a flow chart of a method for identifying surface defects in straw production based on machine vision. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0019] See also Figures 1 to 3 The present invention provides a system and method for identifying surface defects in straw production based on machine vision. The technical solution is as follows: Example 1:

[0020] Although existing technologies have been committed to achieving a fully automated production process for straws from raw materials to packaging, there are still many challenges in real-time, accurate, and comprehensive detection of tiny defects on the surface of straws on high-speed automated production lines. In view of the problem that the degree of automation in the existing straw production process is insufficient and it is difficult to ensure the stability of product surface quality and detection efficiency, this embodiment will describe in detail a system and its working method that can effectively identify surface defects of straws, especially for complex materials such as transparent and reflective materials, and meet the real-time requirements of industrial production. Figure 1 As shown, a surface defect recognition system for straw production based on machine vision includes: refer to Figure 1 The image acquisition module is used to obtain the original RGB image of the straw.

[0021] The original RGB image refers to the straw color image directly captured by the image acquisition module without subsequent digital image processing and represented by the red, green and blue channels.

[0022] Specifically, in this embodiment, an industrial CMOS area array camera is fixedly mounted above or to the side of the straw production line, perpendicular to the direction of straw conveyance. Simultaneously, a ring-shaped LED high-brightness white light source is used to illuminate the straw surface in a coaxial or nearly coaxial manner, thereby reducing shadows and achieving the most uniform lighting effect possible. When the straw passes through the camera's field of view, an external trigger signal (such as a photoelectric sensor) is used to synchronize the camera for image acquisition. Furthermore, this module is tasked with capturing multiple batches of defect-free straw images during system initialization or model training / updating. These images are used to subsequently generate standard templates and construct material reflection models.

[0023] refer to Figure 1 The preprocessing and feature enhancement unit is configured to extract the centerline of the straw in the original RGB image, and based on the centerline, perform pose alignment and illumination artifact normalization on the original RGB image to generate a normalized image; calculate the symmetric residual component of the normalized image based on the centerline, and generate a defect enhancement residual heatmap by combining the template difference component between a preset standard straw template and the normalized image.

[0024] Furthermore, the process of extracting the center line includes: First, the original RGB image is grayscaled to generate a straw grayscale image; The grayscale processing includes: performing weighted averaging on the original RGB image, for example, , converted to a single-channel straw grayscale image. is the pixel brightness value of the converted grayscale image, is the pixel value of the red channel in the original RGB image, is the pixel value of the green channel in the original RGB image, is the pixel value of the blue channel in the original RGB image.

[0025] Then, an edge detection algorithm is applied to the straw grayscale image to generate a binary edge map representing the straw boundary.

[0026] In this example, the Canny edge detection algorithm is applied to a grayscale image of a straw. First, the image is smoothed using a Gaussian filter (for example, with a kernel size of 5×5) to remove noise. Next, the image's gradient magnitude and direction are calculated (for example, using the Sobel operator), and non-maximum suppression is performed to refine the edges. Finally, a double-thresholding method (for example, a low threshold of 50 and a high threshold of 150) and hysteresis concatenation are used to generate a binary edge map that represents the straw's clear boundary.

[0027] Next, a straight line fitting operation is performed on the binary edge image to determine the edge straight line of the straw; In this embodiment, the Hough transform algorithm is used to process the binary edge map. This algorithm performs cumulative voting in a parameter space (e.g., polar rho-theta space) and detects peaks to identify the primary edge lines that form the main outline of the straw. This yields two parallel or nearly parallel lines, representing the two edges of the straw.

[0028] Finally, based on the edge straight line of the straw, a skeletonization algorithm is used to extract and generate the center line with a single pixel width.

[0029] In this embodiment, based on the detected straw edge lines, the axial region of the straw (e.g., the area between the two edge lines) is first determined and converted into a binary mask. The Zhang-Suen skeletonization algorithm is then applied to the binary mask, iteratively removing boundary pixels until a connected skeleton with a single-pixel width is formed. This skeleton serves as the centerline. The centerline is stored as an N×2 floating-point array, representing the (x, y) coordinates of N points.

[0030] By performing grayscale conversion, edge detection, line fitting, and skeletonization on the original RGB image in sequence, the single-pixel width centerline of the straw can be accurately and stably extracted, providing a reliable geometric benchmark for subsequent key image processing steps such as posture alignment and symmetry analysis, thereby improving the accuracy and stability of straw surface defect detection.

[0031] Furthermore, performing posture alignment and illumination artifact normalization processing on the original RGB image includes: Based on the coordinate points of the center line, a principal component analysis method is used to determine the current dominant direction of the straw in the original RGB image. The principal component analysis method can calculate the eigenvector corresponding to the maximum eigenvalue of the point set covariance matrix. The direction of this eigenvector is the current dominant direction of the straw in the original RGB image.

[0032] Calculating a rotation angle between the current dominant direction and a preset target direction (e.g., the horizontal axis of the image); performing a rotation transformation on the original RGB image using the rotation angle with the center point of the center line as the rotation center, using bilinear interpolation during the transformation to generate a directionally aligned image; Calculating a translation vector based on the centerline position after the rotation transformation and a preset target position, and performing a translation transformation on the direction-aligned image to generate a posture-aligned image; Performing logarithmic intensity transformation processing on the pose-aligned image; and applying adaptive histogram equalization processing to the processed pose-aligned image to generate the normalized image.

[0033] Among them, the logarithmic intensity transformation process, for example, for pixel values ​​normalized to [0,1] , the processed pixel value , to compress the high dynamic range and suppress highlights, where is a logarithmic function. Next, the adaptive histogram equalization algorithm divides the processed pose-aligned image into multiple 8×8 pixel local regions (tiles). A histogram is calculated for each tile and equalized. A clipping limit (e.g., 2.0) is set to constrain the degree of contrast amplification. The processed tiles are smoothly concatenated using bilinear interpolation to generate the normalized image.

[0034] By combining principal component analysis to determine the dominant orientation of the straw, rotation and translation transformations are applied accordingly to achieve posture alignment. Logarithmic intensity transformation and adaptive histogram equalization are then implemented to effectively eliminate visual discrepancies introduced by random position and angle offsets during the transmission and imaging processes. This also improves image contrast and blurred details caused by uneven lighting and reflective materials, ultimately generating a standardized image with consistent visual effects and easier identification of defect features.

[0035] Furthermore, if Figure 2 As shown, the process of generating the defect enhancement residual heat map includes: Based on the center line, a pixel mirror operation is performed on the normalized image, where the operation uses the center line as an axis of symmetry to map pixel values ​​on one side of the image to symmetrical positions on the other side; and the absolute difference between corresponding pixels of the normalized image before and after the operation is calculated to obtain the symmetric residual component; Since the defective region disrupts the symmetrical structure of the straw, it will show a stronger response in the symmetrical residual component. If the normalized image is a multi-channel image, the calculated symmetrical residual component can be converted into a single-channel image, for example, by taking the average of each color channel.

[0036] Extracting a real-time grayscale projection of the normalized image along the center line, and obtaining a template difference component by comparing the real-time grayscale projection with the preset standard straw template and calculating the absolute difference between corresponding elements of the two; The symmetric residual component and the template difference component are combined using a preset weighted fusion strategy to generate a fused residual image; for example, the following formula can be used: fused residual image = a × symmetric residual component + b × template difference component, where a and b are preset weights, such as a = 0.7 and b = 0.3.

[0037] Gaussian smoothing filtering is applied to the fused residual image to effectively remove potential image noise and make the response of the defect area more continuous and smooth, thereby generating the defect enhanced residual heat map.

[0038] By separately calculating the symmetry residual component of the normalized image and the template difference component based on a preset standard straw template, and then weightedly fusing and smoothing these two physically distinct residual information, the resulting defect-enhanced residual heatmap can more comprehensively highlight various surface defects. In particular, it demonstrates higher sensitivity and detection rate for minor defects that disrupt symmetry or deviate from the standard morphology, thereby improving the accuracy and stability of straw surface defect detection.

[0039] Furthermore, the process of generating the preset standard straw template includes: obtaining a defect-free straw sample image; extracting the center line of each of the defect-free straw sample images; extracting grayscale projections along the center line of the defect-free straw sample image to obtain grayscale projection features; and performing statistical averaging on the grayscale projection features to generate the preset standard straw template.

[0040] Furthermore, the process of generating the preset standard straw template is usually performed during the system initialization or model update phase, and specifically includes the following steps: First, collect a set of representative images of defect-free straw samples, for example, at least 100 images, covering different production batches and major straw materials (such as plastic and paper).

[0041] Then, for each of the captured defect-free straw sample images, the same method as described in the preprocessing and feature enhancement unit is applied to extract the center line thereof.

[0042] Next, for each defect-free straw sample image with its centerline extracted, a stripe of a predetermined width (e.g., 100 pixels) is extracted perpendicular to the centerline. The average grayscale value of the pixels within this stripe is calculated, forming a one-dimensional grayscale projection feature vector representing the average grayscale distribution at each point along the straw's centerline. To ensure data consistency, the values ​​of each extracted grayscale projection feature vector are normalized to the range [0, 1].

[0043] Finally, the grayscale projection features of all defect-free straw sample images, after normalization, are statistically averaged. For example, the average value of the corresponding elements of these one-dimensional grayscale projection feature vectors is calculated to form the preset standard straw template. This template represents the average grayscale distribution characteristics of an ideal defect-free straw.

[0044] By extracting the center lines of multiple defect-free straw sample images and calculating their average grayscale projection along the center line, a preset standard straw template is generated. This template can objectively and stably characterize the average appearance characteristics of an ideal defect-free straw, providing a reliable benchmark for subsequent identification of deviations from the ideal state through template difference analysis, thereby enhancing the targetedness and accuracy of straw production defect detection.

[0045] refer to Figure 1 The defect segmentation and classification unit is configured to fuse the normalized image (RGB three-channel) with the defect enhancement residual heatmap (single-channel) into a multi-channel input image. The resulting multi-channel input image is a four-channel input image. The multi-channel input image is input into a segmentation neural network including a coordinate attention mechanism to generate an image defect mask.

[0046] Furthermore, the structure of the segmentation neural network adopts an encoder-decoder architecture, specifically including: The encoder is composed of a plurality of convolutional layer units and downsampling units connected in sequence.

[0047] A convolutional layer unit (encoder part), wherein the convolutional layer unit includes at least one convolutional layer and an activation function, and is used to extract a hierarchical feature map from the multi-channel input image; The downsampling unit (encoder part) is usually used in conjunction with the convolutional layer unit, using operations such as strided convolution or maximum pooling to reduce the spatial dimension of the hierarchical feature map and generate low-level features containing more abstract semantic information; The decoder is composed of a plurality of upsampling units connected in sequence.

[0048] An upsampling unit (decoder part), which includes an upsampling layer (such as transposed convolution or bilinear interpolation upsampling), at least one convolutional layer and an activation function, which is used to gradually restore the spatial resolution of the feature map, and generate the restored feature map from the low-level features of the corresponding level of the encoder; The skip connection unit uses multiple skip connection structures to fuse the shallow, high-resolution hierarchical feature maps extracted by the encoder (for example, the feature maps before pooling) with the restored feature maps of the corresponding layers of the decoder by splicing or adding them together, so as to combine low-level detail information and high-level semantic information, thereby helping to achieve more accurate defect boundary positioning.

[0049] A coordinate attention mechanism unit is integrated into the encoder or decoder network layer (for example, after the convolutional layer or upsampling unit). By encoding position information into channel attention, this unit enables the neural network to capture a wide range of contextual dependencies with precise position information, thereby enhancing the ability to represent slender structural features such as straws and enhancing sensitivity to small defects.

[0050] The segmentation neural network, based on the U-Net model, was trained on a dataset of at least 200 straw images with manually annotated defect area masks. A combination of Dice loss and binary cross-entropy loss was used as the loss function, and the Adam optimizer was used to optimize model parameters. To improve the model's generalization capabilities, data augmentation techniques such as random rotation (e.g., ±15 degrees), horizontal / vertical flipping, and brightness / contrast perturbations (e.g., ±20%) were used during training. The segmentation neural network ultimately outputs a single-channel, binary image defect mask (pixel values ​​of 0 or 1) with the same size as the input image. Regions with a pixel value of 1 represent detected defects, and regions with a pixel value of 0 represent background.

[0051] To verify the effectiveness of the segmentation neural network in improving defect segmentation performance on slender straw structures, we compared it with a baseline model: a U-Net variant that lacks the coordinate attention mechanism. As shown in Table 1, the proposed segmentation neural network demonstrated improvements across all key segmentation metrics.

[0052] Table 1 Comparison of segmentation neural network performance

[0053] A segmentation neural network employs an encoder-decoder architecture with specialized units and integrates coordinate attention mechanisms to effectively process multi-channel fused input images. This network leverages the image's structural, texture, and defect-indicating information, while enhancing the representation of the straw's slender structure. This enables high-precision pixel-level segmentation of defective areas on the straw's surface, thereby enhancing the targetedness and accuracy of defect detection in straw production.

[0054] Furthermore, a defective area is extracted from the normalized image according to the image defect mask, and feature extraction and classification processing are performed on the defective area to determine the defect type of the straw.

[0055] Specifically, based on the image defect mask, a connected component analysis algorithm or other region localization method is used to identify and demarcate each independent defect region. Subsequently, corresponding defect region tiles are cropped from the normalized image based on the location of each demarcated defect region. For example, the minimum bounding rectangle of each connected defect region can be used as the basis for tile extraction, and the extracted tiles can be optionally scaled to a preset uniform size, such as 64×64 pixels, to facilitate subsequent network processing.

[0056] Then, each extracted defect area tile is processed using a pre-trained lightweight convolutional neural network (CNN). In this embodiment, the lightweight CNN can use, for example, the MobileNetV3-Small model, which has been pre-trained on large image datasets such as ImageNet and has been specifically fine-tuned and optimized for common defect types of straws (for example, scratches, bubbles, and stains). The lightweight CNN performs deep feature extraction on the input defect area tile. The end of the network is usually connected to a fully connected layer, and finally outputs the probability distribution corresponding to each preset defect type through the Softmax activation function. Based on the probability distribution of each defect type output by the lightweight CNN, the category corresponding to the highest probability value is selected as the final determined straw defect type.

[0057] refer to Figure 1 A real-time output unit is used to output the position and confidence information corresponding to the straw defect type.

[0058] Specifically, the unit receives the image defect mask generated by the defect segmentation and classification unit, along with the determined defect type and classification confidence. Based on the image defect mask, the unit calculates the precise location of the defect in the original image (for example, by extracting its minimum bounding rectangle (LR) through contour extraction or connected component analysis of the defect mask, thereby determining the bounding box coordinates [x, y, w, h]) and size information. The unit then formats this information (including the unique defect ID, the identified defect type string, the confidence level expressed as a floating-point number in the [0, 1] range, and the calculated bounding box coordinates), for example, into a JSON object format, to achieve structured, real-time output of defect information.

[0059] First, by extracting the straw centerline with high precision and using it as a reference for pose alignment, combined with illumination artifact normalization methods such as logarithmic intensity transformation and adaptive histogram equalization, this method effectively overcomes image quality instability caused by varying straw position and angle, as well as reflective and transparent material properties. This results in a normalized image with consistent pose and clear visual features. Secondly, by combining residual analysis based on centerline symmetry with difference analysis based on a preset standard straw template, a defect-enhanced residual heatmap is generated, highlighting subtle and low-contrast defects that are difficult to capture using conventional methods. Furthermore, the normalized image and the defect-enhanced residual heatmap are fed as multi-channel inputs to a segmentation neural network with an encoder-decoder architecture that integrates a coordinate attention mechanism. This improves the pixel-level segmentation and localization accuracy of various defects (such as scratches, bubbles, and stains) on slender straws, and utilizes a lightweight neural network for efficient classification. Finally, through model optimization and edge deployment, high-speed real-time detection and information output are achieved on the production line, improving the reliability and efficiency of straw production quality control.

[0060] Example 2:

[0061] In order to further verify the practical application effect and adaptability of the present invention, this embodiment is illustrated by taking a representative polypropylene (PP) transparent straw automated production line of a straw company as an example. During the production process, the company has an urgent need to improve the surface quality of PP transparent straws, especially to improve the efficiency and accuracy of detecting small defects such as bubbles and scratches. This embodiment will combine the production characteristics of the company to elaborate on the technical effects that the present invention is expected to achieve in an actual production line. Figure 3 As shown, a method for identifying surface defects in straw production based on machine vision includes: Get the original RGB image of the straw; Extracting the centerline of the straw from the original RGB image, and performing pose alignment and illumination artifact normalization on the original RGB image based on the centerline to generate a normalized image; calculating the symmetric residual component of the normalized image based on the centerline, and combining the template difference component between a preset standard straw template and the normalized image to generate a defect enhancement residual heatmap; The normalized image and the defect-enhanced residual heatmap are fused into a multi-channel input image; the multi-channel input image is input into a segmentation neural network including a coordinate attention mechanism to generate an image defect mask; based on the image defect mask, defect regions are extracted from the normalized image, and feature extraction and classification processing are performed on the defect regions to determine the straw defect type; Output the position and confidence information corresponding to the straw defect type.

[0062] Specifically, an automated production line from a straw company was selected. This line includes extrusion, water cooling, pulling and cutting, and conveying. In this embodiment, the system is deployed on a conveyor belt after cutting and before packaging. Straws pass through the inspection area one by one on a stable conveyor belt. The main surface defects of interest in this embodiment include internal bubbles (of varying size and distribution), surface scratches (of varying length, depth, and direction), black spots, and minor localized deformations.

[0063] In order to evaluate the performance of the system of the present invention, the following comparison method is set up: Comparison Method 1 (Traditional Machine Vision Baseline Method): Grayscale the original RGB image; smooth it using Gaussian filtering; perform binary segmentation using an adaptive threshold method (Otsu method) to initially separate the foreground and background; perform morphological operations on the binary image, including opening operations to remove small noise points and closing operations to connect broken areas; finally, extract suspected defect areas through connected component analysis, and perform rule judgment on these areas based on preset area and average grayscale to distinguish defects from non-defects.

[0064] Comparative Method 2 (No Residual Fusion): To verify the effectiveness of the defect-enhanced residual heatmap and its multi-channel fusion strategy in this method, this method does not generate a defect-enhanced residual heatmap. Its defect segmentation and classification unit uses only the normalized image, after pose alignment and illumination artifact normalization, as the input to the segmentation neural network. All other modules remain consistent with the complete system of this invention.

[0065] Using the same dataset construction method and evaluation metrics, we evaluated the performance of our method, Comparative Method 1, and Comparative Method 2 on a test set collected from a straw company's PP transparent straw production line. The evaluation results are shown in Tables 2 and 3, where "N / A" indicates no data.

[0066] According to the results shown in Table 2 and Table 3, it can be clearly seen that the present invention has demonstrated better comprehensive performance in detecting various surface defects of PP transparent straws. Compared with comparative method one, the average defect detection rate and average defect precision of the present invention are both higher, mainly due to the deep learning segmentation and classification model adopted by the present invention, as well as the unique image preprocessing and feature enhancement strategy, especially the introduction of the defect enhancement residual heat map, which improves the recognition ability of tiny and low-contrast defects under complex backgrounds. Compared with comparative method two, although the average processing speed of the present invention has increased slightly, it has achieved improvements in both the average defect detection rate and the average defect precision, and the false alarm rate and the missed alarm rate have also been effectively controlled. This fully demonstrates that the defect enhancement residual heat map proposed in the present invention and its multi-channel fusion input strategy with the normalized image have played a key role in improving the accurate perception and positioning of defects by the segmentation network.

[0067] In contrast, while comparative method 1 achieved acceptable processing speed, it performed poorly in key metrics such as defect detection rate, accuracy, and false positive and false negative rates, making it difficult to detect the diverse and complex surface defects of transparent straws. Therefore, it can be concluded that this invention, by optimizing the image analysis process and applying deep learning models, improves the accuracy and robustness of straw surface defect detection while ensuring real-time industrial production, providing more reliable technical support for quality control in industrial production.

[0068] Table 2 Defect recognition and classification performance of the proposed system on PP transparent straws

[0069] Table 3 Comprehensive performance comparison

[0070] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A surface defect recognition system for straw production based on machine vision, characterized in that: include: Image acquisition module, used to obtain the original RGB image of the straw; a preprocessing and feature enhancement unit, configured to extract a centerline of the straw from the original RGB image, and perform posture alignment and illumination artifact normalization processing on the original RGB image based on the centerline to generate a normalized image; Calculating a symmetric residual component of the normalized image based on the center line, and combining a template difference component between a preset standard straw template and the normalized image to generate a defect enhancement residual heat map; a defect segmentation and classification unit, configured to fuse the normalized image and the defect enhanced residual heatmap into a multi-channel input image; and input the multi-channel input image into a segmentation neural network including a coordinate attention mechanism to generate an image defect mask; extracting a defective area from the normalized image according to the image defect mask, and performing feature extraction and classification processing on the defective area to determine the defect type of the straw; The real-time output unit is used to output the position and confidence information corresponding to the straw defect type.

2. The surface defect recognition system for straw production based on machine vision according to claim 1 is characterized in that: The process of extracting the center line includes: Grayscale processing is performed on the original RGB image to generate a straw grayscale image; Applying an edge detection algorithm to the straw grayscale image to generate a binary edge map representing the straw boundary; Performing a straight line fitting operation on the binary edge image to determine a straight line of the straw edge; Based on the straw edge straight line, a skeletonization algorithm is used to extract and generate the center line with a single pixel width.

3. The surface defect recognition system for straw production based on machine vision according to claim 1 is characterized in that: Performing posture alignment and illumination artifact normalization processing on the original RGB image includes: Based on the coordinate points of the center line, a principal component analysis method is used to determine the current dominant direction of the straw in the original RGB image; Calculating a rotation angle between the current dominant direction and a preset target direction; using the center point of the center line as the rotation center, applying the rotation angle to perform a rotation transformation on the original RGB image to generate a direction-aligned image; Calculating a translation vector based on the centerline position after the rotation transformation and a preset target position, and performing a translation transformation on the direction-aligned image to generate a posture-aligned image; Performing logarithmic intensity transformation processing on the pose-aligned image; and applying adaptive histogram equalization processing to the processed pose-aligned image to generate the normalized image.

4. The surface defect recognition system for straw production based on machine vision according to claim 1 is characterized in that: The process of generating the defect enhancement residual heat map includes: Based on the center line, performing a pixel mirroring operation on the normalized image and calculating the absolute difference between corresponding pixels of the normalized image before and after the operation to obtain the symmetric residual component; extracting a real-time grayscale projection of the normalized image along the center line, and obtaining a template difference component by comparing the real-time grayscale projection with the preset standard straw template; Combining the symmetric residual component and the template difference component using a preset weighted fusion strategy to generate a fused residual image; Gaussian smoothing filtering is applied to the fused residual image to generate the defect enhancement residual heat map.

5. The surface defect recognition system for straw production based on machine vision according to claim 1 is characterized in that: The process of generating a preset standard straw template includes: obtaining a sample image of a defect-free straw; extracting the center line of each of the sample images; extracting a grayscale projection along the center line of the sample image to obtain a grayscale projection feature; and performing statistical averaging on the grayscale projection features to generate the preset standard straw template.

6. The surface defect recognition system for straw production based on machine vision according to claim 1 is characterized in that: The structure of the segmentation neural network specifically includes: A convolutional layer unit, comprising a convolutional layer and an activation function, configured to extract a hierarchical feature map from the multi-channel input image; A downsampling unit, configured to reduce the spatial dimension of the hierarchical feature map to generate low-level features; An upsampling unit, comprising an upsampling layer, a convolution layer, and an activation function, configured to restore spatial resolution and generate a restored feature map in combination with the low-level features; A skip connection unit, configured to perform feature fusion on the hierarchical feature map and the restored feature map; The coordinate attention mechanism unit is integrated into the network layer of the convolutional layer or the upsampling unit to enhance the representation of the slender structural features of the straw.

7. A method for identifying surface defects in straw production based on machine vision, characterized in that: include: Get the original RGB image of the straw; Extracting a centerline of the straw in the original RGB image, and performing posture alignment and illumination artifact normalization processing on the original RGB image based on the centerline to generate a normalized image; Calculating a symmetric residual component of the normalized image based on the center line, and combining a template difference component between a preset standard straw template and the normalized image to generate a defect enhancement residual heat map; fusing the normalized image with the defect enhancement residual heatmap into a multi-channel input image; inputting the multi-channel input image into a segmentation neural network including a coordinate attention mechanism to generate an image defect mask; extracting a defective area from the normalized image according to the image defect mask, and performing feature extraction and classification processing on the defective area to determine the defect type of the straw; Output the position and confidence information corresponding to the straw defect type.

Images