Cotton swab appearance defect detection method and system based on deep learning

Abstract

The invention relates to the technical field of machine vision, and provides a cotton swab appearance defect detection method and system based on deep learning, and the method comprises the steps: calculating a texture anomaly index of a to-be-detected cotton swab through a multi-dimensional visual image of the to-be-detected cotton swab; according to the texture anomaly index, calculating a defect saliency score and a structure anomaly degree of the defect candidate area so as to generate a defect enhancement feature map of the multi-dimensional visual image; and extracting defect discrimination features of the to-be-detected cotton swab from the defect enhancement feature map so as to execute defect detection of the to-be-detected cotton swab. The method can effectively distinguish normal cotton fiber textures and subtle appearance defects on the surfaces of the cotton swabs, and the accuracy of cotton swab appearance defect detection is improved.

Description

A Deep Learning-Based Method and System for Detecting Cotton Swab Appearance Defects Technical Field

[0001] This invention relates to a method and system for detecting cosmetic defects in cotton swabs based on deep learning, belonging to the field of machine vision technology. Background Technology

[0002] As a product that comes into direct contact with the human body and has a wide range of uses, the appearance quality of cotton swabs directly affects the user's safety and experience. Therefore, achieving efficient and accurate detection of cotton swab appearance defects has become a core technical aspect of ensuring the quality of finished cotton swabs and meeting high market standards.

[0003] Traditional methods for detecting cosmetic defects in cotton swabs often rely on visual inspection or image processing based on fixed rules, such as setting brightness thresholds or performing contour template matching. These methods struggle to adapt to the natural variations in the surface texture of cotton swabs and cannot effectively distinguish between normal cotton fibers and subtle cosmetic defects, thus affecting the accuracy of cosmetic defect detection. Summary of the Invention

[0004] This invention provides a method and system for detecting cotton swab appearance defects based on deep learning. Its main purpose is to effectively distinguish between normal cotton texture and minor appearance defects on the surface of cotton swabs, thereby improving the accuracy of cotton swab appearance defect detection.

[0005] To achieve the above objectives, the present invention provides a deep learning-based method for detecting cosmetic defects in cotton swabs, comprising: acquiring a multidimensional visual image of the cotton swab to be detected; calculating a texture anomaly index of the cotton swab based on the multidimensional visual image; locating a defect candidate region of the cotton swab based on the texture anomaly index, and calculating a defect saliency score and a structural anomaly degree of the defect candidate region; generating a defect enhancement feature map of the multidimensional visual image based on the defect saliency score and the structural anomaly degree, and extracting defect discrimination features of the cotton swab from the defect enhancement feature map; and performing defect detection on the cotton swab based on the defect discrimination features to obtain a defect detection result, wherein the defect detection result includes at least a defect category, a defect location, and a defect severity level.

[0006] Optionally, extracting defect discrimination features of the swab to be tested from the defect enhancement feature map includes: determining at least one defect candidate region of the swab to be tested based on the defect enhancement feature map; extracting local depth feature blocks corresponding to the defect candidate regions from the defect enhancement feature map; inputting the local depth feature blocks into a preset feature extraction network, the feature extraction network including a spatial pyramid pooling layer and a fully connected layer; performing multi-scale feature fusion and dimensionality reduction processing on the local depth features through the feature extraction network to obtain a fixed feature vector; and using the fixed feature vector as the defect discrimination feature of the swab to be tested.

[0007] Optionally, the local deep features are fused and dimensionality reduced using the feature extraction network to obtain a fixed feature vector. This includes: performing multi-scale partitioning and pooling operations on the local deep feature blocks using the spatial pyramid pooling layer of the feature extraction network to generate a pooled feature vector; and performing feature dimensionality reduction and high-order nonlinear transformation processing on the pooled feature vector using the fully connected layer of the feature extraction network to obtain the fixed feature vector.

[0008] Optionally, based on the defect discrimination features, defect detection of the swab to be tested is performed to obtain defect detection results, including: calling a preset discrimination model, and calculating the defect category probability and position offset of the swab to be tested based on the defect discrimination features and the discrimination model; and generating the defect detection results of the swab to be tested based on the defect category probability and the position offset.

[0009] Optionally, based on the defect discrimination features and the discrimination model, the defect category probability and position offset of the cotton swab to be detected are calculated, including: inputting the defect discrimination features into the classification branch and the regression branch of the discrimination model; outputting the defect category probability of the cotton swab to be detected from the classification branch; outputting the coordinate offset parameter of the defect candidate region of the cotton swab to be detected from the regression branch, and using the coordinate offset parameter as the position offset of the cotton swab to be detected.

[0010] Optionally, the defect category probability of the cotton swab to be tested is output by the classification branch, including: inputting the defect discrimination features into the classification fully connected layer in the classification branch; identifying the high-dimensional classification features in the defect discrimination features through the classification fully connected layer; inputting the high-dimensional classification features into the Softmax function in the classification branch, and calculating the normalized probability of the cotton swab to be tested belonging to each preset defect category through the Softmax function; and determining the defect category probability of the cotton swab to be tested based on the normalized probability.

[0011] Optionally, the coordinate offset parameters of the candidate defect region of the swab to be detected are output by the regression branch, including: inputting the defect discrimination features into the regression fully connected layer in the regression branch; performing linear transformation and feature encoding on the defect discrimination features through the regression fully connected layer to obtain a regression feature vector; mapping the regression feature vector to the geometric adjustment offset of the candidate defect region, and determining the geometric adjustment offset as the coordinate offset parameter of the candidate defect region.

[0012] Optionally, generating a defect enhancement feature map of the multidimensional visual image based on the defect saliency score and the structural anomaly degree includes: mapping the defect saliency score and the structural anomaly degree to a first feature enhancement weight map and a second feature enhancement weight map corresponding to the space of the multidimensional visual image, respectively; performing weighted fusion of the first feature enhancement weight map and the second feature enhancement weight map according to a preset fusion coefficient to generate a spatial attention weight map; extracting the original convolutional feature map from the multidimensional visual image, and multiplying the spatial attention weight map element-wise with the original convolutional feature map to obtain the defect enhancement feature map.

[0013] Optionally, calculating the structural anomaly degree of the defect candidate region based on the texture anomaly index includes: calculating the morphological distortion variable of the defect candidate region; obtaining the contour stability parameter of the defect candidate region in a multi-frame temporal image; dynamically correcting the morphological distortion variable based on the contour stability parameter to obtain a corrected morphological distortion variable; and calculating the structural anomaly degree of the defect candidate region based on the texture anomaly index and the corrected morphological distortion variable.

[0014] To address the aforementioned problems, this invention also provides a deep learning-based cotton swab appearance defect detection system. The system includes: a texture analysis module for acquiring a multi-dimensional visual image of the cotton swab to be detected, and calculating a texture anomaly index of the cotton swab based on the multi-dimensional visual image; a defect localization module for locating candidate defect regions of the cotton swab to be detected based on the texture anomaly index, and calculating a defect saliency score and structural anomaly degree of the candidate defect regions; a feature extraction module for generating a defect enhancement feature map of the multi-dimensional visual image based on the defect saliency score and the structural anomaly degree, and extracting defect discrimination features of the cotton swab to be detected from the defect enhancement feature map; and a defect result output module for performing defect detection on the cotton swab to be detected based on the defect discrimination features, and obtaining a defect detection result, wherein the defect detection result includes at least the defect category, defect location, and defect severity level.

[0015] Compared to the problems described in the background art, the embodiments of the present invention calculate the texture anomaly index of the cotton swab to be detected based on the multidimensional visual image, which can transform the original image data into a quantitative index that can initially and adaptively distinguish normal cotton texture from various potential defects. Furthermore, the embodiments of the present invention calculate the defect saliency score and structural anomaly degree of the defect candidate region based on the texture anomaly index, which can perform dual credibility verification and preliminary defect nature judgment on the initially located suspected areas, thereby achieving efficient filtering of a large number of simple false alarm areas before subsequent defect result determination. Based on the defect saliency score and the structural anomaly degree, the embodiments of the present invention generate a defect enhancement feature map of the multidimensional visual image, which can identify seemingly blurry and subtle abnormal areas in the cotton swab image through... The scoring and anomaly analysis are visually enhanced and highlighted, enabling subsequent algorithms to more accurately and stably identify and judge defects. Furthermore, this invention extracts defect discrimination features from the enhanced defect feature map of the cotton swab to be tested. This transforms the high-dimensional features that significantly enhance defect area information into a set of more discriminative feature vectors, directly and efficiently serving subsequent defect classification, localization, and severity assessment. Finally, based on the defect discrimination features, this invention performs defect detection on the cotton swab to obtain the defect detection results. This allows the system to automatically and accurately identify true defects from seemingly chaotic cotton textures, fundamentally replacing manual visual inspection and rigid rule-based judgment, significantly improving the accuracy and stability of cotton swab appearance defect detection. Therefore, this invention can effectively distinguish between normal cotton textures and minor appearance defects on the surface of cotton swabs, improving the accuracy of cotton swab appearance defect detection. Attached Figure Description

[0016] Figure 1 is a flowchart illustrating a deep learning-based method for detecting cotton swab appearance defects according to an embodiment of the present invention; Figure 2 is a top view of a cotton swab implementing the deep learning-based method for detecting cotton swab appearance defects according to an embodiment of the present invention; Figure 3 is a side view of a cotton swab implementing the deep learning-based method for detecting cotton swab appearance defects according to an embodiment of the present invention; Figure 4 is a module diagram illustrating the deep learning-based method for detecting cotton swab appearance defects according to an embodiment of the present invention; Figure 5 is a schematic diagram of a computer device for implementing the deep learning-based method for detecting cotton swab appearance defects according to an embodiment of the present invention; The objectives, features, and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0017] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0018] This application provides a deep learning-based method for detecting cosmetic defects in cotton swabs. The execution entity of this deep learning-based method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the deep learning-based method for detecting cosmetic defects in cotton swabs can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0019] Referring to Figure 1, a flowchart illustrating a deep learning-based method for detecting cotton swab appearance defects is provided in an embodiment of the present invention. In this embodiment, the deep learning-based method for detecting cotton swab appearance defects includes: S1, acquiring a multi-dimensional visual image of the cotton swab to be detected, and calculating the texture anomaly index of the cotton swab to be detected based on the multi-dimensional visual image.

[0020] This invention provides a complete and high-contrast raw data foundation for subsequent defect discrimination feature extraction by acquiring multi-dimensional visual images of the cotton swab to be inspected. The cotton swab to be inspected refers to the physical cotton swab that needs to be checked for appearance defects. The multi-dimensional visual image refers to a set of image data collected from the same cotton swab under multiple different angles and lighting conditions.

[0021] In a preferred embodiment of the present invention, the hardware system for acquiring the multidimensional visual image includes: a horizontal conveyor belt, a PLC-controlled triggering and positioning mechanism, two 2D cameras located directly above the cotton swab, and a right-angle prism mounted on the side of the end of the cotton swab in the direction of travel. When the cotton swab to be tested moves to the imaging station with the conveyor belt, the PLC synchronously triggers the two cameras. The first camera directly acquires a top-down view of the cotton swab, while simultaneously, light from the side surface of the cotton swab is reflected by the right-angle prism and enters the second camera, which can simultaneously acquire a side view of the cotton swab. The two images obtained are strictly corresponding in time and space, and together constitute the multidimensional visual image.

[0022] Referring to Figure 2, which is a top view of a cotton swab as an embodiment of the deep learning-based cotton swab appearance defect detection method provided by the present invention, this image is acquired by a 2D camera fixed directly above the horizontal conveyor belt. The resolution is 2 million pixels and the frame rate is set to 30fps. It is one of the core components of the multi-dimensional visual image of the present invention. During the acquisition process, when the cotton swab to be detected moves to the imaging station with the conveyor belt, the PLC triggers the camera to capture images synchronously. The image data is transmitted to the industrial computer in real time as the raw input for subsequent defect detection.

[0023] It should be noted that, to ensure the complete acquisition of information from the side of the cotton swab, this solution introduces a right-angle prism structure. Referring to Figure 3, which shows a side view of the cotton swab according to an embodiment of the deep learning-based cotton swab appearance defect detection method, this figure intuitively illustrates the light path principle of light entering the camera through the reflection of the prism, enabling the side camera to acquire a complete image of the side surface of the cotton swab. Thus, the top view acquired by the first camera and the side view acquired by the second camera through the prism correspond strictly in time and space, together constituting the multi-dimensional visual image. This provides a multi-angle, high-contrast raw data foundation for subsequent defect detection, effectively avoiding defect omissions caused by a single viewpoint.

[0024] Furthermore, based on the multidimensional visual image, the embodiments of the present invention calculate the texture anomaly index of the cotton swab to be detected, which can transform the original image data into a quantitative index that can initially and adaptively distinguish normal cotton texture from various potential defects. The potential defects include at least rough edges, twists, and pits. The texture anomaly index is a quantitative index that measures the degree to which the texture features of each region of the cotton swab to be detected deviate from the benchmark of normal cotton swab samples.

[0025] As an embodiment of the present invention, the texture anomaly index of the cotton swab to be detected is calculated based on the multidimensional visual image, including: extracting the texture primitive features of each visual image in the multidimensional visual image; calculating the texture regularity index and texture contrast index of each image region of the cotton swab to be detected based on the texture primitive features; and generating the texture anomaly index of the cotton swab to be detected based on the texture regularity index and the texture contrast index.

[0026] The texture primitive features refer to the basic mathematical descriptions extracted from the pixel grayscale information of the multidimensional visual image, representing the local texture patterns of the cotton swab image to be detected. The image region refers to an independent image region pre-divided in the multidimensional visual image according to the structural characteristics of each part of the cotton swab. For example, the cotton swab head image can be divided into an independent image region, and the cotton swab stalk image can be divided into several rectangular image regions of equal length. The texture regularity index is a numerical metric used to quantify the orderliness and consistency of the texture primitive patterns in each image region. Optionally, the texture regularity index is obtained by calculating the statistical entropy of the texture primitive feature values ​​of all pixels in each image region. The lower the entropy value, the more ordered and consistent the texture arrangement in the region. The texture contrast index is a numerical metric used to quantify the degree of texture difference between each image region and its adjacent background region. It can be obtained by calculating the histograms of the texture primitive feature values ​​of the target image region and its surrounding neighborhood, and by calculating the chi-square distance between the two histograms. The larger the distance value, the more significant the texture difference between the region and the background.

[0027] Optionally, the texture primitive features of each visual image in the multidimensional visual image can be extracted by the Local Binary Pattern (LBP) method; the texture anomaly index of the cotton swab to be detected can be generated by a weighted fusion algorithm.

[0028] S2. Based on the texture anomaly index, locate the defect candidate area of ​​the cotton swab to be detected, and calculate the defect significance score and structural anomaly degree of the defect candidate area.

[0029] According to the texture anomaly index, the defect candidate region of the cotton swab to be tested is located, which can provide a clear target area for subsequent defect detection, improve the efficiency and targeting of the overall detection process. The defect candidate region refers to one or more image sub-regions in the multi-dimensional visual image of the cotton swab to be tested, which are identified by a preset threshold segmentation algorithm based on the spatial distribution and numerical level of the texture anomaly index.

[0030] For example, if the texture anomaly index is superimposed on the original image in the form of a heatmap, the system can binarize the texture anomaly index heatmap according to a preset anomaly threshold, and determine the pixels in the image with index values ​​higher than the threshold as foreground; then, perform connected component analysis on the obtained set of foreground pixels, and merge all spatially adjacent foreground pixels into the same connected region; finally, for each identified connected region, the corresponding region range in the original image is delineated by calculating the boundary of its pixel coordinates, and the range is defined as an independent defect candidate region.

[0031] Furthermore, in this embodiment of the invention, the defect saliency score and structural anomaly degree of the defect candidate region are calculated based on the texture anomaly index. This allows for dual credibility verification and preliminary judgment of defect nature for the initially located suspected regions, thereby achieving efficient filtering of a large number of simple false alarm regions before subsequent defect result determination.

[0032] The defect salience score measures the visual prominence of the defect area, typically calculated based on the contrast between the area and the surrounding background in terms of color, texture, or brightness. The structural anomaly score assesses the degree of deviation of the defect area from normal standards in terms of geometry or structure, such as contour deformation or curvature anomalies.

[0033] As an embodiment of the present invention, calculating the defect saliency score of the defect candidate region based on the texture anomaly index includes: identifying adjacent background regions of the defect candidate region through the boundary range of the defect candidate region; calculating the color difference metric and gradient difference metric between the defect candidate region and the adjacent background region respectively; fusing the color difference metric and the gradient difference metric to determine the visual contrast intensity of the defect candidate region; performing weighted fusion processing on the visual contrast intensity and the texture anomaly index to obtain a comprehensive feature vector; and calculating the defect saliency score of the defect candidate region based on the comprehensive feature vector.

[0034] The boundary range refers to the closed geometric contour of the defect candidate area obtained through image segmentation technology; the adjacent background area refers to a reference area that is topologically adjacent to the boundary range and whose visual features meet normal standards; the color difference metric refers to the degree of deviation between the defect candidate area and the adjacent background area in terms of color distribution patterns. Specifically, for cotton swab stalk staining defects, the a-channel histograms of the defect area and the normal stalk area in the CIELab color space can be extracted separately, and the specific value of the color difference metric can be obtained by calculating the Barcol distance between the two histograms; the gradient difference metric refers to the degree of difference between the defect edge features and the normal texture edge features. Specifically, when detecting loose cotton head defects, the gradient magnitude variance of the defect area and the normal cotton head area after processing by the Laplacian operator can be calculated separately, and the gradient difference metric is obtained by logarithmically transforming the ratio of the two variances; the visual contrast intensity refers to a comprehensive index formed by fusing color and gradient difference features. For example, when simultaneously detecting stains and scratches, the ratio of the Euclidean distance of the color difference to the gradient difference is mapped to the 0-1 range using a sigmoid function. The resulting value represents the visual contrast intensity of the defect in the current context. The comprehensive feature vector refers to a data structure that jointly represents visual contrast intensity and texture anomaly index in the feature space. The dimension and feature weights of this vector are optimized and determined using machine learning methods. For example, multiplying the visual contrast intensity and texture anomaly index by trained weight coefficients, and then combining them with the geometric features of the defect area, forms a six-dimensional feature vector, which is the comprehensive feature vector.

[0035] Optionally, the defect saliency score of the defect candidate region can be calculated using a convolutional neural network model. For example, the comprehensive feature vector can be input into a pre-trained deep convolutional neural network. The network performs hierarchical abstraction and feature extraction of the input features through multiple convolutional and pooling layers. At the end of the network, a fully connected layer maps the high-dimensional features to a defect saliency score. The convolutional neural network is trained in a supervised manner using labeled samples containing various cotton swab defects, and the network parameters are optimized through a backpropagation algorithm to ensure that the score output by the network is consistent with the defect saliency of manually labeled defects.

[0036] As another embodiment of the present invention, calculating the structural anomaly degree of the defect candidate region based on the texture anomaly index includes: calculating the morphological distortion variable of the defect candidate region; obtaining the contour stability parameter of the defect candidate region in a multi-frame temporal image; dynamically correcting the morphological distortion variable based on the contour stability parameter to obtain a corrected morphological distortion variable; and calculating the structural anomaly degree of the defect candidate region based on the texture anomaly index and the corrected morphological distortion variable.

[0037] Further, calculating the morphological distortion of the defect candidate region includes: acquiring the original image of the defect candidate region and extracting the contour envelope features of the defect candidate region in the original image; and calculating the morphological distortion of the defect candidate region based on the contour envelope features.

[0038] The original image refers to the initial digital image of the cotton swab directly acquired by the image acquisition device without geometric correction or content cropping. The contour envelope feature refers to a set of quantitative features describing the geometric shape of the outer boundary of the defect candidate area. The contour envelope feature can be obtained by extracting the contour pixel sequence using a boundary tracking algorithm and then calculating it using Fourier descriptors. The morphological distortion variable refers to the comprehensive quantitative value of the deviation between the actual contour of the defect candidate area and the standard geometric shape. The multi-frame temporal image refers to a sequence of images continuously acquired at the same detection station with a temporal order. These images record the visual state of the cotton swab at different times during transmission. For example, five frames of images of the cotton swab passing through the detection area are continuously acquired at a sampling rate of 30 frames per second. The contour stability parameter refers to the degree of consistency of the contour features of the defect candidate area in the multi-frame temporal image. The corrected morphological distortion variable refers to the morphological distortion variable after temporal stability correction, with a value between 0 and 1. For example, multiplying the initial morphological distortion variable by the reciprocal of the contour stability parameter reduces the distortion variable value when the contour jitter is large, thereby improving the robustness of the detection system to motion blur.

[0039] Optionally, the morphological distortion of the defect candidate region can be calculated using the Hu invariant moment method; the contour stability parameter of the defect candidate region in multi-frame temporal images can be obtained using the optical flow method.

[0040] For example, the structural anomaly degree of the defect candidate region is calculated using the following formula. It should be noted that this calculation method is only one possible method and does not affect the implementation of the basic scheme above:

[0041] in, This indicates the structural anomaly degree of the defect candidate region. This indicates a correction for morphological distortion. Indicates the texture anomaly index. This represents the texture-structure coupling coefficient.

[0042] In detail, the texture-structure coupling coefficient is a normal constant used to quantify the nonlinear enhancement effect of texture anomalies on the degree of structural anomalies. Optionally, the texture-structure coupling coefficient can be calibrated experimentally. For example, by collecting cotton swab structural defect samples containing different degrees of severity, the value of α can be determined using the gradient descent method while ensuring optimal overall performance of the detection system.

[0043] It should be noted that the above formula establishes a nonlinear coupling relationship between the texture anomaly index and the corrected morphological distortion variable through the natural exponential function: when the texture anomaly index... When the value is 0 or close to 0, the exponential term is approximately equal to 1. In this case, the structural anomaly degree S mainly depends on the correction morphological distortion variable. This reflects the contribution of pure geometric deformation; when the texture anomaly index... As the exponential term increases, the same correction morphological distortion variable increases rapidly. This creates an amplification effect. This ensures that even with small geometric deformation, the system can output a high degree of structural anomaly when the surface texture of the cotton swab is severely abnormal.

[0044] S3. Based on the defect saliency score and the structural anomaly degree, generate a defect enhancement feature map of the multidimensional visual image, and extract the defect discrimination features of the cotton swab to be detected from the defect enhancement feature map.

[0045] This invention generates a defect enhancement feature map of the multidimensional visual image based on the defect saliency score and the structural anomaly degree. This can visually enhance and highlight seemingly blurry or subtle abnormal areas in the cotton swab image through scoring and anomaly degree analysis, thereby enabling subsequent algorithms to more accurately and stably identify and judge defects. The defect enhancement feature map refers to an intermediate visual feature representation generated by fusing defect saliency score and structural anomaly degree information based on the original multidimensional visual image. For example, in the defect enhancement feature map, the visual representation of suspected defect areas in the original image is significantly enhanced, making them more prominent in the image; while normal areas are correspondingly weakened.

[0046] As an embodiment of the present invention, generating a defect enhancement feature map of the multidimensional visual image based on the defect saliency score and the structural anomaly degree includes: mapping the defect saliency score and the structural anomaly degree to a first feature enhancement weight map and a second feature enhancement weight map corresponding to the space of the multidimensional visual image, respectively; performing weighted fusion of the first feature enhancement weight map and the second feature enhancement weight map according to a preset fusion coefficient to generate a spatial attention weight map; extracting the original convolutional feature map from the multidimensional visual image, and multiplying the spatial attention weight map element-wise with the original convolutional feature map to obtain the defect enhancement feature map.

[0047] The first feature enhancement weight map refers to expanding, filling, or mapping the calculated defect saliency score into a two-dimensional matrix that is completely consistent with the width and height of the multidimensional visual image according to its corresponding spatial position in the original image. The second feature enhancement weight map is generated in the same logic as the first feature enhancement weight map, but its data source is structural anomaly degree, which will not be elaborated here. The fusion coefficient refers to a set of numerical parameters learned through training, used to quantify the relative importance of the first weight map and the second weight map in the final decision. The spatial attention weight map refers to a comprehensive weight distribution map obtained by weighting and summing the first and second feature enhancement weight maps through the fusion coefficient. The original convolutional feature map refers to the intermediate layer multidimensional data representation containing various basic and high-level features of the cotton swab, which is automatically extracted by the model when the multidimensional visual image is input into a pre-trained convolutional neural network model. For example, after inputting a 224×224 pixel cotton swab head image into a trained ResNet-18 network, the feature map output by a certain intermediate layer of the network, which has a size of 14×14 pixels and 256 channels, is the original convolutional feature map. The element-wise multiplication refers to multiplying the value of the spatial attention weight map one-to-one with all channel feature values ​​at each spatial location in the original convolutional feature map, thereby realizing the mathematical operation of selectively enhancing or suppressing different spatial regions in the feature map.

[0048] It should be explained that the mapping operation specifically includes: for each defect candidate region calculated based on the multidimensional visual image, the defect saliency score or structural anomaly degree corresponding to the candidate region is uniformly assigned to all pixels covered by the candidate region, thereby generating a single-channel two-dimensional matrix aligned with the original image space, where the value of each pixel represents the defect probability or anomaly degree of its region.

[0049] Furthermore, in this embodiment of the invention, by extracting the defect discrimination features of the cotton swab to be tested from the defect enhancement feature map, the high-dimensional features that significantly enhance the information of the defect area can be transformed into a set of more discriminative feature vectors, so as to directly and efficiently serve the subsequent defect classification, localization and severity assessment. The defect discrimination features refer to a set of core data indicators that are further extracted from the defect enhancement feature map and can be directly used to classify, locate or quantitatively assess the defects of the cotton swab. Specifically, the defect discrimination features can be geometric morphology indicators, surface texture indicators and response intensity indicators, such as the density, aspect ratio, roughness, complexity and feature response intensity of the defect area.

[0050] As an embodiment of the present invention, extracting defect discrimination features of the cotton swab to be tested from the defect enhancement feature map includes: determining at least one defect candidate region of the cotton swab to be tested based on the defect enhancement feature map; extracting local depth feature blocks corresponding to the defect candidate regions from the defect enhancement feature map; inputting the local depth feature blocks into a preset feature extraction network, the feature extraction network including a spatial pyramid pooling layer and a fully connected layer; performing multi-scale feature fusion and dimensionality reduction processing on the local depth features through the feature extraction network to obtain a fixed feature vector; and using the fixed feature vector as the defect discrimination feature of the cotton swab to be tested.

[0051] Wherein, the defect candidate region refers to one or more continuous image regions in the defect enhancement feature map that contain cotton swab appearance defects at a high frequency; the local depth feature block refers to a three-dimensional feature tensor set that is directly cropped from the defect enhancement feature map according to the boundary coordinates of the defect candidate region and is spatially completely corresponding to the region, which contains depth feature information at all positions and in all channels within the candidate region; the feature extraction network is a pre-built small neural network module designed to receive local depth feature blocks of variable size, take the local depth feature blocks as input, and output a fixed-length, highly discriminative feature vector through its internal spatial pyramid pooling layer and fully connected layer; the fixed feature vector refers to a one-dimensional numerical array with a predetermined and uniform dimension output by the feature extraction network, the dimension being 128-dimensional, 256-dimensional, or 512-dimensional.

[0052] It should be noted that the feature extraction network is a lightweight network module specifically designed for secondary feature encoding of the local deep feature blocks. It does not undertake the initial image feature extraction task, but instead receives regional feature blocks of varying sizes extracted from the defect enhancement feature map as input. The basic structure of the network consists of a spatial pyramid pooling layer, followed by one or two fully connected layers. Non-linear activation functions such as ReLU are typically used to connect the fully connected layers.

[0053] Furthermore, the local deep features are fused and dimensionality reduced through the feature extraction network to obtain a fixed feature vector. This includes: performing multi-scale partitioning and pooling operations on the local deep feature blocks through the spatial pyramid pooling layer of the feature extraction network to generate a pooled feature vector; and performing feature dimensionality reduction and high-order nonlinear transformation processing on the pooled feature vector through the fully connected layer of the feature extraction network to obtain the fixed feature vector.

[0054] The spatial pyramid pooling layer is a special network layer that is independent of the input size. It divides the input local depth feature block spatially into grids of different scales (e.g., 1x1, 2x2, 4x4), performs pooling operations on all feature values ​​within each grid cell, and then concatenates the pooling results from all scales to form a fixed-length vector. The multi-scale partitioning and pooling operation are the core functions of the spatial pyramid pooling layer, and its specific execution process is as follows: First, the input local depth feature block is divided into multi-scale grids, for example, simultaneously using 1×1, 2×2, and 4×4 grids of different fineness to cover the feature block; then, a pooling function operation is performed on all feature values ​​within each grid, for example, taking the maximum value, thereby compressing the feature information within each grid into a single value representing the features of that region; finally... The pooling results of all grids at all the above scales are concatenated in order of their scale and spatial location to form a unified pooling feature vector. The pooling feature vector refers to the intermediate feature vector with fixed dimensions generated after the multi-scale partitioning and pooling operation of the spatial pyramid pooling layer. The fully connected layer is a basic layer type in neural networks. In this embodiment, it is located after the spatial pyramid pooling layer. Each neuron in the fully connected layer is connected to the pooling feature vector of the previous layer. The feature dimensionality reduction process mainly achieves the mapping of data from high-dimensional space to low-dimensional space by designing the number of output neurons of the fully connected layer to be less than the dimension of the input vector. The high-order nonlinear transformation process includes two steps executed sequentially: first, a linear transformation is performed on the input, and then a nonlinear activation function, such as the ReLU function, is immediately applied to the result of the linear transformation.

[0055] S4. Based on the defect discrimination features, perform defect detection on the cotton swab to be tested to obtain the defect detection result.

[0056] Based on the aforementioned defect discrimination features, this invention performs defect detection on the cotton swab to be tested, obtaining defect detection results. This enables the system to automatically and accurately identify true defects from seemingly messy cotton textures, fundamentally replacing manual visual inspection and rigid rule-based judgment. This significantly improves the accuracy and stability of cotton swab appearance defect detection. The defect detection results refer to the final structured conclusion information about the appearance quality of the cotton swab to be tested, output after comprehensive judgment based on the aforementioned defect discrimination features and a preset defect detection model. The defect detection results include at least the defect category, defect location, and defect severity level.

[0057] As an embodiment of the present invention, based on the defect discrimination features, defect detection of the cotton swab to be detected is performed to obtain defect detection results, including: calling a preset discrimination model, and calculating the defect category probability and position offset of the cotton swab to be detected based on the defect discrimination features and the discrimination model; and generating the defect detection results of the cotton swab to be detected according to the defect category probability and the position offset.

[0058] The discriminant model refers to a neural network module that undergoes supervised learning using a dataset of labeled cotton swab defect images. Its core architecture employs a dual-branch parallel design, with each branch responsible for class determination and position calibration, respectively. The branch architecture shares the underlying information input by the defect discrimination features. The defect category probability refers to a set of normalized values ​​output by the classification branch of the discriminant model, where each value represents the probability that the cotton swab to be detected belongs to a preset defect category, such as "rough," "twisted," "bent," or "no defect," and the sum of all values ​​is 1. The position offset refers to a set of specific numerical parameters output by the regression branch of the discriminant model, and the position offset is usually a multi-dimensional vector, with each component corresponding to the adjustment of the candidate region center point coordinates (X, Y) and the bounding box size (width W, height H).

[0059] As an optional embodiment of the present invention, the defect category probability and position offset of the cotton swab to be detected are calculated based on the defect discrimination features and the discrimination model, including: inputting the defect discrimination features into the classification branch and the regression branch of the discrimination model; outputting the defect category probability of the cotton swab to be detected by the classification branch; outputting the coordinate offset parameter of the defect candidate region of the cotton swab to be detected by the regression branch, and using the coordinate offset parameter as the position offset of the cotton swab to be detected.

[0060] The classification branch is a sub-network parallel to the regression branch in the discrimination model. It consists of multiple fully connected layers, with each neuron connected to all outputs of the previous layer. Its core function is to process defect discrimination features and output defect category probabilities. Specifically, after receiving defect discrimination features, the classification branch generates numerical vectors representing the matching degree of various defects through weighted summation and nonlinear transformation of its internal fully connected layers. These vectors are then normalized using the Softmax function to obtain the probability that the swab to be detected belongs to each preset defect category. The category with the highest probability is the initially determined defect type. The regression branch is set in parallel with the classification branch and is structurally independent. It consists of 1-2 fully connected layers connected sequentially, and the final output layer uses a linear activation function. Its core function is to output coordinate offset parameters of the defect candidate region based on the defect discrimination features. Specifically, after receiving defect discrimination features, the regression branch learns the adjustment rules of defect position and geometry through linear transformation and feature encoding of the fully connected layers, and finally outputs coordinate offset parameters in the form of a multi-dimensional vector to achieve accurate calibration of the defect position.

[0061] Further, the defect category probability of the cotton swab to be tested is output by the classification branch, including: inputting the defect discrimination features into the classification fully connected layer in the classification branch; identifying the high-dimensional classification features in the defect discrimination features through the classification fully connected layer; inputting the high-dimensional classification features into the Softmax function in the classification branch, and calculating the normalized probability of the cotton swab to be tested belonging to each preset defect category through the Softmax function; and determining the defect category probability of the cotton swab to be tested based on the normalized probability.

[0062] The fully connected classification layer is one of the core neural network layers constituting the classification branch. It is a layer composed of multiple neurons, each of which is connected to all outputs of the previous layer. The main function of this layer is to perform a complex weighted summation and nonlinear transformation. Its purpose is to further extract, combine and map high-level abstract information that is more directly related to the defect category judgment from the input defect discrimination features, in order to prepare for the subsequent Softmax probability calculation. For example, assuming the input "defect discrimination feature" is a 256-dimensional vector, the "classification fully connected layer" may have 128 neurons. It will take this 256-dimensional input, perform a weighted summation through its internal 128×256 weights and 128 biases, and typically output a new 128-dimensional vector after passing through a ReLU or other non-linear activation function. The high-dimensional classification feature refers to the new feature vector obtained after the defect discrimination feature has been processed by the classification fully connected layer, such as the 128-dimensional vector output by the classification fully connected layer. The Softmax function is a specific mathematical function that is commonly used as the final output layer for multi-class classification tasks in deep learning. Its core function is to perform probability normalization. Specifically, it can transform an arbitrary real-valued vector into a probability distribution vector; the preset defect category refers to the specific types of swab appearance defects that need to be identified and distinguished by the system before model training, based on the quality inspection requirements in actual production, such as rough edges, twisted strands, bent stalks, and poor head formation; the normalized probability refers to the output vector after processing by the Softmax function, which represents the standardized probability measure of the swab to be detected belonging to each preset defect category.

[0063] In detail, the coordinate offset parameters of the candidate defect region of the cotton swab to be detected are output by the regression branch, including: inputting the defect discrimination features into the regression fully connected layer in the regression branch; performing linear transformation and feature encoding processing on the defect discrimination features through the regression fully connected layer to obtain a regression feature vector; mapping the regression feature vector to the geometric adjustment offset of the candidate defect region, and determining the geometric adjustment offset as the coordinate offset parameter of the candidate defect region.

[0064] The regression fully connected layer is the core neural network layer constituting the regression branch in the discriminant model. Its core function is to learn and extract mapping rules related to the precise adjustment of the spatial location and geometric shape of defects based on the input defect discrimination features, providing key support for generating coordinate offset parameters. Its structure is similar to that of the classification fully connected layer in the classification branch, usually consisting of 1-2 fully connected layers connected sequentially. Each layer contains several neurons, and each neuron is connected to all output features of the previous layer, forming a fully connected mapping relationship. Unlike the classification fully connected layer, in order to ensure that the output coordinate offset parameters are continuous values, the final output layer of the regression fully connected layer uses a linear activation function. The regression vector refers to a fixed-dimensional numerical vector obtained after the defect discrimination features have undergone linear transformation and feature encoding processing by the regression fully connected layer. The geometric adjustment offset refers to a set of specific numerical parameters finally output by the regression branch. It is usually also a multi-dimensional vector, with each dimension corresponding to the adjustment values ​​(ΔX, ΔY, ΔW, ΔH) required to be applied to the x-coordinate (X), y-coordinate (Y), width (W), and height (H) of the initial candidate box.

[0065] Optionally, the regression feature vector can be mapped to the geometric adjustment offset of the defect candidate region by using a linear mapping algorithm combined with a known image processing method of inverse normalization transform.

[0066] Figure 4 shows the functional module diagram of the cotton swab appearance defect detection system based on deep learning of the present invention.

[0067] The deep learning-based cotton swab appearance defect detection system 400 of this invention can be installed in an electronic device. Depending on the functions implemented, the deep learning-based cotton swab appearance defect detection system includes a texture analysis module 401, a defect localization module 402, a feature extraction module 403, and a defect result output module 404. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and is stored in the memory of the electronic device.

[0068] In this embodiment of the invention, the functions of each module / unit are as follows: the texture analysis module 401 is used to acquire a multi-dimensional visual image of the swab to be detected, and calculate the texture anomaly index of the swab based on the multi-dimensional visual image; the defect localization module 402 is used to locate the defect candidate region of the swab to be detected according to the texture anomaly index, and calculate the defect saliency score and structural anomaly degree of the defect candidate region; the feature extraction module 403 is used to generate a defect enhancement feature map of the multi-dimensional visual image based on the defect saliency score and the structural anomaly degree, and extract the defect discrimination features of the swab to be detected from the defect enhancement feature map; the defect result output module 404 is used to perform defect detection of the swab to be detected based on the defect discrimination features, and obtain defect detection results, wherein the defect detection results include at least the defect category, defect location and defect severity level.

[0069] In detail, each module in the deep learning-based cotton swab appearance defect detection system 400 described in this embodiment of the invention employs the same technical means as the deep learning-based cotton swab appearance defect detection method described in Figure 1 above, and can produce the same technical effect, which will not be repeated here.

[0070] In one embodiment, a computer device is provided, which can be a server or a client, and its internal structure diagram is shown in Figure 5. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a deep learning-based cotton swab appearance defect detection method on the server or client side.

[0071] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: S1, acquiring a multidimensional visual image of a swab to be tested, and calculating a texture anomaly index of the swab based on the multidimensional visual image; S2, locating a defect candidate region of the swab to be tested according to the texture anomaly index, and calculating a defect saliency score and structural anomaly degree of the defect candidate region; S3, generating a defect enhancement feature map of the multidimensional visual image based on the defect saliency score and the structural anomaly degree, and extracting defect discrimination features of the swab to be tested from the defect enhancement feature map; S4, performing defect detection on the swab to be tested based on the defect discrimination features, and obtaining a defect detection result, wherein the defect detection result includes at least a defect category, defect location, and defect severity level.

[0072] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program performs the following steps: S1, acquiring a multidimensional visual image of a swab to be tested, and calculating a texture anomaly index of the swab based on the multidimensional visual image; S2, locating a defect candidate region of the swab to be tested according to the texture anomaly index, and calculating a defect saliency score and structural anomaly degree of the defect candidate region; S3, generating a defect enhancement feature map of the multidimensional visual image based on the defect saliency score and the structural anomaly degree, and extracting defect discrimination features of the swab to be tested from the defect enhancement feature map; S4, performing defect detection on the swab to be tested based on the defect discrimination features, and obtaining a defect detection result, wherein the defect detection result includes at least a defect category, a defect location, and a defect severity level.

[0073] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0074] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0075] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0076] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0077] Finally, it should be noted that in the above embodiments, each embodiment can be combined with each other or independent. Deleting any one of them will not affect the technical implementation of other embodiments. The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for detecting cosmetic defects in cotton swabs based on deep learning, characterized in that, The method includes: acquiring a multidimensional visual image of a swab to be tested; calculating a texture anomaly index of the swab based on the multidimensional visual image; locating a candidate defect region of the swab based on the texture anomaly index, and calculating a defect saliency score and structural anomaly degree of the candidate defect region; generating a defect enhancement feature map of the multidimensional visual image based on the defect saliency score and the structural anomaly degree, and extracting defect discrimination features of the swab from the defect enhancement feature map; and performing defect detection on the swab based on the defect discrimination features to obtain a defect detection result, wherein the defect detection result includes at least a defect category, defect location, and defect severity level.

2. The method for detecting cotton swab appearance defects based on deep learning as described in claim 1, characterized in that, Extracting defect discrimination features from the defect enhancement feature map of the swab to be tested includes: determining at least one defect candidate region of the swab to be tested based on the defect enhancement feature map; extracting local depth feature blocks corresponding to the defect candidate regions from the defect enhancement feature map; inputting the local depth feature blocks into a preset feature extraction network, the feature extraction network including a spatial pyramid pooling layer and a fully connected layer; performing multi-scale feature fusion and dimensionality reduction processing on the local depth features through the feature extraction network to obtain a fixed feature vector; and using the fixed feature vector as the defect discrimination feature of the swab to be tested.

3. The method for detecting cosmetic defects in cotton swabs based on deep learning as described in claim 2, characterized in that, The feature extraction network performs multi-scale feature fusion and dimensionality reduction on the local deep features to obtain a fixed feature vector, including: performing multi-scale partitioning and pooling operations on the local deep feature blocks through the spatial pyramid pooling layer of the feature extraction network to generate a pooled feature vector; and performing feature dimensionality reduction and high-order nonlinear transformation processing on the pooled feature vector through the fully connected layer of the feature extraction network to obtain the fixed feature vector.

4. The method for detecting cosmetic defects in cotton swabs based on deep learning as described in claim 1, characterized in that, Based on the defect discrimination features, defect detection of the swab to be tested is performed to obtain defect detection results, including: calling a preset discrimination model, and calculating the defect category probability and position offset of the swab to be tested based on the defect discrimination features and the discrimination model; and generating the defect detection results of the swab to be tested based on the defect category probability and the position offset.

5. The method for detecting cotton swab appearance defects based on deep learning as described in claim 4, characterized in that, Based on the defect discrimination features and the discrimination model, the defect category probability and position offset of the cotton swab to be detected are calculated, including: inputting the defect discrimination features into the classification branch and the regression branch of the discrimination model; outputting the defect category probability of the cotton swab to be detected by the classification branch; outputting the coordinate offset parameter of the defect candidate region of the cotton swab to be detected by the regression branch, and using the coordinate offset parameter as the position offset of the cotton swab to be detected.

6. The method for detecting cotton swab appearance defects based on deep learning as described in claim 5, characterized in that, The process of outputting the defect category probability of the cotton swab to be tested from the classification branch includes: inputting the defect discrimination features into the classification fully connected layer in the classification branch; identifying the high-dimensional classification features in the defect discrimination features through the classification fully connected layer; inputting the high-dimensional classification features into the Softmax function in the classification branch, and calculating the normalized probability of the cotton swab to be tested belonging to each preset defect category through the Softmax function; and determining the defect category probability of the cotton swab to be tested based on the normalized probability.

7. The method for detecting cosmetic defects in cotton swabs based on deep learning as described in claim 5, characterized in that, The coordinate offset parameters of the candidate defect region of the swab to be detected are output by the regression branch, including: inputting the defect discrimination features into the regression fully connected layer in the regression branch; performing linear transformation and feature encoding on the defect discrimination features through the regression fully connected layer to obtain a regression feature vector; mapping the regression feature vector to the geometric adjustment offset of the candidate defect region, and determining the geometric adjustment offset as the coordinate offset parameter of the candidate defect region.

8. The method for detecting cosmetic defects in cotton swabs based on deep learning as described in claim 1, characterized in that, Based on the defect saliency score and the structural anomaly degree, a defect enhancement feature map of the multidimensional visual image is generated, including: mapping the defect saliency score and the structural anomaly degree to a first feature enhancement weight map and a second feature enhancement weight map corresponding to the space of the multidimensional visual image, respectively; weighting and fusing the first feature enhancement weight map and the second feature enhancement weight map according to a preset fusion coefficient to generate a spatial attention weight map; extracting the original convolutional feature map from the multidimensional visual image, and multiplying the spatial attention weight map element-wise with the original convolutional feature map to obtain the defect enhancement feature map.

9. The method for detecting cosmetic defects in cotton swabs based on deep learning as described in claim 1, characterized in that, The structural anomaly degree of the defect candidate region is calculated based on the texture anomaly index, including: calculating the morphological distortion variable of the defect candidate region; obtaining the contour stability parameter of the defect candidate region in a multi-frame temporal image; dynamically correcting the morphological distortion variable based on the contour stability parameter to obtain a corrected morphological distortion variable; and calculating the structural anomaly degree of the defect candidate region based on the texture anomaly index and the corrected morphological distortion variable.

10. A cotton swab appearance defect detection system based on deep learning, characterized in that, The system includes: a texture analysis module for acquiring a multidimensional visual image of the swab to be tested, and calculating a texture anomaly index of the swab based on the multidimensional visual image; a defect localization module for locating a defect candidate region of the swab to be tested according to the texture anomaly index, and calculating a defect saliency score and structural anomaly degree of the defect candidate region; a feature extraction module for generating a defect enhancement feature map of the multidimensional visual image based on the defect saliency score and the structural anomaly degree, and extracting defect discrimination features of the swab to be tested from the defect enhancement feature map; and a defect result output module for performing defect detection on the swab to be tested based on the defect discrimination features, and obtaining a defect detection result, wherein the defect detection result includes at least a defect category, defect location, and defect severity level.

Images