Industrial visual defect detection method and system based on Mamba framework

Abstract

The present invention belongs to the technical field of industrial visual defect detection, and specifically discloses an industrial visual defect detection method and system based on the Mamba framework, the method comprising the following steps: collecting and preprocessing the original product color image data set, building a defect detection model based on the Mamba framework and convolutional neural network, training the defect detection model, and verifying and testing to obtain an optimized defect detection model; collecting a color image of the product to be detected and inputting the optimized defect detection model to obtain an industrial visual defect detection result. The technical solution is adopted to utilize the efficient computing power and unique architectural advantages of the Mamba framework to improve the model training speed, enhance the adaptability to small sample data, and reduce the model complexity while ensuring the detection accuracy, so as to meet the visual defect detection needs in different industrial environments.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial visual defect detection and relates to an industrial visual defect detection method and system based on the Mamba framework. Background Art

[0002] In industrial production, surface defect detection is a critical step in ensuring product quality. With the continuous improvement of industrial automation, machine vision technology has been widely used in defect detection. Traditional industrial visual defect detection methods include those based on image processing technology and those based on traditional machine learning algorithms.

[0003] Image processing-based methods typically use image features such as grayscale, edges, and texture for defect detection. For example, they binarize the image by setting a threshold, then analyze the connected regions within the binarized image to determine whether a defect exists. This method is effective for simple defect types and images with relatively simple backgrounds, but its detection capabilities are limited for complex backgrounds and diverse defects, and it is also sensitive to environmental factors such as lighting.

[0004] Traditional machine learning methods, such as support vector machines (SVMs) and decision trees, require manual feature extraction, which is then used to train a classifier to determine whether a product is defective. However, this manual feature extraction process is cumbersome and relies on specialized knowledge. It struggles to adapt to different defect types and product variations, and suffers from poor generalization.

[0005] In recent years, deep learning technology has achieved tremendous success in image recognition, bringing new opportunities to industrial visual defect detection. Deep learning models can automatically learn features from large amounts of image data, demonstrating strong feature representation and generalization capabilities. However, existing deep learning models still face challenges when processing industrial visual data, such as limited training speed, adaptability to small sample sizes, and the conflict between model complexity and computing resource requirements. Summary of the Invention

[0006] The purpose of the present invention is to provide an industrial visual defect detection method and system based on the Mamba framework to solve the problems of low model training efficiency, poor adaptability to small sample data, and the contradiction between model complexity and computing resource requirements.

[0007] To achieve the above objectives, the basic solution of the present invention is: an industrial visual defect detection method based on the Mamba framework, comprising the following steps:

[0008] Collect original product color image datasets and perform preprocessing;

[0009] Based on the Mamba framework and convolutional neural network, a defect detection model is built. The defect detection model includes multiple convolutional layers, pooling layers, and fully connected layers.

[0010] The preprocessed original product color image dataset is divided into a training set, a validation set, and a test set. The training parameters are set, and the adaptive gradient descent algorithm of the Mamba framework is used to train the defect detection model. The model is then verified and tested to obtain the optimized defect detection model.

[0011] Collect color images of the product to be inspected and input them into the optimized defect detection model to obtain industrial visual defect detection results.

[0012] The working principle and beneficial effects of this basic solution are as follows: The overall industrial visual defect detection system architecture of this technical solution includes image acquisition, preprocessing, model building based on the Mamba framework, training, and detection modules, and their collaborative working method. Leveraging the unique gating mechanism of Mamba units, it effectively captures the temporal and spatial features in image sequences, enabling accurate detection of industrial visual defects. The efficient computing power and unique architecture of the Mamba framework enable the model to better learn complex features in images, thereby improving defect detection accuracy. Compared with existing technologies, defect detection accuracy can be improved by more than 10%.

[0013] The Mamba framework's data processing and computational optimization strategies significantly shorten model training time. Under the same hardware conditions, model training speed can be increased by 2-3 times, enabling rapid model updates for new products or production line adjustments, improving production efficiency.

[0014] Regularization techniques and feature extraction methods within the Mamba framework help prevent overfitting and improve the model's generalization capabilities when using small sample sizes. On small-sample defect datasets, the model's performance stability significantly outperforms traditional methods, enhancing its adaptability to small sample sizes.

[0015] By optimizing the model structure and computational process, the Mamba framework-based defect detection system can reduce the demand for computing resources (such as GPU memory and CPU computing power) while maintaining detection accuracy. This enables the system to be more widely used in resource-constrained industrial environments, reducing equipment costs and deployment difficulties for enterprises, and reducing the demand for computing resources.

[0016] Furthermore, the method for preprocessing the original product color image is as follows:

[0017] Grayscale image:

[0018] Convert the collected original product color image into a grayscale image, and the grayscale is:

[0019] Gray=0.299R+0.587G+0.144B

[0020] Among them, R, G, and B are the pixel values ​​of the red, green, and blue channels in the color image, respectively, and Gray is the grayscale value;

[0021] Image filtering and denoising:

[0022] Using the median filtering algorithm, let the grayscale image be I(x,y), the filter window size be n×n, and n be an odd number, then the image I′(x,y) after median filtering is:

[0023] I′(x,y)=median{I(i,j)|(i,j)∈W n×n (x,y)}

[0024] Among them, W n×n (x,y) represents the n×n neighborhood window centered at the coordinate point (x,y), median represents the median operation, and i,j are the horizontal and vertical coordinates;

[0025] Image enhancement:

[0026] The image contrast is enhanced by the histogram equalization method, and the grayscale histogram of the image is stretched. Assuming the grayscale range of the original image is [0, L-1], n i’ is the number of pixels of gray level i', N is the total number of pixels in the image, then the probability distribution of gray level i'

[0027] The mapping formula of gray level j' after histogram equalization is:

[0028]

[0029] Among them, j' is the gray level after equalization, represents the cumulative probability distribution.

[0030] Preprocessing methods such as median filtering, histogram equalization and normalization are used to optimize processing parameters based on the characteristics of industrial images, improve image quality, enhance defect features, and provide high-quality input data for subsequent model detection.

[0031] Furthermore, the convolution layer uses two sizes of convolution kernels: 3×3 and 5×5. The number of convolution kernels in each layer is 64, the stride is 1, and the padding mode is 'same'.

[0032] For a 3×3 convolution kernel, depth convolution is performed first, and the depth convolution calculation formula is:

[0033]

[0034] Among them, l is the channel number of the convolutional layer, k is the input channel number, is the output eigenvalue of the lth depth convolution kernel at position (i, j), is the weight of the lth depth convolution kernel at the (m,n) position, is the pixel value of the k-th input channel at position (i+m,j+n);

[0035] Perform point-by-point convolution, and the calculation formula is:

[0036]

[0037] in, is the output eigenvalue of the lth point-by-point convolution kernel at the (i, j) position, is the weight of the lth point-by-point convolution kernel on the kth input channel, b l is the bias term of the lth point-by-point convolution kernel, σ is the activation function, and C is the number of input channels;

[0038] The dilated convolution is used to increase the receptive field. The dilated convolution sets the dilation rate to 2 based on the 3×3 convolution kernel. The calculation formula is:

[0039]

[0040] Where r is the void ratio, is the output eigenvalue of the dilated convolution at position (i, j). The meanings of other parameters are similar to those of the common convolution kernel.

[0041] Introducing dynamic feature fusion mechanism, and The output eigenvalues ​​of the 3×3 and 5×5 convolution kernels at the (k, k) position are respectively, and the fused eigenvalue F j,k for:

[0042]

[0043] in, and is the dynamically calculated fusion weight, which is calculated based on the information entropy and gradient change of the local feature area centered at (k,k).

[0044] The convolutional layer introduces a dynamic feature fusion mechanism to improve feature extraction capabilities.

[0045] Furthermore, a maximum pooling layer is set after the convolutional layer, with a pooling window size of 2×2 and a stride of 2. The calculation formula for the maximum pooling is:

[0046] P j2,k =max{O j2,m,n |(m,n)∈W 2×2(k)}

[0047] Among them, j2 is the channel number of the pooling layer, P j2,k is the output value of the j2th channel at the kth pooling window position, O j2,m,n For the j2th channel in the pooling window W 2×2 (k) The pixel value in, max means taking the maximum value operation;

[0048] The pooling layer adopts an adaptive pooling strategy. According to the global feature distribution of the input feature map, the starting position and step size of the pooling window are dynamically adjusted. Let H be the global feature entropy of the input feature map, the starting position (x0, y0) of the pooling window and the step size s be:

[0049] x0=αH x , y0=βH y , s=γH

[0050] Among them, α, β, γ are learnable parameters, H x and H y are the feature entropies of the feature map in the horizontal and vertical directions, respectively.

[0051] This adaptive pooling strategy can better retain important features in the image and improve the model's ability to detect defects of different scales.

[0052] Furthermore, after multiple convolution and pooling operations, the obtained feature map is expanded and classified through a fully connected layer. The fully connected layer contains 128 neurons, and its calculation formula is:

[0053] y=f(Wx+b),

[0054] Where y is the output vector, W is the weight matrix, x is the input feature vector, b is the bias vector, and f is the activation function;

[0055] The number of neurons in the last fully connected layer depends on the number of defect categories. For the binary classification problem: defective and non-defective, the number of neurons is 1, and the sigmoid activation function is used. Compress the output value to the interval [0,1] as the predicted value of the defect probability;

[0056] For multi-classification problems: There are many different types of defects, the number of neurons is the number of defect categories, and the softmax activation function is used. Convert the output into a probability distribution for each category, where y p is the predicted probability of the p-th category, z p is the output value of the pth neuron, z q is the output value of the qth neuron, C1 is the total number of categories;

[0057] Introducing regularization technology, L2 regularization Where λ is the regularization coefficient, w v is the weight parameter.

[0058] Through the collaborative work of various layers of the Mamba framework and the careful design of the model structure, the constructed defect detection model can effectively process industrial visual data, accurately detect various defects on the product surface, and provide reliable technical support for quality control in industrial production.

[0059] Furthermore, the steps for training the defect detection model and obtaining the optimized defect detection model are as follows:

[0060] The pre-processed original product color image dataset is divided into training set, validation set and test set according to the ratio of 70%, 15% and 15% respectively;

[0061] For the binary defect detection problem, binary cross entropy BCE is used as the loss function, and the calculation formula is:

[0062]

[0063] Among them, n1 is the total number of samples, i3 is the sample number, y i3 is the true label of the i3th sample, 0 means no defect, 1 means defect; is the defect probability value predicted by the model;

[0064] For multi-classification problems, the cross entropy loss function CE is used, which is:

[0065]

[0066] Among them, y i3j is the true label of the i3th sample in the j3rd category, is the probability value of the i 3 th sample in the j 3 th category predicted by the model, and C1 is the total number of categories;

[0067] The adaptive gradient descent algorithm Mamba-Adagrad provided by the Mamba framework is used. Its parameters are set as follows: the initial value of the learning rate is 0.01, and the constant ∈ = 1e-8, specifically:

[0068]

[0069] Among them, G t,i3 is the cumulative value of the squared gradient of parameter i3 at the tth iteration, g t,i3 is the gradient of parameter i3 at the tth iteration, θ t,i3 is the value of the parameter at the tth iteration, α is the learning rate;

[0070] Based on Early Stopping: During the training process, the loss function value is calculated on the validation set after every 5 epochs;

[0071] If the validation set loss function value does not decrease for three consecutive epochs, the training is stopped and the parameters of the current model are saved as the final model.

[0072] By optimizing the model structure and computational process, the Mamba framework-based defect detection system can reduce the demand for computing resources (such as GPU memory and CPU computing power) while maintaining detection accuracy. This enables the system to be more widely used in resource-constrained industrial environments, reducing equipment costs and deployment difficulties for enterprises.

[0073] The present invention also provides an industrial visual defect detection system based on the Mamba framework, comprising a data acquisition unit and a processing unit, wherein the data acquisition unit is used to acquire an original product color image data set and a color image of a product to be detected;

[0074] The input end of the processing unit is connected to the output end of the data acquisition unit, and the processing unit executes the method of the present invention to perform industrial visual defect detection.

[0075] This system is based on the Mamba framework and uses data acquisition units and processing units to output defect probability or defect category based on the input image through calculation of convolutional neural networks.

[0076] Furthermore, a defect detection model is provided in the processing unit, and the defect detection model includes one or more Mamba layers, convolution layers, pooling layers, CNN fully connected layers, one or more LSTM layers, LSTM fully connected layers, and an output layer connected in sequence.

[0077] The processing unit reasonably sets the Mamba layer, convolution layer, pooling layer and fully connected layer, and uses the unique gating mechanism of the Mamba unit to effectively capture the temporal and spatial features in the image sequence, thereby realizing accurate detection of industrial visual defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 Schematic diagram of the process of industrial visual defect detection method based on Mamba framework of the present invention;

[0079] Figure 2 It is a structural diagram of the defect detection model of the industrial visual defect detection system based on the Mamba framework of the present invention. DETAILED DESCRIPTION

[0080] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0081] In the description of the present invention, it should be understood that the terms "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention.

[0082] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal communication between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0083] The present invention discloses an industrial visual defect detection method based on the Mamba framework, which utilizes the efficient computing power and unique architectural advantages of the Mamba framework to improve the model training speed and enhance the adaptability to small sample data. At the same time, it reduces the model complexity while ensuring the detection accuracy, so as to meet the visual defect detection needs in different industrial environments. Figure 1 As shown in the figure, the industrial visual defect detection method based on the Mamba framework includes the following steps:

[0084] Collect original product color image datasets and perform preprocessing;

[0085] Based on the Mamba framework and convolutional neural networks, a defect detection model was constructed. This model includes multiple convolutional layers, pooling layers, and fully connected layers, all of which work closely together to achieve accurate defect detection. Depthwise separable convolutions were introduced in the convolutional layers, performing depthwise convolution followed by pointwise convolution to reduce parameters and computational complexity. Atrous convolutions were used to increase the receptive field and better capture long-range defect feature relationships. Max pooling and adaptive pooling strategies were used in the pooling layers, and a batch normalization layer was added before the fully connected layers. Dropout technology was employed to prevent overfitting.

[0086] The Mamba framework utilizes a unique layered architecture consisting of a data layer, a computation layer, and a model layer. The data layer is responsible for reading, preprocessing, and storing data, enabling efficient processing of large-scale image data. The computation layer, based on an efficient tensor computation library, implements fast matrix operations and convolutional operations. The model layer provides a rich set of neural network layer types and model building tools to facilitate the construction of complex deep learning models.

[0087] The preprocessed original product color image dataset is divided into a training set, a validation set, and a test set. The training parameters are set, and the adaptive gradient descent algorithm of the Mamba framework is used to train the defect detection model. The model is then verified and tested to obtain the optimized defect detection model.

[0088] Collect color images of the product to be inspected and input them into the optimized defect detection model to obtain industrial visual defect detection results.

[0089] Preferably, based on the model's output, product images with detected defects are marked. For images with a defect probability greater than a set threshold of 0.5, the defective area is outlined in red, with the defect probability or defect category noted next to it. The system can also alert operators through audible alarms, text message notifications, or pop-up alerts on the monitoring interface, allowing them to promptly address defective products.

[0090] In a preferred embodiment of the present invention, the method for preprocessing the original product color image is:

[0091] Grayscale image:

[0092] Convert the collected original product color image into a grayscale image to reduce the data dimension and the amount of calculation. The grayscale image is:

[0093] Gray=0.299R+0.587G+0.144B

[0094] Among them, R, G, and B are the pixel values ​​of the red, green, and blue channels in the color image, respectively, and Gray is the grayscale value;

[0095] Image filtering and denoising:

[0096] Using the median filtering algorithm (median filtering is a nonlinear filtering method that replaces the grayscale value of each pixel in the image with the median of the grayscale values ​​of the pixels in its neighborhood, thereby effectively removing impulse noise such as salt and pepper noise), let the grayscale image be I(x, y), the filter window size be n×n, where n is an odd number, then the image I′(x, y) after median filtering is:

[0097] I′(x,y)=median{I(i,j)|(i,j)∈W n×n (x,y)}

[0098] Among them, W n×n (x,y) represents the n×n neighborhood window centered at the coordinate point (x,y), median represents the median operation, and i,j are the horizontal and vertical coordinates;

[0099] Image enhancement:

[0100] The histogram equalization method is used to enhance the image contrast and make the defect features in the image more obvious. The principle of histogram equalization is to stretch the grayscale histogram of the image to make its distribution more uniform, thereby improving the image contrast. Assume that the grayscale range of the original image is [0, L-1], n i’ is the number of pixels of gray level i', N is the total number of pixels in the image, then the probability distribution of gray level i'

[0101] The mapping formula of gray level j' after histogram equalization is:

[0102]

[0103] Among them, j' is the gray level after equalization, represents the cumulative probability distribution.

[0104] Optimize processing parameters based on the characteristics of industrial images, improve image quality, enhance defect features, and provide high-quality input data for subsequent model detection.

[0105] In a preferred embodiment of the present invention, the convolution layer uses two sizes of convolution kernels: 3×3 and 5×5. The number of convolution kernels in each layer is 64, the step size is 1, and the filling mode is 'same'.

[0106] For a 3×3 convolution kernel, depth convolution is performed first, and the depth convolution calculation formula is:

[0107]

[0108] Among them, l is the channel number of the convolutional layer, k is the input channel number, is the output eigenvalue of the lth depth convolution kernel at position (i, j), is the weight of the lth depth convolution kernel at the (m,n) position, is the pixel value of the k-th input channel at position (i+m,j+n);

[0109] Perform point-by-point convolution, and the calculation formula is:

[0110]

[0111] in, is the output eigenvalue of the lth point-by-point convolution kernel at the (i, j) position, is the weight of the lth point-by-point convolution kernel on the kth input channel, b l is the bias term of the lth point-by-point convolution kernel, σ is the activation function, and C is the number of input channels;

[0112] For the 5×5 convolution kernel, depthwise convolution is performed first and then point-by-point convolution, and the calculation method is similar.

[0113] The dilated convolution is used to increase the receptive field. The dilated convolution sets the dilation rate to 2 based on the 3×3 convolution kernel. The calculation formula is:

[0114]

[0115] Where r is the void ratio, is the output eigenvalue of the dilated convolution at position (i, j). The meanings of other parameters are similar to those of the common convolution kernel. The dilated convolution can better capture the feature relationship of long-distance defects.

[0116] In order to further improve the feature extraction capability, the convolution layer introduces a dynamic feature fusion mechanism. During the calculation process, the fusion weights of the output features of different convolution kernels are dynamically adjusted according to the local feature complexity of the input image. and The output eigenvalues ​​of the 3×3 and 5×5 convolution kernels at the (k, k) position are respectively, and the fused eigenvalue F j1,k for:

[0117]

[0118] in, and is the dynamically calculated fusion weight, which is calculated based on the information entropy and gradient change of the local feature area centered at (k,k).

[0119] In a preferred embodiment of the present invention, a maximum pooling layer is set after the convolution layer, the pooling window size is 2×2, the step size is 2, and the calculation formula of the maximum pooling is:

[0120] P j2,k =max{O j2,m,n |(m,n)∈W 2×2 (k)}

[0121] Among them, j2 is the channel number of the pooling layer, P j2,k is the output value of the j2th channel at the kth pooling window position, O j2,m,n For the j2th channel in the pooling window W 2×2 (k) The pixel value in, max means taking the maximum value operation;

[0122] The pooling layer adopts an adaptive pooling strategy. According to the global feature distribution of the input feature map, the starting position and step size of the pooling window are dynamically adjusted. Let H be the global feature entropy of the input feature map, the starting position (x0, y0) of the pooling window and the step size s be:

[0123] x0=αH x , y0=βH y , s=γH

[0124] Among them, α, β, γ are learnable parameters, H x and H y are the feature entropies of the feature map in the horizontal and vertical directions, respectively.

[0125] This adaptive pooling strategy can better retain important features in the image and improve the model's ability to detect defects of different scales.

[0126] In a preferred embodiment of the present invention, after multiple convolution and pooling operations, the obtained feature map is expanded and classified through a fully connected layer. The fully connected layer contains 128 neurons, and its calculation formula is:

[0127] y=f(Wx+b),

[0128] Where y is the output vector, W is the weight matrix, x is the input feature vector, b is the bias vector, and f is the activation function (such as ReLU function);

[0129] The number of neurons in the last fully connected layer depends on the number of defect categories. For the binary classification problem: defective and non-defective, the number of neurons is 1, and the sigmoid activation function is used. Compress the output value to the interval [0,1] as the predicted value of the defect probability;

[0130] For multi-classification problems: There are many different types of defects, the number of neurons is the number of defect categories, and the softmax activation function is used. Convert the output into a probability distribution for each category, where y p is the predicted probability of the p-th category, z p is the output value of the pth neuron, z q is the output value of the qth neuron, C1 is the total number of categories;

[0131] Introducing regularization technology, L2 regularization Where λ is the regularization coefficient, w v is a weight parameter to prevent overfitting and improve the generalization ability of the model.

[0132] Through the collaborative work of various layers of the Mamba framework and the careful design of the model structure, the constructed defect detection model can effectively process industrial visual data, accurately detect various defects on the product surface, and provide reliable technical support for quality control in industrial production.

[0133] In a preferred embodiment of the present invention, the steps of training the defect detection model to obtain an optimized defect detection model are as follows:

[0134] The preprocessed original product color image dataset is divided into training set, validation set and test set in the ratio of 70%, 15% and 15% respectively. For example, if there are 10,000 preprocessed images, the first 7,000 images are selected as the training set for model parameter learning and optimization; the 7,001st to 8,500th images are selected as the validation set to evaluate the performance of the model during training and prevent overfitting; the remaining 1,500 images are used as the test set for the final evaluation of the model's generalization ability after model training is completed.

[0135] For the binary defect detection problem, binary cross entropy BCE is used as the loss function, and the calculation formula is:

[0136]

[0137] Among them, n1 is the total number of samples, i3 is the sample number, y i3 is the true label of the i3th sample, 0 means no defect, 1 means defect; is the defect probability value predicted by the model;

[0138] For multi-classification problems, the cross entropy loss function CE is used, which is:

[0139]

[0140] Among them, y i3j is the true label of the i3th sample in the j3rd category (in one-hot encoding form), is the probability value of the i 3 th sample in the j 3 th category predicted by the model, and C1 is the total number of categories;

[0141] The adaptive gradient descent algorithm Mamba-Adagrad provided by the Mamba framework is used. Its parameters are set as follows: the initial value of the learning rate is 0.01, and the constant ∈ = 1e-8, specifically:

[0142]

[0143] Among them, G t,i3 is the cumulative value of the squared gradient of parameter i3 at the tth iteration, g t,i3 is the gradient of parameter i3 at the tth iteration, θt,i3 is the value of the parameter at the tth iteration, α is the learning rate;

[0144] Based on Early Stopping: During the training process, after every 5 epochs (a complete dataset training), the loss function value is calculated on the validation set;

[0145] If the validation set loss function value does not decrease for three consecutive epochs, training is stopped and the current model parameters are saved as the final model. For example, if the validation set loss function value does not decrease between the 20th and 24th epochs, training is stopped after the 24th epoch and the model parameters of the 24th epoch are saved.

[0146] The Adam optimization algorithm is combined with the early stopping method to dynamically adjust the learning rate according to the gradient changes during model training to prevent overfitting, improve model training efficiency and generalization ability, and ensure the stability and accuracy of the model in different industrial scenarios.

[0147] The present invention also provides an industrial visual defect detection system based on the Mamba framework, which includes a data acquisition unit and a processing unit. The data acquisition unit is used to acquire an original product color image data set and a color image of the product to be detected.

[0148] Ideally, the data acquisition unit should use a high-resolution industrial camera. The resolution should be determined based on the required inspection accuracy, for example, cameras with 12 megapixels or higher to ensure clear capture of product surface details. The camera lens should be selected based on the inspection distance and field of view to ensure that the captured image covers the entire product inspection area. Furthermore, a suitable lighting system, such as a ring light or bar light, should be used to provide uniform and stable lighting conditions, minimizing the impact of shadows and reflections on image quality.

[0149] The camera's frame rate is set based on the product's speed and inspection frequency. For example, for small products on a high-speed production line, the frame rate can be set to 50-100 frames per second to ensure that no defects are missed.

[0150] Exposure Time Adjustment: Automatically or manually adjust exposure time to maintain moderate image brightness, avoiding loss of detail due to overbrightness or underbrightness. Exposure time adjustment can be performed in real time based on ambient light changes, ensuring high-quality image capture under varying lighting conditions.

[0151] The input of the processing unit is electrically connected to the output of the data acquisition unit, and the processing unit executes the method of the present invention to perform industrial visual defect detection. Preferably, a high-speed Gigabit Ethernet (GigE Vision) interface is used for image transmission, which has the advantages of fast transmission speed, high stability, and long transmission distance (up to 100 meters or more), making it suitable for large-scale industrial production scenarios. USB 3.0 interface transmission is also supported as a backup solution for use in some small devices or scenarios with low transmission distance requirements, thereby improving system compatibility and flexibility.

[0152] In a preferred embodiment of the present invention, a defect detection model is provided in the processing unit, such as Figure 2 As shown, the defect detection model includes one or more Mamba layers, convolutional layers, pooling layers, CNN fully connected layers, one or more LSTM layers, LSTM fully connected layers, and an output layer, which are connected in sequence.

[0153] The processing unit reasonably sets the Mamba layer, convolution layer, pooling layer and fully connected layer, and uses the unique gating mechanism of the Mamba unit to effectively capture the temporal and spatial features in the image sequence, thereby realizing accurate detection of industrial visual defects.

[0154] This invention leverages the efficient computing power and unique architecture of the Mamba framework, enabling the model to better learn complex features in images, thereby improving the accuracy of defect detection. Compared with existing technologies, the defect detection accuracy can be increased by more than 10%, improving detection accuracy.

[0155] Accelerate model training: The Mamba framework's data processing and computational optimization strategies significantly shorten model training time. Under the same hardware conditions, model training speed can be increased by 2-3 times, enabling rapid model updates for new products or production line adjustments, improving production efficiency.

[0156] Improved adaptability to small sample data: Regularization techniques and feature extraction methods in the Mamba framework help avoid overfitting and improve the model's generalization capabilities when using small sample data. On small sample defect datasets, the model's performance stability significantly outperforms traditional methods.

[0157] Reduced computing resource requirements: By optimizing the model structure and computational process, the Mamba framework-based defect detection system can reduce its demand for computing resources (such as GPU memory and CPU computing power) while maintaining detection accuracy. This enables the system to be more widely used in resource-constrained industrial environments, reducing equipment costs and deployment difficulties for enterprises.

[0158] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0159] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. An industrial visual defect detection method based on the Mamba framework, characterized in that: The steps include: Collect the original product color image data set and perform preprocessing; Based on the Mamba framework and convolutional neural network, a defect detection model is built. The defect detection model includes multiple convolutional layers, pooling layers, and fully connected layers. The preprocessed original product color image dataset is divided into a training set, a validation set, and a test set. The training parameters are set, and the adaptive gradient descent algorithm of the Mamba framework is used to train the defect detection model. The model is then verified and tested to obtain the optimized defect detection model. Collect color images of the product to be inspected and input them into the optimized defect detection model to obtain industrial visual defect detection results.

2. The industrial visual defect detection method based on the Mamba framework according to claim 1, characterized in that: The method for preprocessing the original product color image is: Grayscale image: Convert the collected original product color image into a grayscale image, and the grayscale is: Gray=0.299R+0.587G+0.144B Among them, R, G, B are the pixel values ​​of the red, green, and blue channels in the color image, and Gray is the grayscale value; Image filtering and denoising: Using the median filtering algorithm, let the grayscale image be I(x,y), the filter window size be n×n, and n be an odd number, then the image I′(x,y) after median filtering is: I′(x,y)=median{I(i,j)|(i,j)∈W n×n (x,y)} Among them, W n×n (x,y) represents the n×n neighborhood window centered at the coordinate point (x,y), median represents the median operation, and i,j are the horizontal and vertical coordinates; Image Enhancement: The image contrast is enhanced by the histogram equalization method, and the grayscale histogram of the image is stretched. Assuming that the grayscale range of the original image is [0, L-1], n i’ is the number of pixels of gray level i', N is the total number of pixels in the image, then the probability distribution of gray level i' The mapping formula of gray level j' after histogram equalization is: Among them, j' is the gray level after equalization, represents the cumulative probability distribution.

3. The industrial visual defect detection method based on the Mamba framework as claimed in claim 1, characterized in that: The convolution layer uses two sizes of convolution kernels: 3×3 and 5×5. The number of each convolution kernel is 64, the step size is 1, and the filling method is 'same'. For the 3×3 convolution kernel, the depth convolution is performed first, and the depth convolution calculation formula is: Among them, l is the channel number of the convolutional layer, k is the input channel number, is the output eigenvalue of the lth depth convolution kernel at position (i, j), is the weight of the lth depth convolution kernel at the (m,n) position, is the pixel value of the kth input channel at position (i+m,j+n); Perform point-by-point convolution, and the calculation formula is: in, is the output eigenvalue of the lth point-by-point convolution kernel at position (i, j), is the weight of the lth point-by-point convolution kernel on the kth input channel, b l is the bias term of the lth point-by-point convolution kernel, σ is the activation function, and C is the number of input channels; The dilated convolution is used to increase the receptive field. The dilated convolution sets the dilation rate to 2 based on the 3×3 convolution kernel. The calculation formula is: Where r is the void ratio, is the output eigenvalue of the dilated convolution at position (i, j). The meanings of other parameters are similar to those of ordinary convolution kernels. Introducing dynamic feature fusion mechanism, and The output eigenvalues ​​of the 3×3 and 5×5 convolution kernels at the (k, k) position are respectively, and the fused eigenvalue F j,k for: in, and is the dynamically calculated fusion weight, which is calculated based on the information entropy and gradient change of the local feature area centered on (k, k).

4. The industrial visual defect detection method based on the Mamba framework according to claim 1, characterized in that: A maximum pooling layer is set after the convolutional layer, with a pooling window size of 2×2 and a step size of 2. The calculation formula for the maximum pooling is: P j2,k =max{O j2,m,n |(m,n)∈W 2×2 (k)} Among them, j2 is the channel number of the pooling layer, P j2,k is the output value of the j2th channel at the kth pooling window position, O j2,m,n For the j2th channel in the pooling window W 2×2 (k) The pixel value in, max means taking the maximum value operation; The pooling layer adopts an adaptive pooling strategy. According to the global feature distribution of the input feature map, the starting position and step size of the pooling window are dynamically adjusted. Let H be the global feature entropy of the input feature map, and the starting position (x0, y0) and step size s of the pooling window are: x0=αH x ,y0=βH y ,s=γH Among them, α, β, γ are learnable parameters, H x and H y are the feature entropies of the feature map in the horizontal and vertical directions, respectively.

5. The industrial visual defect detection method based on the Mamba framework as claimed in claim 1, characterized in that: After multiple convolution and pooling operations, the obtained feature map is expanded and classified through the fully connected layer. The fully connected layer contains 128 neurons, and its calculation formula is: y=f(Wx+b), Among them, y is the output vector, W is the weight matrix, x is the input feature vector, b is the bias vector, and f is the activation function; The number of neurons in the last fully connected layer depends on the number of defect categories. For the binary classification problem: defective and non-defective, the number of neurons is 1, and the sigmoid activation function is used. Compress the output value to the interval [0,1] as the predicted value of the defect probability; For multi-classification problems: multiple different types of defects, the number of neurons is the number of defect categories, and the softmax activation function is used. Convert the output to a probability distribution for each category, where y p is the predicted probability of the pth category, z p is the output value of the pth neuron, z q is the output value of the qth neuron, C1 is the total number of categories; Introducing regularization technology, L2 regularization Where λ is the regularization coefficient, w v is the weight parameter.

6. The industrial visual defect detection method based on the Mamba framework according to claim 1, characterized in that: The steps to train the defect detection model and obtain the optimized defect detection model are as follows: The preprocessed original product color image dataset is divided into a training set, a validation set, and a test set in a ratio of 70%, 15%, and 15%; For the binary classification defect detection problem, binary cross entropy BCE is used as the loss function, and the calculation formula is: Among them, n1 is the total number of samples, i3 is the sample number, y i3 is the true label of the i3th sample, 0 means no defect, 1 means defect; is the defect probability value predicted by the model; For multi-classification problems, the cross entropy loss function CE is used, which is: Among them, y i3j is the true label of the i3th sample in the j3rd category, is the probability value of the i-th sample in the j-th category predicted by the model, and C1 is the total number of categories; The adaptive gradient descent algorithm Mamba-Adagrad provided by the Mamba framework is used, and its parameters are set as follows: the initial value of the learning rate is 0.01, and the constant ∈=1e-8, specifically: Among them, G t,i3 is the cumulative value of the squared gradient of parameter i3 at the tth iteration, g t,i3 is the gradient of parameter i 3 at the tth iteration, θ t,i3 is the value of the parameter at the tth iteration, α is the learning rate; Based on Early Stopping: During the training process, the loss function value is calculated on the validation set after every 5 epochs; If the validation set loss function value does not decrease for three consecutive epochs, the training is stopped and the parameters of the current model are saved as the final model.

7. An industrial visual defect detection system based on the Mamba framework, characterized in that: It includes a data acquisition unit and a processing unit, wherein the data acquisition unit is used to acquire an original product color image data set and a product color image to be detected; The input end of the processing unit is connected to the output end of the data acquisition unit, and the processing unit executes the method described in any one of claims 1-6 to perform industrial visual defect detection.

8. The industrial visual defect detection system based on the Mamba framework as claimed in claim 7, characterized in that: A defect detection model is provided in the processing unit, and the defect detection model includes one or more Mamba layers, a convolution layer, a pooling layer, a CNN fully connected layer, one or more LSTM layers, an LSTM fully connected layer, and an output layer connected in sequence.

Images