PCB (Printed Circuit Board) defect detection method based on Mamba model

Abstract

The invention provides a PCB (Printed Circuit Board) defect detection method based on a Mama model, which belongs to the technical field of computer vision, and comprises the following steps of: inputting a time sequence change sequence of a multi-scale feature as a neighborhood feature of a pixel point and a feature vector with a domain change trend as a new token sequence into a Mama encoder, and extracting the feature vector of the domain change trend in a time sequence form of a state space sequence; multi-scale feature fusion is realized, and the requirement of feature scale invariance is met; in addition, positive and negative samples are constructed, and triple training is adopted, so that the feature specificity of PCB defect points is enhanced; according to the invention, image features of different sizes and proportions can be adaptively processed, an alignment-free and registration-free defect detection method is realized, and the defect detection accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular, relates to a PCB board defect detection method based on a Mamba model. Background Art

[0002] Artificial intelligence aims to create machines or software that can perform tasks that usually require human intelligence, covering learning, reasoning, problem solving, language understanding and creation. The main research directions include machine learning, deep learning, natural language processing, etc. Among them, computer vision, as a key technology that enables machines to understand and interpret images, involves tasks such as image recognition, object detection, scene segmentation, and video analysis. With the advancement of hardware technology and the growth of market demand, computer vision has been widely used in many fields such as industrial inspection, autonomous driving, and medical imaging diagnosis.

[0003] Computer vision technology has evolved from early 3D feature recognition methods to current algorithms based on deep learning. From the initial LeNet network to the Unet network and ResNet network, convolutional neural networks (CNNs) have played an important role in the field of computer vision. In recent years, the Transformer model and its visual derivative model, Vision Transformer, have replaced traditional CNNs by processing sequence data and image block sequences through a self-attention mechanism, demonstrating excellent performance.

[0004] As a new sequence model, the Mamba model is based on the Transformer architecture, introduces the state space model SSM, and simplifies the model structure, effectively solving the shortcomings of the Transformer in processing very long sequences and video memory consumption. Its application model in computer vision, Vision Mamba, processes images in a sequential manner by dividing the input image into flattened two-dimensional image blocks (patches), projecting them into vectors (tokens) and inputting them into the encoder, and uses a bidirectional SSM model in the encoder to simplify the visual representation. The model performs well in classification tasks and is highly efficient and accurate in GPU memory and inference time when processing high-resolution images.

[0005] In the industrial field, traditional defect detection methods rely on manual visual inspection, which is inefficient and has limited accuracy. The relevant models can significantly improve detection efficiency and accuracy and reduce human errors through automated detection. In addition, the model can continuously optimize performance through continuous learning and adapt to new types of defects or changing production conditions that may occur during the production process.

[0006] In the Vision Mamba model, the image needs to be converted into a token sequence consisting of patches. The Mamba model processes the tokens in time, so that the model can capture the temporal association of the token sequence to improve the model's understanding of the contextual relationship of the PCB image details. However, defect detection needs to achieve feature scale invariance, that is, when the PCB image scale changes, the feature description and detection of the defect point can still remain stable and consistent. In addition, it is necessary to highlight the feature specificity of the defect point and improve the model's sensitivity to the defect point. Therefore, the Mamba model needs to add a multi-scale feature fusion strategy and feature specificity training so that the model can capture features of different scales. Summary of the invention

[0007] In view of the above problems, the present invention proposes a Mamba-based PCB Defect Detection method, which can adaptively process image features of different sizes and proportions, realize an alignment-free and registration-free defect detection method, improve the accuracy of defect detection, and solve the defect detection problem of industrial devices.

[0008] The present invention is achieved through the following technical solutions:

[0009] A PCB board defect detection method based on Mamba model:

[0010] The method specifically comprises the following steps:

[0011] Step 1: Preprocess the existing PCB image data and scratch data set, randomly generate scratches of varying numbers, sizes, and types on industrial devices to simulate the types of scratches that may occur in actual industrial production, and convert the images in the data set into grayscale images;

[0012] Step 2: Perform multi-scale neighborhood feature fusion, obtain the token sequence T0 through feature extraction, feature mapping and linear projection, so as to realize feature representation of pixel points in different neighborhoods;

[0013] Step 3: Encode the token sequence T0 obtained in step 2 through the Vision Mamba Encoder composed of multiple layers of Vim Blocks, extract the features of the pixel neighborhood change trend in a time series manner, and obtain the output token sequence T L , as pixel features;

[0014] Step 4: Sequence the tokens in step 3 to T L Convert it to an image, and through bilinear interpolation and convolution operations, get the output image img after removing scratches out;

[0015] Step 5: construct a positive and negative sample training model, using triplet training to highlight the feature specificity of the defect points;

[0016] Step 6: Input the original image img in With the model output image img out Calculate the pixel-level difference to get the defect detection result img scratch .

[0017] Furthermore, in step 1,

[0018] First, the PCB board image img p Perform uniform size adjustment; for the PCB board image img s , scale according to the randomly generated size, and perform rotation, translation and padding;

[0019] Then, according to Formula 1, the scratch is placed on the PCB board image img s The position in is mapped to the adjusted PCB board image img p In the figure, it is used as the region of interest (ROI), denoted as roi;

[0020] roi=img p [y start :y end ,x start :x end ] (1)

[0021] where y start ,y end Respectively represent ROI in img s The starting and ending row coordinates in x start 、x end Respectively represent ROI in img s The starting and ending column coordinates in ;

[0022] In the scratch image img s After converting to grayscale and binarizing, generate mask and reverse mask inv ; The reverse mask mask inv Obtained by bitwise inversion of mask; using these two masks, roi and img are respectively s According to formula 2 and 3, we can perform bitwise AND operation to get the background part img bg and the foreground part img fg ;

[0023] img bg =roi∧mask inv (2)

[0024] img fg =img p ∧mask (3)

[0025] The background part img bg and foreground img fg Add them together to get the final composite image dst; finally, replace dst with img according to formula 4 p The corresponding position of the image is converted into a grayscale image to meet the needs of subsequent model training;

[0026] img p [0:rows,0:cols]=dst (4)

[0027] Where rows and cols are the number of rows and columns of dst respectively.

[0028] Furthermore, in step 2, it includes:

[0029] Step 2.1, feature extraction: The input image undergoes multi-level convolution and downsampling operations to achieve multi-level feature extraction; after the above operations, the image is downsampled into four levels of feature maps of different scales, denoted as f1, f2, f3, and f4; among them, f1 is the lowest level feature map, emphasizing the local detail features of the PCB board at a small scale; and f4 is the highest level feature map, containing the global context information of the PCB board image at a large scale;

[0030] Step 2.2, feature mapping: For the four-level feature maps f1, f2, f3, f4 of different scales, the four-level feature maps are mapped to the same spatial dimension through convolution and upsampling operations, and f1′, f2′, f3′, f4′ are obtained through nonlinear activation functions;

[0031] Step 2.3, linear projection: Project f′1, f′2, f′3, and f′4 through the learnable projection matrix W to form a token sequence T0, and add the scale embedding E size , to achieve the fusion of cross-scale features, as shown in Formula 5:

[0032] T0=[f′1W; f′2W; f′3W; f′4W]+E size (5).

[0034] Furthermore, in step 3,

[0035] For the l-th layer of Vim Block (where l∈[0,l-1]), first accept the token sequence T of the previous layer l-1 Normalize through a normalization layer;

[0036] T′ l-1 =Norm(T l-1 ) (6)

[0037] Then the standardized sequence T′ l-1 Linearly map to the main branch vector x and the gated branch vector z to achieve information filtering and capture long-term dependencies;

[0038] x=Linear x (T′ l-1 ) (7)

[0039] z=Linear z (T′ l-1 ) (8)

[0040] The main branch vector x is processed from both the forward and backward directions; for each direction, x is first convolved in one dimension and activated to obtain the main branch encoding vector x′0

[0041] x′0=SiLU(Conv1d0(x)) (9)

[0042] Linearly project x′0 into input influence matrix B0, state influence matrix C0, and step size Δ0, respectively, using the preset parameters and the log function ensures that Δ0 is positive,

[0043]

[0044] Then, A0 and B0 are discretized by step size Δ0 and converted into discrete matrices

[0045]

[0046] The discrete matrix With the state influence matrix C0 and x′0 as the input of the state space model (SSM), the output vector y0 is obtained;

[0047]

[0048] Perform the above operations from the forward and reverse directions respectively to obtain the forward output vector y forward , reverse output vector y backward , and then do Hadamard product with the gated branch vector z after the activation function to get the forward encoding vector y′ forward and the reverse encoding vector y′ backward , thus allowing the model to selectively memorize information to better handle long-term dependencies in sequences;

[0049] y′ forward =y forward⊙SiLU(z) (16)

[0050] y′ backward =y backward ⊙SiLU(z) (17)

[0051] y′ forward and y′ backward After adding and linear transformation, it is combined with the token sequence T l-1 The residual connections are added to obtain the token sequence T of the current layer. l ;

[0052] T l =Linear T (y′ forward +y′ backward )+T l-1 (18)

[0053] The token sequence T of the current layer l Input to the next layer of Vim Block, repeat the above steps, and get the final output toke sequence T L .

[0054] Furthermore, in step 4,

[0055] First, get the token sequence T obtained in step 3 L And the corresponding dimension information, the shape of T (B, N, C) L Reshape into a feature map Feature of shape (B, C, H, W), where B is the batch size, N is the sequence length, C is the feature dimension, H is the image height, and W is the image width;

[0056] Then, the feature map Feature is upsampled using the bilinear interpolation method to match the target image size; finally, convolution is performed to refine the image details and adjust the number of image channels to obtain the final output image img out .

[0057] Furthermore, in step 5,

[0058] The image area containing defect points is taken as a positive sample, and the image area without defect points is taken as a negative sample; an anchor point sample, a positive sample and a negative sample form a triplet, where the anchor point is an image containing defects, the positive sample is an image with the same security defect, and the negative sample is an image without defects;

[0059] Through triplet training, we ensure that the distance between the anchor point and the positive sample is smaller than the distance between the anchor point and the negative sample, thereby strengthening the model's specific recognition of defect point features.

[0060] Further, in step 6,

[0061] The original input image img in With the model output image img out Calculate the pixel-level difference according to formula 19 and get the defect detection result img scratch ;

[0062] img scratch (x,y)=|img in (x,y)-img out (x,y)| (19)

[0063] Among them, x and y represent the horizontal and vertical coordinates of the pixel in the image respectively.

[0064] A PCB board defect detection system based on Mamba model:

[0065] The detection system includes a preprocessing module, a multi-scale neighborhood feature fusion module, a pixel feature encoding module, an image restoration and generation module, and a model training and output module:

[0066] The preprocessing module preprocesses the existing PCB image data and scratch data set, randomly generates scratches of different numbers, sizes and types on industrial devices to simulate the types of scratches that may occur in actual industrial production, and converts the images in the data set into grayscale images;

[0067] The multi-scale neighborhood feature fusion module obtains the token sequence T0 through feature extraction, feature mapping and linear projection, so as to realize feature representation of pixel points in different neighborhood ranges;

[0068] The pixel feature encoding module encodes the token sequence T0 obtained by the multi-scale neighborhood feature fusion module through the encoder Vision MambaEncoder composed of multiple layers of Vim Block, extracts the features of the pixel neighborhood change trend in a time series manner, and obtains the output token sequence T L , as pixel features;

[0069] The image restoration and generation module converts the token sequence T of the pixel feature encoding module into L Convert it to an image, and through bilinear interpolation and convolution operations, get the output image img after removing scratches out ;

[0070] The model training and output module constructs a positive and negative sample training model, adopts a triplet training method, highlights the feature specificity of the defect point, and converts the original input image img in With the model output image img outCalculate the pixel-level difference to get the defect detection result img scratch .

[0071] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0072] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the steps of the above method are implemented.

[0073] Beneficial effects of the present invention

[0074] (1) The Mamba model adopted in the present invention converts an image into a token sequence composed of patches, and processes the token sequence in a time-series manner, so that the model can capture the time-series correlation of the token sequence to improve the model's understanding of the contextual relationship of the PCB image details.

[0075] (2) The multi-scale neighborhood feature fusion module used in the present invention generates four multi-scale information from low to high levels after the input PCB image undergoes multi-level feature extraction operations, representing the local detail features and global context information of the PCB board during the scale change from small to large. This enables the model to represent the features of pixels within different neighborhoods, achieving scale invariance of defect point features.

[0076] (3) The positive and negative samples and triplet training strategy adopted by the present invention ensures that the model shortens the distance between the anchor point and the positive sample in the feature space and pushes the distance between the anchor point and the negative sample away by constructing a triplet of anchor point samples containing defects, positive samples also containing defects, and negative samples without defects. This strategy not only enhances the model's specific recognition of defect point features and improves the accuracy of defect detection, but also optimizes the feature space and enhances the robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 It is the overall framework diagram of the present invention.

[0078] Figure 2 This is an example diagram of the detection effect of the present invention. DETAILED DESCRIPTION

[0079] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0080] The experimental methods used in the following examples are conventional methods unless otherwise specified. The materials, reagents, methods and instruments used are conventional materials, reagents, methods and instruments in the art unless otherwise specified, and can be obtained through commercial channels by those skilled in the art.

[0081] The present invention uses the time-series variation sequence of multi-scale features as the neighborhood feature of the pixel point, and inputs the feature vector with the domain variation trend into the Mamba encoder as a new token sequence, so as to realize multi-scale feature fusion in the time-series form of a state space sequence, and meet the requirement of feature scale invariance. In addition, positive and negative samples are constructed, and triple training is adopted to enhance the feature specificity of the defect points of the PCB board; the model can adaptively process the image features of different sizes and proportions, realize the alignment-free and registration-free defect detection method, and improve the accuracy of defect detection.

[0082] A PCB board defect detection method based on Mamba model, such as Figure 1 As shown:

[0083] The method specifically comprises the following steps:

[0084] Step 1, data preprocessing: Use image segmentation extraction, smoothing, geometric transformation, image fusion and other technologies to process the existing PCB image data and scratch data set, randomly generate scratches of varying numbers, sizes and types on industrial devices to simulate the types of scratches that may occur in actual industrial production, and achieve data enhancement. The images in the data set are converted into grayscale images to reduce the computational complexity of model training while increasing the model's sensitivity to PCB image features;

[0085] First, the PCB board image img p Perform uniform size adjustment. For the PCB board image img s , scale according to the randomly generated size, and perform operations such as rotation, translation, and padding.

[0086] Then, according to Formula 1, the scratch is placed on the PCB board image img s The position in is mapped to the adjusted PCB board image img p In the image, it is taken as the region of interest (ROI), denoted as roi.

[0087] roi=img p [y start :y end ,x start :x end ] (1)

[0088] where y start ,y endRespectively represent ROI in img s The starting and ending row coordinates in x start 、x end Respectively represent ROI in img s The starting and ending column coordinates in ;

[0089] In the scratch image img s After converting to grayscale and binarizing, generate mask and reverse mask inv . The reverse mask mask inv Obtained by bitwise inversion of mask. Using these two masks, roi and img are respectively s According to formula 2 and 3, we can perform bitwise AND operation to get the background part img bg and the foreground part img fg .

[0090] img bg =roi∧mask inv (2)

[0091] img fg =img p ∧mask (3)

[0092] The background part img bg and foreground img fg Add them together to get the final composite image dst. Finally, replace dst with img according to formula 4. p , and converted into a grayscale image to meet the needs of subsequent model training.

[0093] img p [0:rows,0:cols]=dst (4)

[0094] Where rows and cols are the number of rows and columns of dst respectively.

[0095] Step 2: Perform multi-scale neighborhood feature fusion, and obtain the token sequence T0 through feature extraction, feature mapping and linear projection to achieve feature representation of pixels in different neighborhoods.

[0096] In the multi-scale neighborhood feature fusion module, the input image generates four-level multi-scale information graphs f1, f2, f3, and f4 from low to high levels after undergoing multi-level convolution and sampling feature extraction operations. After the nonlinear activation function, the four-level multi-scale information is linearly projected into a token sequence T0, and the scale embedding E is added. size , in order to realize the feature representation of pixels in different neighborhoods.

[0097] Step 2.1, feature extraction: The input image undergoes multi-level convolution and downsampling operations to achieve multi-level feature extraction. Specifically, after the above operations, the image is downsampled into four levels of feature maps of different scales, denoted as f1, f2, f3, and f4. Among them, f1 is the lowest level feature map, emphasizing the local detail features of the PCB board at a small scale; and f4 is the highest level feature map, containing the global context information of the PCB board image at a large scale.

[0098] Step 2.2, feature mapping: For the four-level feature maps f1, f2, f3, f4 of different scales, the four-level feature maps are mapped to the same spatial dimension through convolution and upsampling operations, and f′1, f′2, f′3, f′4 are obtained through nonlinear activation functions.

[0099] Step 2.3, linear projection: Project f′1, f′2, f′3, and f′4 through the learnable projection matrix W to form a token sequence T0, and add the scale embedding E size , to achieve the fusion of cross-scale features, as shown in Formula 5:

[0100] T0=[f′1W; f′2W; f′3W; f′4W]+E size (5).

[0102] Step 3: Encode the token sequence T0 obtained in step 2 through the encoder Vision Mamba Encoder composed of multiple layers of Vim Block, extract the features of the pixel neighborhood change trend in a time series manner, and obtain the output token sequence T L , as pixel features.

[0103] Encode the token sequence T0 obtained in step 2 through the encoder Vision Mamba Encoder to obtain the output token sequence T L The encoder consists of multiple layers of Vim Block. In the lth layer of Vim Block (l∈[0,L-1]), it first receives the token sequence T output by the previous layer. l-1 And normalized, and then linearly mapped into the main branch vector x and the gated branch vector z, where the main branch vector x is used to capture and encode the sequential features of the input data, and the gated branch vector z is used to adjust the information flow of the forward and backward features. Then pass through the one-dimensional convolution and SSM modules from the forward and reverse directions respectively to obtain the forward output vector y forward and the reverse output vector y backward On this basis, the gated branch vector z is used to control y forward ,y backwardPerform gating to adjust the flow of feature information, and add them together to output a new token sequence T l . The token sequence T of this layer l Pass it to the l+1th layer of the Vim encoder to get the output token sequence T of the l+1th layer l+1 ...until the final token sequence T is obtained from the output of the last Vim Block L The feature extraction of the pixel neighborhood change trend is performed in a time series manner as the pixel feature.

[0104] For the l-th layer of Vim Block (where l∈[0,L-1]), first accept the token sequence T of the previous layer l-1 Normalize through a normalization layer.

[0105] T′ l-1 =Norm(T l-1 ) (6)

[0106] Then the standardized sequence T′ l-1 Linearly map to the main branch vector x and the gated branch vector z to achieve information filtering and capture of long-term dependencies.

[0107] x=Linear x (T′ l-1 ) (7)

[0108] z=Linear z (T′ l-1 ) (8)

[0109] The main branch vector x is processed from both the forward and backward directions. For each direction, x is first convolved in one dimension and activated to obtain the main branch encoding vector x′0

[0110] x′0=SiLU(Conv1d0(x)) (9)

[0111] Linearly project x′0 into input influence matrix B0, state influence matrix C0, and step size Δ0, respectively, using the preset parameters and the log function ensures that Δ0 is positive,

[0112]

[0113] Then, A0 and B0 are discretized by step size Δ0 and converted into discrete matrices

[0114]

[0115] The discrete matrix Together with the state influence matrix C0 and x′0, the output vector y0 is obtained as the input of the state space model (SSM).

[0116]

[0117] Perform the above operations from the forward and reverse directions respectively to obtain the forward output vector y forward , reverse output vector y backward , and then do Hadamard product with the gated branch vector z after the activation function to get the forward encoding vector y′ forward and the reverse encoding vector y′ backward , thus allowing the model to selectively memorize information to better handle long-term dependencies in sequences.

[0118] y′ forward =y forward ⊙SiLU(z) (16)

[0119] y′ backward =y backward ⊙SiLU(z) (17)

[0120] y′ forward and y′ backward After adding and linear transformation, it is combined with the token sequence T l-1 The residual connections are added to obtain the token sequence T of the current layer. l .

[0121] T l =Linear T (y′ forward +y′ backward )+T l-1 (18)

[0122] The token sequence T of the current layer l Input to the next layer of Vim Block, repeat the above steps, and get the final output toke sequence T L .

[0123] Step 4: In the Prediction layer, the token sequence T from step 3 is L Convert it to an image, and through bilinear interpolation and convolution operations, get the output image img after removing scratches out .

[0124] First, get the token sequence T obtained in the previous step L And the corresponding dimension information, the shape of T (B, N, C) LReshape into a feature map Feature of shape (B, C, H, W), where B is the batch size, N is the sequence length, C is the feature dimension, H is the image height, and W is the image width. Then use the bilinear interpolation method to upsample the feature map Feature to match the target image size. Finally, convolution is used to refine the image details and adjust the number of image channels to obtain the final output image img out .

[0125] Step 5, model training: construct positive and negative samples, and use triplet training. The positive sample refers to the image area containing defects, while the negative sample refers to the image area without defects. At the same time, after triplet training, ensure that the distance between the anchor point and the positive sample is smaller than the distance between the anchor point and the negative sample, so that the model can distinguish the key features of defects and non-defective areas, thereby highlighting the feature specificity of the defect point.

[0126] The image area containing defects is taken as a positive sample, and the image area without defects is taken as a negative sample. An anchor sample, a positive sample, and a negative sample form a triplet, where the anchor is an image containing defects, the positive sample is an image with the same security defect, and the negative sample is an image without defects. Through triplet training, it is ensured that the distance between the anchor and the positive sample is smaller than the distance between the anchor and the negative sample, thereby strengthening the model's specific recognition of defect point features.

[0127] Step 6: Output result processing: convert the original input image img in With the model output image img out Calculate the pixel-level difference to get the defect detection result img scratch .

[0128] The original input image img in With the model output image img out Calculate the pixel-level difference according to formula 19 and get the defect detection result img scratch .

[0129] img scratch (x,y)=|img in (x,y)-img out (x,y)| (19)

[0130] Among them, x and y represent the horizontal and vertical coordinates of the pixel in the image respectively.

[0131] Finally, the test results of the embodiment are as follows Figure 2 As shown in Table 1, the accuracy is shown in Table 1. Compared with the existing defect detection methods, the method of the present invention shows higher accuracy.

[0132] Table 1 Comparison results of defect detection accuracy between the method of the present invention and the existing method

[0133]

[0134] A PCB board defect detection system based on Mamba model:

[0135] The detection system includes a preprocessing module, a multi-scale neighborhood feature fusion module, a pixel feature encoding module, an image restoration and generation module, and a model training and output module:

[0136] The preprocessing module preprocesses the existing PCB image data and scratch data set, randomly generates scratches of different numbers, sizes and types on industrial devices to simulate the types of scratches that may occur in actual industrial production, and converts the images in the data set into grayscale images;

[0137] The multi-scale neighborhood feature fusion module obtains the token sequence T0 through feature extraction, feature mapping and linear projection, so as to realize feature representation of pixel points in different neighborhood ranges;

[0138] The pixel feature encoding module encodes the token sequence T0 obtained by the multi-scale neighborhood feature fusion module through the encoder Vision MambaEncoder composed of multiple layers of Vim Block, extracts the features of the pixel neighborhood change trend in a time series manner, and obtains the output token sequence T L , as pixel features;

[0139] The image restoration and generation module converts the token sequence T of the pixel feature encoding module into L Convert it to an image, and through bilinear interpolation and convolution operations, get the output image img after removing scratches out ;

[0140] The model training and output module constructs a positive and negative sample training model, adopts a triplet training method, highlights the feature specificity of the defect point, and converts the original input image img in With the model output image img out Calculate the pixel-level difference to get the defect detection result img scratch .

[0141] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0142] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the steps of the above method are implemented.

[0143] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory, ROM, a programmable read-only memory, PROM, an erasable programmable read-only memory, EPROM, an electrically erasable programmable read-only memory, EEPROM, or a flash memory. The volatile memory may be a random access memory, RAM, which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory static RAM, SRAM, dynamic random access memory dynamic RAM, DRAM, synchronous dynamic random access memory synchronous DRAM, SDRAM, double data rate synchronous dynamic random access memory double data rate SDRAM, DDR SDRAM, enhanced synchronous dynamic random access memory enhanced SDRAM, ESDRAM, synchronous link dynamic random access memory synchlink DRAM, SLDRAM, and direct memory bus random access memory direct rambus RAM, DR RAM. It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0144] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center through a wired method such as coaxial cable, optical fiber, digital subscriber line digital subscriber line, DSL or wireless such as infrared, wireless, microwave, etc. to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium such as a floppy disk, a hard disk, a tape, an optical medium such as a high-density digital video disc digital video disc, DVD, or a semiconductor medium such as a solid state hard disk solid state disc, SSD, etc.

[0145] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0146] It should be noted that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor can be combined to perform. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0147] The above is a detailed introduction to the PCB board defect detection method based on the Mamba model proposed in the present invention, and the principle and implementation mode of the present invention are explained. The description of the above embodiment is only used to help understand the method and core idea of ​​the present invention; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A PCB board defect detection method based on the Mamba model, characterized in that: The method specifically comprises the following steps: Step 1: Preprocess the existing PCB image data and scratch data set, randomly generate scratches of varying numbers, sizes, and types on industrial devices to simulate the types of scratches that may occur in actual industrial production, and convert the images in the data set into grayscale images; Step 2: Perform multi-scale neighborhood feature fusion, obtain the token sequence T0 through feature extraction, feature mapping and linear projection, so as to realize feature representation of pixel points in different neighborhoods; Step 3: Encode the token sequence T0 obtained in step 2 through the Vision Mamba Encoder composed of multiple layers of Vim Blocks, extract the features of the pixel neighborhood change trend in a time series manner, and obtain the output token sequence T L , as pixel features; Step 4: Sequence the tokens in step 3 to T L Convert it to an image, and through bilinear interpolation and convolution operations, get the output image img after removing scratches out ; Step 5: construct a positive and negative sample training model, using triplet training to highlight the feature specificity of the defect points; Step 6: Convert the original input image imgi n And the model output image imgou t Calculate the pixel-level difference to get the defect detection result img scratch .

2. The detection method according to claim 1, characterized in that: In step 1, First, the PCB board image img p Perform uniform size adjustment; for the PCB board image img s , scale according to the randomly generated size, and perform rotation, translation and padding; Then, according to Formula 1, the scratch is placed on the PCB board image img s The position in is mapped to the adjusted PCB board image img p In the figure, it is used as the region of interest (ROI), denoted as roi; roi=img p [and start :and end ,x start :x end ] (1) where y start ,y end Respectively represent ROI in img s The starting and ending row coordinates in x start 、x end Respectively represent ROI in img s The starting and ending column coordinates in ; In the scratch image img s After converting to grayscale and binarizing, generate mask and reverse mask inv ; The reverse mask mask inv Obtained by bitwise inversion of mask; using these two masks, roi and img are respectively s According to formula 2 and 3, we can perform bitwise AND operation to get the background part img bg and the foreground part img fg ; img bg =roi∧mask inv (2) img fg =img p ∧mask (3) The background part img bg and foreground img fg Add them together to get the final composite image dst; finally, replace dst with img according to formula 4 p The corresponding position of the image is converted into a grayscale image to meet the needs of subsequent model training; img p [0:rows,0:cols]=dst (4) Where rows and cols are the number of rows and columns of dst respectively.

3. The detection method according to claim 2, characterized in that: In step 2, include: Step 2.1, feature extraction: The input image undergoes multi-level convolution and downsampling operations to achieve multi-level feature extraction; after the above operations, the image is downsampled into four levels of feature maps of different scales, denoted as f1, f2, f3, and f4; among them, f1 is the lowest level feature map, emphasizing the local detail features of the PCB board at a small scale; and f4 is the highest level feature map, containing the global context information of the PCB board image at a large scale; Step 2.2, feature mapping: For the four-level feature maps f1, f2, f3, and f4 of different scales, the four-level feature maps are mapped to the same spatial dimension through convolution and upsampling operations, and f′1, f′2, f′3, and f′4 are obtained through nonlinear activation functions; Step 2.3, linear projection: Project f′1, f′2, f′3, and f′4 through the learnable projection matrix W to form a token sequence T0, and add the scale embedding E size , to achieve the fusion of cross-scale features, as shown in Formula 5: T0=[f′1W;f′2W;f′3W;f′4W]+E size (5)。 4. The detection method according to claim 3, characterized in that: In step 3, For the l-th layer of Vim Block (where l∈[0, L-1]), first accept the token sequence T of the previous layer l-1 Normalize through a normalization layer; T′ l-1 =Norm(T l-1 ) (6) Then the standardized sequence T′ l-1 Linearly map to the main branch vector x and the gated branch vector z to achieve information filtering and capture of long-term dependencies; x=Linear x (T′ l-1 ) (7) z=Linear z (T′ l-1 ) (8) The main branch vector x is processed from both the forward and backward directions; for each direction, x is first convolved in one dimension and activated to obtain the main branch encoding vector x′0 x′0=SiLU(Conv1d0(x)) (9) Linearly project x′0 into input influence matrix B0, state influence matrix C0, and step length Δ0, respectively, using the preset parameters and the log function ensures that Δ0 is positive, Then, A0 and B0 are discretized by step size Δ0 and converted into discrete matrices The discrete matrix With the state influence matrix C0 and x′0 as the input of the state space model (SSM), the output vector y0 is obtained; Perform the above operations from the forward and reverse directions respectively to obtain the forward output vector y forward , reverse output vector y backward , and then do Hadamard product with the gated branch vector z after the activation function to get the forward encoding vector y′ forward and the reverse encoding vector y′ backward , thus allowing the model to selectively memorize information to better handle long-term dependencies in sequences; and' forward =and forward ⊙SiLU(z) (16) and' backward =and backward ⊙SiLU(z) (17) y′ forward and y′ backward After adding and linear transformation, it is combined with the token sequence T l-1 The residual connections are added to obtain the token sequence T of the current layer. l ; T l =Linear T (y′ forward +y′ backward )+T l-1 (18) The token sequence T of the current layer l Input to the next layer of Vim Block, repeat the above steps, and get the final output toke sequence T L .

5. The detection method according to claim 4, characterized in that: In step 4, First, get the token sequence T obtained in step 3 L And the corresponding dimension information, the shape of T (B, N, C) L Reshape into a feature map Feature of shape (B, C, H, W), where B is the batch size, N is the sequence length, C is the feature dimension, H is the image height, and W is the image width; Then, the feature map Feature is upsampled using the bilinear interpolation method to match the target image size; finally, convolution is performed to refine the image details and adjust the number of image channels to obtain the final output image img out .

6. The detection method according to claim 5, characterized in that: In step 5, The image area containing defect points is taken as a positive sample, and the image area without defect points is taken as a negative sample; an anchor point sample, a positive sample and a negative sample form a triplet, where the anchor point is an image containing defects, the positive sample is an image with the same security defect, and the negative sample is an image without defects; Through triplet training, we ensure that the distance between the anchor point and the positive sample is smaller than the distance between the anchor point and the negative sample, thereby strengthening the model's specific recognition of defect point features.

7. The detection method according to claim 6, characterized in that: In step 6, The original input image img in With the model output image img out Calculate the pixel-level difference according to formula 19 and get the defect detection result img scratch ; img scratch (x,y)=|img in (x,y)-img out (x,y)| (19) Among them, x and y represent the horizontal and vertical coordinates of the pixel in the image respectively.

8. A detection system for executing the PCB board defect detection method based on the Mamba model as claimed in any one of claims 1 to 7, characterized in that: The detection system includes a preprocessing module, a multi-scale neighborhood feature fusion module, a pixel feature encoding module, an image restoration and generation module, and a model training and output module: The preprocessing module preprocesses the existing PCB image data and scratch data set, randomly generates scratches of different numbers, sizes and types on industrial devices to simulate the types of scratches that may occur in actual industrial production, and converts the images in the data set into grayscale images; The multi-scale neighborhood feature fusion module obtains the token sequence T0 through feature extraction, feature mapping and linear projection, so as to realize feature representation of pixel points in different neighborhood ranges; The pixel feature encoding module encodes the token sequence T0 obtained by the multi-scale neighborhood feature fusion module through the encoder Vision MambaEncoder composed of multiple layers of Vim Block, extracts the features of the pixel neighborhood change trend in a time series manner, and obtains the output token sequence T L , as pixel features; The image restoration and generation module converts the token sequence T of the pixel feature encoding module into L Convert it to an image, and through bilinear interpolation and convolution operations, get the output image img after removing scratches out ; The model training and output module constructs a positive and negative sample training model, adopts a triplet training method, highlights the feature specificity of the defect point, and converts the original input image img in With the model output image img out Calculate the pixel-level difference to get the defect detection result img scratch .

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images