PCB alignment detection method based on self-supervised symmetric perception

Abstract

The invention discloses a PCB alignment detection method based on self-supervised symmetric perception, and the method comprises the following steps: collecting a data set of a surface-mounted element of a printed circuit board, which contains a symmetric characteristic, and carrying out the preprocessing of the data set; analyzing the central symmetry and axial symmetry relation of the surface-mounted element; constructing an image transformation unit by using symmetric information, generating various auxiliary training views, and constructing a convolutional neural network to respectively extract features of a symmetric center, a symmetric axis angle and a bounding box size; inputting the extracted features into a three-branch regression sub-network for dimension reduction processing, and calculating a consistency loss function to realize self-supervised learning; and finally outputting the position of the symmetric center, the direction of the symmetric axis and the size information of the bounding box of the surface-mounted element. According to the method, the symmetry characteristic of the surface-mounted element is fully utilized, efficient detection of displacement, direction and size can be achieved without labeling data, and the method is suitable for the field of PCB quality detection with the high real-time requirement.

Description

Technical Field

[0001] The present invention relates to a PCB alignment detection method based on self-supervised symmetry perception, belonging to the technical field of two-dimensional object detection. Background Art

[0002] As a core component of modern electronic products, the quality of surface-mounted components (SMCs) on a printed circuit board (PCB) directly affects the performance and reliability of the product. In surface mount technology (SMT), component displacement and orientation anomalies are common defects, which may lead to degraded electrical performance and even equipment failure. Traditional quality inspection mainly relies on manual or rule-based algorithms, but with the increasing complexity of PCB design and production scale, these methods gradually show problems such as low efficiency and insufficient robustness.

[0003] In recent years, automatic optical inspection (AOI) technology based on deep learning has made remarkable progress in the field of PCB defect detection. Researchers have proposed various methods, such as using convolutional neural networks (CNNs) to extract features, designing rotated bounding box annotations to optimize the positioning performance, and constructing lightweight models to accelerate inference. However, these methods usually rely on a large amount of labeled data and are difficult to efficiently meet the alignment detection requirements of symmetric components. SMCs on the PCB surface generally have rotational symmetry and reflection symmetry characteristics, but existing research has not fully utilized this characteristic, resulting in insufficient detection accuracy of the model in symmetric scenarios.

[0004] For example, many SMCs such as resistors, capacitors, and diodes have symmetry characteristics that determine the high-precision detection requirements for their displacement and orientation. Therefore, how to use the symmetry characteristics of components to achieve efficient alignment detection without annotation has become a key technical issue for improving the level of PCB quality inspection, with important research value and broad application prospects. Summary of the Invention

[0005] Technical Problem. The object of the present invention is to provide a PCB alignment detection method based on self-supervised symmetry perception for the deficiencies existing in the prior art. By designing a self-supervised learning framework and combining an image transformation unit to generate diverse auxiliary views, this method strengthens the learning of rotational symmetry and reflection symmetry features, aiming to improve the detection accuracy of SMCs in the actual production environment. By optimizing the angle regression and symmetry center prediction capabilities of the model, the present invention can achieve efficient and accurate PCB component alignment detection, especially suitable for real-time detection tasks in SMT.

[0006] Technical Solution. To solve the above technical problems, the present invention proposes a PCB alignment detection method based on self-supervised symmetry perception, which includes the following steps:

[0007] Step 1: Collect a dataset of surface-mounted components on a printed circuit board (PCB) with central symmetry and axial symmetry characteristics, and preprocess the dataset, including data augmentation operations such as random horizontal flipping, vertical flipping, rotation, and scaling.

[0008] Step 2: Analyze the symmetry of the surface-mounted components on the PCB. Based on the characteristics of the symmetry center and axis of symmetry, construct an Image Transformation Unit (ITU) to generate auxiliary views for training, including 180° rotation, random translation, vertical flipping, random rotation, and random scaling views.

[0009] Step 3: Design a self-supervised symmetry-aware network. The network consists of a convolutional neural network backbone and an angle regression branch, a center regression branch, and a bounding box regression branch. Input the data from the original dataset in Step 1 and the data after image transformation in Step 2 into the network. The angle regression branch is used to predict the axis of symmetry angle of the surface-mounted component, the center regression branch is used to predict the symmetry center position, and the bounding box regression branch is used to predict the target box size of the surface-mounted component.

[0010] Step 4: Construct a consistency loss function including angle consistency loss, center consistency loss, and bounding box consistency loss to guide model training.

[0011] Step 5: Use the trained model for PCB alignment detection.

[0012] Furthermore, the specific method of Step 1 is as follows:

[0013] The dataset of surface-mounted components on the PCB has undergone the following processing:

[0014] Step 1.1: Extract the images of surface-mounted components from the original image data, and perform normalization processing on them. The resolution of each image is uniformly adjusted to 128×128 pixels, and the data undergoes image enhancement, including rotation, scaling, and color transformation processing.

[0015] Step 1.2: All images are calibrated by an Automatic Optical Inspection (AOI) device during the acquisition process, and the images are denoised and edge-detected.

[0016] Step 1.3: Divide the dataset into a training set, a validation set, and a test set according to a ratio of 60:20:20.

[0017] Furthermore, the specific method of Step 2 is as follows:

[0018] Step 2.1: Obtain the auxiliary views related to the symmetry center point during the training process: Generate views of the training center through 180° rotation and random translation transformations. The 180° rotation rotates the image 180° around the origin, and the random translation translates the original image through a random translation vector Δ, with a translation threshold T set.tsl Control the range of the random translation vector Δ, and the specific calculation formula is as follows:

[0019]

[0020] Among them, Δ xr and Δ yr are random numbers between -1 and 1, and ω and h are the width and height of the input image respectively;

[0021] Step 2.2, obtain the auxiliary view related to the symmetry axis angle during the training process: generate the view of the training angle through vertical flipping and random rotation transformation. The vertical flipping flips the image up and down, and the random rotation rotates the original image counterclockwise by a random rotation angle R. The random rotation angle is controlled by the rotation threshold T rot The calculation formula for the rotation angle R is as follows:

[0022] R = T rot × R r × 90°

[0023] Among them, R r is a random number between -1 and 1, and T rot controls the range of random rotation;

[0024] Step 2.3, obtain the auxiliary view related to the bounding box: perform a scaling transformation on the original picture with a random scaling ratio. Assume the original image size is W0×H0, where W0 and H0 represent the width and height of the original image respectively. Scale the image with a random scaling ratio S ∈ [0.8, 1.2] to obtain the new image size W′×H′, where:

[0025] W′ = W0×S, H′ = H0×S.

[0026] Furthermore, the specific method of step 3 is as follows:

[0027] Step 3.1, Convolutional Neural Network stage

[0028] The Convolutional Neural Network stage includes a CNN backbone network and three sub-networks. The CNN backbone network uses a pre-trained ResNet18 model, and its classification layer is replaced by an angle regression sub-network, a center point regression sub-network, and a bounding box regression sub-network. The input image is the preprocessed PCB surface mount component dataset image, and the output is the output values of the three sub-networks;

[0029] Assume there are three sub-networks: f ang (i), f cen (I) and f bbox(I), which are respectively used to output the symmetry axis angle θ, the symmetry center position (x, y) and the bounding box size (ω, h), are defined as follows:

[0030] f ang (I) = θ

[0031] f cen (I) = (x, y)

[0032] f bbox (I) = (ω, h)

[0033] wherein, the center of the image is set as the coordinate origin (0, 0);

[0034] Step 3.2, Angle Regression Sub-network

[0035] The angle regression sub-network is used to predict the symmetry axis angle θ of the PCB in the image. The input of this sub-network is the image after angle branch transformation and the original image. The input passes through a 1×1 convolution, a sigmoid activation function and a phase shift encoder, and outputs a predicted symmetry axis angle value θ;

[0036] Step 3.3, Center Point Regression Sub-network

[0037] The center point regression sub-network is used to predict the symmetry center position (x, y) of the PCB in the image. The input of this sub-network is the image after center point branch transformation and the original image. The input passes through a 1×1 convolution and a sigmoid activation function, and outputs a predicted symmetry center position (x, y);

[0038] Step 3.4, Bounding Box Regression Sub-network

[0039] The bounding box regression sub-network is used to predict the bounding box size (ω, h) of the PCB components in the image. The input of this sub-network is the image after bounding box branch transformation and the original image. The input passes through a 1×1 convolution and a sigmoid activation function, and outputs a predicted bounding box size (ω, h).

[0040] Furthermore, the specific method of step 4 is as follows:

[0041] Step 4.1, Consistency Loss Function

[0042] 1. Angle Consistency Loss

[0043] The angle consistency loss is used to optimize the angle prediction under rotational symmetry characteristics and is defined as follows:

[0044] L ang = α × L flp + L rot

[0045] Among them, α is a hyperparameter for controlling the weight of the sub-loss, and L flp is the angle loss obtained through the vertical flip transformation, which is defined as:

[0046] L flp = L(f ang (I flp ), -θ)

[0047] Among them, θ is the target symmetry axis angle, and L rot is the angle loss obtained through the rotation transformation, which is defined as:

[0048] L rot = L(f ang (I rot ), -θ, R)

[0049] Among them, I flp and I rot respectively represent the flipped and rotated images generated by the image transformation unit, R is the random rotation angle, and L is the loss function;

[0050] 2. Center consistency loss

[0051] The center consistency loss is used to optimize the center position prediction under the translational symmetry characteristic, and is defined as follows:

[0052] L cen = β × L rot180 + L tsl

[0053] Among them, β is a hyperparameter for controlling the weight of the sub-loss, and L rot180 is the center loss obtained through the 180° rotation transformation, which is defined as:

[0054] L rot180 = L(f cen (I rot180 ), -p)

[0055] L tsl is the center loss obtained through the random translation transformation, which is defined as:

[0056] L tsl = L(f cen (I tsl ), p, Δ)

[0057] Among them, I rot180 and I tsl respectively represent the 180° rotation and translation images generated by the image transformation unit, p is the target symmetry center, Δ is the random translation vector, and L is the loss function;

[0058] 3. Bounding box consistency loss

[0059] The bounding box consistency loss is used to optimize the accuracy of the model's bounding box prediction under transformations such as rotation, flipping, and scaling, and is defined as follows:

[0060] L bbox = γ × L flp + L rot + L sca

[0061] where γ is a hyperparameter that controls the weight of the sub-loss, and L flp is the bounding box loss obtained through the vertical flipping transformation and is defined as:

[0062] L flp = L(f bbox (I flp ), -b)

[0063] where b = (ω, h) is the target bounding box, and L rot is the bounding box loss obtained through the rotation transformation and is defined as:

[0064] L rot = L(f bbox (I rot ), -b, R)

[0065] where R is the random rotation angle, and L sca is the bounding box loss obtained through the scaling transformation and is defined as:

[0066] L sca = L(f bbox (I sca ), -b, S)

[0067] where I flp , I rot and I sca represent the flipped, rotated, and scaled images generated by the image transformation unit respectively, S is the scaling factor, and L is the loss function;

[0068] Step 4.2, Loss function optimization

[0069] Combining the angle consistency loss and the center consistency loss, the total loss function formula is defined as:

[0070] L total = ω1 · L ang + ω2 · L cen + ω3 · L sca

[0071] Among them, ω1, ω2, and ω3 are the weight coefficients of the angle, center point, and bounding box losses respectively, which are hyperparameters used to balance the contributions of the three sub-losses. During the training process, the network parameters are optimized through backpropagation to gradually minimize the total loss function.

[0072] Further, the specific method of step 5 is as follows:

[0073] Step 5.1, the input PCB image generates an auxiliary view.

[0074] Step 5.2, the generated image and the original image are input into the CNN stage, and are respectively input into the angle regression sub-network, the center regression sub-network, and the bounding box regression sub-network, and the symmetry axis angle of the target is output Symmetric center coordinates and the target bounding box size

[0075] Step 5.3, the model calculates the rotation detection box of the target through the prediction results of the symmetry axis angle Symmetric center coordinates and the target bounding box size of the target.

[0076] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0077] (1) The present invention provides a PCB alignment detection method based on self-supervised symmetry perception, which fully utilizes the rotational symmetry and reflection symmetry characteristics of surface-mounted components, combines the auxiliary views generated by the image transformation unit (ITU), and adopts the collaborative design of the convolutional neural network backbone and the regression sub-network. This method extracts the rotation angle and symmetric center position of the surface-mounted components through the angle regression sub-network and the center regression sub-network respectively, and optimizes them through the consistency loss function, so as to achieve efficient and accurate alignment detection.

[0078] (2) This method uses the self-supervised learning method, does not rely on manually labeled data, and generates auxiliary views by using the rotational symmetry and reflection symmetry characteristics, significantly reducing the cost of data annotation and improving the generalization ability of the model.

[0079] (3) The network structure is simple and efficient, can perform fast alignment detection, and the inference speed is greatly improved, meeting the high-efficiency requirements of real-time detection in the actual industrial environment.

[0080] (4) The model is optimized by using the angle, center, and bounding box scale consistency losses, making the prediction of the rotation angle, symmetric center position, and target bounding box size of the surface-mounted components more accurate, and enhancing the robustness of the model under complex production conditions.

[0081] (5) This method has significant advantages in the alignment detection of surface-mounted components on PCB, and can be widely applied to the quality control and automatic optical inspection fields of electronic products, promoting the further development of automated inspection technology. Through the above innovative points, the present invention can provide an efficient, accurate and economical PCB alignment detection solution, which is suitable for the real-time detection requirements in the actual production environment and has broad application prospects. Brief Description of the Drawings

[0082] Figure 1 is the overall flowchart of the PCB alignment detection method based on self-supervised symmetry perception provided by the present invention;

[0083] Figure 2 is the network structure diagram of the PCB alignment detection method based on self-supervised symmetry perception provided by the present invention. Detailed Description of the Invention

[0084] As Figure 1 shown, the present invention proposes a PCB alignment detection method based on self-supervised symmetry perception, and the method includes the following steps:

[0085] Step 1, collect a dataset of surface-mounted components on a printed circuit board (PCB) with central symmetry and axial symmetry characteristics, and preprocess the dataset, including data augmentation operations such as random horizontal flipping, vertical flipping, rotation, and scaling;

[0086] Step 2, analyze the symmetry of the surface-mounted components on the PCB, and based on the characteristics of the symmetry center and symmetry axis, construct an image transformation unit (ITU) to generate auxiliary views for training, including 180° rotation, random translation, vertical flipping, random rotation, and random scaling views;

[0087] Step 3, design a self-supervised symmetry perception network, which consists of a convolutional neural network backbone and an angle regression branch, a center regression branch, and a bounding box regression branch. Input the data of the original dataset in Step 1 and the data after image transformation in Step 2 into the network. The angle regression branch is used to predict the symmetry axis angle of the surface-mounted component, the center regression branch is used to predict the symmetry center position, and the bounding box regression branch is used to predict the target box size of the surface-mounted component;

[0088] Step 4, construct a consistency loss function including angle consistency loss, center consistency loss, and bounding box consistency loss to guide model training;

[0089] Step 5, use the trained model for PCB alignment detection.

[0090] Furthermore, the specific method of Step 1 is as follows:

[0091] The dataset of surface-mounted components on the PCB has undergone the following processing:

[0092] Step 1.1: Extract the surface mount component images from the original image data, and perform normalization processing. The resolution of each image is uniformly adjusted to 128×128 pixels, and the data is enhanced by image enhancement, including rotation, scaling, and color transformation processing;

[0093] Step 1.2: All images are calibrated by an automatic optical inspection (AOI) device during the acquisition process, and the images are denoised and edge-detected;

[0094] Step 1.3: Divide the dataset into a training set, a validation set, and a test set according to 60:20:20.

[0095] Furthermore, the specific method of step 2 is as follows:

[0096] Step 2.1: Obtain the auxiliary views related to the symmetric center point during the training process: Generate the views of the training center through 180° rotation and random translation transformation. The 180° rotation rotates the image 180° around the origin, and the random translation translates the original image through a random translation vector Δ, and set the translation threshold T tsl Control the range of the random translation vector Δ, and the specific calculation formula is as follows:

[0097]

[0098] where Δ xt and Δ yr are random numbers between -1 and 1, and ω and h are the width and height of the input image respectively;

[0099] Step 2.2: Obtain the auxiliary views related to the symmetry axis angle during the training process: Generate the views of the training angle through vertical flipping and random rotation transformation. The vertical flipping flips the image up and down, and the random rotation rotates the original image counterclockwise by a random rotation angle R. The random rotation angle is controlled by the rotation threshold T rot The calculation formula for the rotation angle R is as follows:

[0100] R = T rot ×R r ×90°

[0101] where R r is a random number between -1 and 1, and T rot controls the range of random rotation;

[0102] Step 2.3, obtaining the auxiliary view related to the bounding box: Perform a scaling transformation on the original image with a random scaling ratio. Let the original image size be \(W_0\times H_0\), where \(W_0\) and \(H_0\) represent the width and height of the original image respectively. Perform a scaling transformation on the image with a random scaling ratio \(S\in[0.8,1.2]\) to obtain the new image size \(W'\times H'\), where:

[0103] \(W' = W_0\times S\), \(H' = H_0\times S\).

[0104] Furthermore, the specific method of step 3 is as follows:

[0105] Step 3.1, Convolutional Neural Network stage

[0106] The Convolutional Neural Network stage includes a CNN backbone network and three sub-networks. The CNN backbone network uses a pre-trained ResNet18 model, and its classification layer is replaced by an angle regression sub-network, a center point regression sub-network, and a bounding box regression sub-network. The input image is the pre-processed PCB surface mount component dataset image, and the output is the output values of the three sub-networks;

[0107] Assume there are three sub-networks: \(f\) ang (I), \(f\) cen (I) and \(f\) bbox (I), which are used to output the axis of symmetry angle \(\theta\), the center of symmetry position \((x,y)\), and the bounding box size \((\omega,h)\) respectively, and are defined as follows:

[0108] \(f\) ang (I)=\(\theta\)

[0109] \(f\) cen (I)=(x,y)

[0110] \(f\) bbox (I)=(\omega,h)

[0111] Among them, the center of the image is set as the coordinate origin \((0,0)\);

[0112] Step 3.2, Angle Regression Sub-network

[0113] The Angle Regression Sub-network is used to predict the axis of symmetry angle \(\theta\) of the PCB in the image. The input of this sub-network is the image after angle branch transformation and the original image. The input passes through a 1×1 convolution, a sigmoid activation function, and a phase shift encoder, and outputs a predicted axis of symmetry angle value \(\theta\);

[0114] Step 3.3, Center Point Regression Sub-network

[0115] The center point regression sub-network is used to predict the symmetric center position (x, y) of the PCB in the image. The input of this sub-network is the image after the center point branch transformation and the original image. The input passes through a 1×1 convolution and a sigmoid activation function, and outputs a predicted symmetric center position (x, y).

[0116] Step 3.4, the bounding box regression sub-network

[0117] The bounding box regression sub-network is used to predict the bounding box size (ω, h) of the PCB components in the image. The input of this sub-network is the image after the bounding box branch transformation and the original image. The input passes through a 1×1 convolution and a sigmoid activation function, and outputs a predicted bounding box size (ω, h).

[0118] Furthermore, the specific method of step 4 is as follows:

[0119] Step 4.1, the consistency loss function

[0120] 1. Angle consistency loss

[0121] The angle consistency loss is used to optimize the angle prediction under the rotation symmetry characteristic, and is defined as follows:

[0122] L ang =α×L flp +L rot

[0123] Among them, α is a hyperparameter that controls the weight of the sub-loss, and L flp is the angle loss obtained through the vertical flip transformation, and is defined as:

[0124] L flp =L(f ang (I flp ), -θ)

[0125] Among them, θ is the target symmetry axis angle, and L rot is the angle loss obtained through the rotation transformation, and is defined as:

[0126] L rot =L(f ang (I rot ), -θ, R)

[0127] Among them, i flp and i rot respectively represent the flipped and rotated images generated by the image transformation unit, R is the random rotation angle, and l is the loss function;

[0128] 2. Center consistency loss

[0129] The center consistency loss is used to optimize the center position prediction under translational symmetry and is defined as follows:

[0130] l cen = β × L rot180 + L tsl

[0131] where β is a hyperparameter that controls the weight of the sub-loss, and L rot180 is the center loss obtained through a 180° rotation transformation and is defined as:

[0132] L rot180 = L(f cen (I rot180 ), -p)

[0133] L tsl is the center loss obtained through a random translation transformation and is defined as:

[0134] L tsl = L(f cen (I tsl ), p, Δ)

[0135] where I rot180 and I tsl respectively represent the 180° rotation and translation images generated by the image transformation unit, p is the target symmetry center, Δ is the random translation vector, and L is the loss function;

[0136] 3. Bounding box consistency loss

[0137] The bounding box consistency loss is used to optimize the accuracy of the bounding box prediction under transformations such as rotation, flipping, and scaling, and is defined as follows:

[0138] L bbox = γ × L flp + L rot + L sca

[0139] where γ is a hyperparameter that controls the weight of the sub-loss, and L flp is the bounding box loss obtained through a vertical flipping transformation and is defined as:

[0140] L flp = L(f bbox (I flp ), -b)

[0141] where b = (ω, h) is the target bounding box, and L rot is the bounding box loss obtained through a rotation transformation and is defined as:

[0142] L rot = L(f bbox (I rot), -b, R)

[0143] where R is the random rotation angle, and L sca is the bounding box loss obtained through scaling transformation, defined as:

[0144] L sca = L(f bbox (I sca ), -b, S)

[0145] where I flp , I rot and I sca respectively represent the flipped, rotated, and scaled images generated by the image transformation unit, L is the scaling factor, and L is the loss function;

[0146] Step 4.2, Loss function optimization

[0147] Combining the angle consistency loss and the center consistency loss, the total loss function formula is defined as:

[0148] L total = ω1·L ang + ω2·L cen + ω3·L sca

[0149] where ω1, ω2, and ω3 are the weight coefficients of the angle, center point, and bounding box losses respectively, which are hyperparameters used to balance the contributions of the three sub-losses. During the training process, the network parameters are optimized through backpropagation to gradually minimize the total loss function.

[0150] Furthermore, the specific method of step 5 is as follows:

[0151] Step 5.1, The input PCB image generates auxiliary views;

[0152] Step 5.2, The generated image and the original image are input into the CNN stage, and are respectively input into the angle regression sub-network, the center regression sub-network, and the bounding box regression sub-network, and the symmetry axis angle of the target symmetric center coordinates and the target bounding box size

[0153] Step 5.3, The model calculates the rotation detection box of the target based on the prediction results of the symmetry axis angle symmetric center coordinates and the target bounding box size .

[0154] Implementation case

[0155] This implementation case uses Python 3.7 and the PyTorch deep learning framework as the experimental platform, and uses a GeForce RTX 3070 graphics card with 8GB of video memory for training. The computing infrastructure is CPU: Intel i5-10600KF, GPU: Nvidia RTX3070, operating system: Ubuntu 20.04.2 LTS, PyTorch: 1.13.0+cu117, CUDA: 11.7. To verify the accuracy and generalization ability of the model, the dataset provides the location information of each target, adopts the annotation method in DOTA format, and the dataset is divided into a training set (accounting for 60% of the total dataset), a validation set (accounting for 20% of the total dataset), and a test set (accounting for 20% of the total dataset). In the experiment, the main hyperparameter settings for hyperparameter training are: the maximum number of training iterations is 300; batch_size: 64; optimizer: AdamW; initial learning rate: 0.00001; weight_decay: 0.0001; test metrics: absolute error (AE), inference time, number of parameters, mean average precision (mAP). The following further elaborates on the present invention for the above example. The process of the present invention includes:

[0156] Step 1: The PCB surface-mounted component dataset has been processed as follows:

[0157] Step 1.1: Extract surface-mounted component images from the original image data and perform normalization processing on them. The resolution of each image is uniformly adjusted to 128×128 pixels. The data has undergone necessary image enhancement, including rotation, scaling, and color transformation, etc.

[0158] Step 1.2: All images are calibrated by an automatic optical inspection (AOI) device during the acquisition process, and the images are denoised and edge-detected.

[0159] Step 1.3: Divide the dataset into a training set, a validation set, and a test set according to 60:20:20.

[0160] Step 2:

[0161] Step 2.1: To obtain auxiliary views related to the symmetric center point during the training process, ITU generates views of the training center through 180° rotation and random translation transformation. The 180° rotation rotates the image 180° around the origin. The random translation translates the original image through a random translation vector Δ. To prevent the loss of effective features, a translation threshold T is set tsl to control the range of the random translation vector Δ, and the specific calculation formula is as follows:

[0162]

[0163] where, Δ xr and Δ yr are random numbers between -1 and 1, and ω and h are the width and height of the input image respectively;

[0164] Step 2.2: To obtain the auxiliary views related to the axis of symmetry angle during the training process, ITU generates views of the training angles through vertical flipping and random rotation transformations. Vertical flipping flips the image up and down, and random rotation rotates the original image counterclockwise by a random rotation angle R. The random rotation angle is controlled by the rotation threshold T rot and is calculated as follows:

[0166] R = T rot × R r × 90°

[0167] where, R r is a random number between -1 and 1, and T rot controls the range of random rotation.

[0168] Step 2.3: To obtain the auxiliary views related to the bounding box, ITU performs a scaling transformation on the original picture with a random scaling ratio (0.8 - 1.2). Let the original image size be W0×H0, where W0 and H0 represent the width and height of the original image respectively. The image is scaled through a random scaling ratio S ∈ [0.8, 1.2] to obtain a new image size W′×H′, where:

[0169] W′ = W0×S, H′ = H0×S

[0170] Step 3:

[0171] Convolutional Neural Network Stage

[0172] To extract the axis of symmetry angle, symmetric center position, and bounding box size of the surface mount component, the convolutional neural network stage includes a CNN backbone network and three sub-networks. The CNN backbone network uses a pre-trained ResNet18 model, and its classification layer is replaced by an angle regression sub-network, a center point regression sub-network, and a bounding box regression sub-network. The input image is the preprocessed PCB surface mount component dataset image, and the output is the output values of the three sub-networks.

[0173] Assume there are three sub-networks: f ang (I), f cen (I), and f bbox (I), which are used to output the axis of symmetry angle θ, the symmetric center position (x, y), and the bounding box size (ω, h) respectively, and are defined as follows:

[0174] f ang (I) = θ

[0175] f cen (I) = (x, y)

[0176] f bbox (I) = (ω, h)

[0177] Wherein, the center of the image is set as the coordinate origin (0, 0). The specific design is as follows:

[0178] Step 3.2: Angle regression sub-network

[0179] The angle regression sub-network is used to predict the symmetry axis angle θ of the PCB in the image. The input of this sub-network is the picture after the ITU angle branch transformation and the original picture. The input passes through a 1×1 convolution, a sigmoid activation function, and a phase shifting coder (PSC), and outputs a predicted symmetry axis angle value θ.

[0180] Step 3.3: Center point regression sub-network

[0181] The center point regression sub-network is used to predict the symmetry center position (x, y) of the PCB in the image. The input of this sub-network is the picture after the ITU center point branch transformation and the original picture. The input passes through a 1×1 convolution and a sigmoid activation function, and outputs a predicted symmetry center position (x, y).

[0182] Step 3.4: Bounding box regression sub-network

[0183] The bounding box regression sub-network is used to predict the bounding box size (ω, h) of the PCB components in the image. The input of this sub-network is the picture after the ITU bounding box branch transformation and the original picture. The input passes through a 1×1 convolution and a sigmoid activation function, and outputs a predicted bounding box size (ω, h).

[0184] Step 4:

[0185] Step 4.1: Consistency loss function

[0186] To optimize the learning ability of the model, this method designs a consistency loss function, which includes three sub-parts: angle consistency loss, center point consistency loss, and bounding box consistency loss.

[0187] 1. Angle consistency loss

[0188] The angle consistency loss is used to optimize the angle prediction under rotational symmetry characteristics, and is defined as follows:

[0189] L ang = α × L flp + L rot

[0190] Among them, α is a hyperparameter for controlling the weight of the sub-loss, and L flp is the angle loss obtained by vertical flip transformation, defined as:

[0191] L flp = L(f ang (I flp ), -θ)

[0192] Among them, θ is the target symmetry axis angle, and L rot is the angle loss obtained by rotation transformation, defined as:

[0193] L rot = L(f ang (I rot ), -θ, R)

[0194] Among them, i flp and i rot respectively represent the flipped and rotated images generated by the image transformation unit, R is the random rotation angle, and l is the loss function.

[0195] 2. Center consistency loss

[0196] The center consistency loss is used to optimize the center position prediction under translational symmetry characteristics, and is defined as follows:

[0197] l cen = β × L rot180 + L tsl

[0198] Among them, β is a hyperparameter for controlling the weight of the sub-loss, and L rot180 is the center loss obtained by 180° rotation transformation, defined as:

[0199] L rot180 = L(f cen (I rot180 ), -p)

[0200] L tsl is the center loss obtained by random translation transformation, defined as:

[0201] L tsl = L(f cen (I tsl ), p, Δ)

[0202] Among them, I rot180 and I tsl respectively represent the 180° rotation and translation images generated by the image transformation unit, p is the target symmetry center, Δ is the random translation vector, and L is the loss function.

[0203] 3. Bounding Box Consistency Loss

[0204] The bounding box consistency loss is used to optimize the accuracy of the model's bounding box prediction under transformations such as rotation, flipping, and scaling, and is defined as follows:

[0205] L bbox = γ × L flp + L rot + L sca

[0206] where γ is a hyperparameter that controls the weight of the sub-loss, and L flp is the bounding box loss obtained through the vertical flipping transformation, defined as:

[0207] L flp = L(f bbox (I flp ), -b)

[0208] where b = (ω, h) is the target bounding box, and L rot is the bounding box loss obtained through the rotation transformation, defined as:

[0209] L rot = L(f bbox (I rot ), -b, R)

[0210] where R is the random rotation angle, and L sca is the bounding box loss obtained through the scaling transformation, defined as:

[0211] L sca = L(f bbox (I sca ), -b, S)

[0212] where I flp 、I rot and I sca represent the flipped, rotated, and scaled images generated by the image transformation unit respectively, S is the scaling factor, and L is the loss function.

[0213] Step 4.3: Loss Function Optimization

[0214] Combining the angle consistency loss and the center consistency loss, the total loss function formula is defined as:

[0215] L total = ω1 · L ang + ω2 · L cen + ω3 · L sca

[0216] Among them, ω1, ω2, and ω3 are the weight coefficients of the angle, center point, and bounding box losses respectively, which are hyperparameters used to balance the contributions of the three sub-losses, and the hyperparameters are determined through ablation experiments. During the training process, the network parameters are optimized through backpropagation to gradually minimize the total loss function.

[0217] Step 5:

[0218] Step 5.1: The input PCB image generates an auxiliary view through ITU.

[0219] Step 5.2: The generated image and the original image are input into the CNN stage, and are respectively input into the angle regression sub-network, center regression sub-network, and bounding box regression sub-network, and the axis angle of symmetry of the target is output Symmetric center coordinates and the size of the target bounding box

[0220] Step 5.3: The model can calculate the rotation detection box of the target through the prediction results of the axis angle of symmetry Symmetric center coordinates and the size of the target bounding box of the target.

Claims

1. A PCB alignment detection method based on self-supervised symmetry perception, characterized in that, The method includes the following steps: Step 1, collect a dataset of surface-mounted components on a printed circuit board (PCB) with central symmetry and axial symmetry characteristics, and preprocess the dataset, including data augmentation operations such as random horizontal flipping, vertical flipping, rotation, and scaling; Step 2, analyze the symmetry of the surface-mounted components on the PCB. Based on the characteristics of the symmetry center and the axis of symmetry, construct an image transformation unit (ITU) to generate auxiliary views for training, including 180° rotation, random translation, vertical flipping, random rotation, and random scaling views; Step 3, design a self-supervised symmetry-aware network. The network consists of a convolutional neural network backbone and an angle regression branch, a center regression branch, and a bounding box regression branch. Input the data from the original dataset in Step 1 and the data after image transformation in Step 2 into the network. The angle regression branch is used to predict the angle of the axis of symmetry of the surface-mounted component, the center regression branch is used to predict the position of the symmetry center, and the bounding box regression branch is used to predict the size of the target box of the surface-mounted component; Step 4, construct a consistency loss function including angle consistency loss, center consistency loss, and bounding box consistency loss to guide model training; Step 5, use the trained model for PCB alignment detection.

2. The PCB alignment detection method based on self-supervised symmetry perception according to claim 1, wherein, The specific method of Step 1 is as follows: The dataset of the surface-mounted components on the PCB has undergone the following processing: Step 1.1, extract the images of the surface-mounted components from the original image data, and perform normalization processing on them. The resolution of each image is uniformly adjusted to 128×128 pixels, and the data undergoes image enhancement, including rotation, scaling, and color transformation processing; Step 1.2, all images are calibrated by an automatic optical inspection (AOI) device during the acquisition process, and the images are denoised and edge-detected; Step 1.3, divide the dataset into a training set, a validation set, and a test set according to 60:20:

20.

3. A PCB alignment detection method based on self-supervised symmetry perception according to claim 1, characterized in that The specific method of Step 2 is as follows: Step 2.1, obtain the auxiliary view related to the symmetry center point during the training process: generate the view of the training center through 180° rotation and random translation transformation. The 180° rotation rotates the image 180° around the origin, and the random translation translates the original image through the random translation vector Δ. Set the translation threshold T tsl Control the range of the random translation vector Δ, and the specific calculation formula is as follows: where Δ xr and Δ yr are random numbers between -1 and 1, and ω and h are the width and height of the input image respectively; Step 2.2, obtaining an auxiliary view related to the axis of symmetry angle during the training process: Views of the training angles are generated through vertical flipping and random rotation transformation. Vertical flipping flips the image vertically, and random rotation rotates the original image counterclockwise by a random rotation angle R. The random rotation angle is controlled by a rotation threshold T rot The calculation formula for the rotation angle R is as follows: R = T rot × R r × 90° Among them, R r is a random number between -1 and 1, and T rot controls the range of random rotation; Step 2.3, obtain the auxiliary views related to the bounding box: perform a scaling transformation on the original picture with a random scaling ratio. Let the original image size be W0×H0, where W0 and H0 represent the width and height of the original image respectively. Scale the image with a random scaling ratio S∈[0.8,1.2] to obtain a new image size W′×H′, where: W′ = W0×S, H′ = H0×S.

4. A PCB alignment detection method based on self-supervised symmetry perception according to claim 1, characterized in that, The specific method of Step 3 is as follows: Step 3.1, convolutional neural network stage The convolutional neural network stage includes a CNN backbone network and three sub-networks. The CNN backbone network uses a pre-trained ResNet18 model, and its classification layer is replaced by an angle regression sub-network, a center point regression sub-network, and a bounding box regression sub-network. The input image is the image of the dataset of surface-mounted components on the PCB after preprocessing, and the output is the output values of the three sub-networks; Suppose there are three sub-networks: f ang (I), f cen (I) and f bbox (I), which are used to output the symmetry axis angle θ, the symmetry center position (x, y) and the bounding box size (ω, h) respectively, and are defined as follows: f ang (I) = θ f cen (I) = (x, y) f bbox (I) = (ω, h) Among them, the center of the image is set as the coordinate origin (0,0); Step 3.2, angle regression sub-network The angle regression sub-network is used to predict the symmetry axis angle θ of the PCB in the image. The input of this sub-network is the image after the angle branch transformation and the original image. The input passes through a 1×1 convolution, a sigmoid activation function, and a phase shift encoder, and outputs a predicted symmetry axis angle value θ; Step 3.3, the center point regression sub-network The center point regression sub-network is used to predict the symmetry center position (x, y) of the PCB in the image. The input of this sub-network is the image after the center point branch transformation and the original image. The input passes through a 1×1 convolution and a sigmoid activation function, and outputs a predicted symmetry center position (x, y); Step 3.4, the bounding box regression sub-network The bounding box regression sub-network is used to predict the bounding box size (ω, h) of the PCB components in the image. The input of this sub-network is the image after the bounding box branch transformation and the original image. The input passes through a 1×1 convolution and a sigmoid activation function, and outputs a predicted bounding box size (ω, h).

5. A PCB alignment detection method based on self-supervised symmetry perception according to claim 1, characterized in that, The specific method of the said Step 4 is as follows: Step 4.1, the consistency loss function 1. Angle consistency loss The angle consistency loss is used to optimize the angle prediction under the rotational symmetry characteristic, and is defined as follows: L ang = α × L flp + L rot where α is a hyperparameter for controlling the weight of the sub-loss, and L flp is the angular loss obtained through vertical flip transformation, defined as: L flp = L(f ang (I flp ), -θ) where θ is the target axis of symmetry angle, and L rot is the angle loss obtained by rotation transformation, defined as: L rot = L(f ang (I rot ), -θ, R) Among them, I flp and I rot respectively represent the flipped and rotated images generated by the image transformation unit, R is the random rotation angle, and L is the loss function; 2. Center consistency loss The center consistency loss is used to optimize the center position prediction under the translational symmetry characteristic, and is defined as follows: L cen = β × L rot180 + L tsl where β is a hyperparameter for controlling the weight of the sub-loss, and L rot180 is the center loss obtained by a 180° rotation transformation, defined as: L rot180 = L(f cen (I rot180 ), -p) L tsl The center loss obtained by random translation transformation is defined as: L tsl = L(f cen (I tsl ), p, Δ) where, I rot180 and I tsl respectively represent the 180° rotated and translated images generated by the image transformation unit, p is the target symmetry center, Δ is the random translation vector, and L is the loss function; 3. Bounding box consistency loss The bounding box consistency loss is used to optimize the bounding box prediction accuracy of the model under transformations such as rotation, flipping, and scaling, and is defined as follows: L bbox = γ × L flp + L rot + L sca where γ is a hyperparameter controlling the weight of the sub-loss, and L flp defined as the bounding box loss obtained by vertical flipping transformation is: L flp = L(f bbox (I flp ), -b) where \(b = (\omega, h)\) is the target bounding box, and \(L\) rot is the bounding box loss obtained by rotation transformation, defined as: L rot = L(f bbox (I rot ), -b, R) where R is the random rotation angle, and L sca is the bounding box loss obtained through the scaling transformation, defined as: L sca = L(f bbox (I sca ), -b, S) Among them, I flp , I rot and I sca respectively represent the flipped, rotated, and scaled images generated by the image transformation unit, L is the scaling factor, and L is the loss function; Step 4.2, loss function optimization Combining the angle consistency loss and the center consistency loss, the total loss function formula is defined as: L total = ω1·L ang + ω2·L cen + ω3·L sca Where ω1, ω2, and ω3 are the weight coefficients of the angle, center point, and bounding box losses respectively, which are hyperparameters used to balance the contributions of the three sub-losses. During the training process, the network parameters are optimized through backpropagation to gradually minimize the total loss function.

6. The PCB alignment detection method based on self-supervised symmetry perception according to claim 1, characterized in that The specific method of the said Step 5 is as follows: Step 5.1, the input PCB image generates auxiliary views; Step 5.2, in the stage of generating the image and inputting the original image into the CNN, input them into the angle regression sub-network, the center regression sub-network, and the bounding box regression sub-network respectively, and output the axis symmetry angle of the target Symmetric center coordinates and the target bounding box size Step 5.3, the model calculates the rotated detection box of the target through the prediction results of the axis of symmetry angle symmetric center coordinates and the target bounding box size ​

Images