Rigid circuit board registration model and method based on depth corner feature matching
Through the deep angle feature matching neural network model, the lack of accuracy and flexibility of rigid circuit board registration technology is solved, and efficient and robust registration effect is achieved, overcoming the low computational efficiency and feature extraction calculation of traditional methods Poor generalization.
Patent Information
- Application Number
- CN202410996344.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-08
AI Technical Summary
The existing rigid circuit board registration technology has insufficient accuracy and flexibility, especially the camera calibration technology is difficult to ensure high accuracy and inflexibility. The traditional image registration method is inefficient in calculation efficiency and is not suitable for nonlinear transformation. Feature-based The registration method is greatly affected by resolution and lighting, and the transformation domain-based registration method is not effective in small overlap areas.
Adopt a neural network model based on deep angle point feature matching, and combines cross attention and self-attention to realize the end To-end corner feature matching suppresses noise interference and improves matching accuracy.
Improve registration accuracy and flexibility, overcome the low computational efficiency of traditional methods and poor generalization of feature extraction operators, and can achieve efficient registration in non-rigid body transformation and diversified image scenarios, enhancing the robustness of the model and controllability.
Smart Images

Figure CN118918155A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of circuit board registration, and in particular to a rigid circuit board registration model based on depth corner point feature matching. Background Art
[0002] PCB registration refers to a technology that matches PCB and PCB template. Accurately registering PCB is essential for component installation, aperture measurement, edge cutting, defect monitoring, etc. of PCB on the production line. Rigid PCB is a PCB with hard material, not easy to bend, and stable shape. There are two main mainstream rigid circuit board registration technologies at present. One is based on camera calibration technology, which completes image registration of circuit board images and their template images taken at fixed positions (such as fixed by slots) through known translation, rotation, and scaling relationships; the other is traditional image registration technology, such as grayscale-based, feature-based, and transform domain-based registration methods. The grayscale-based registration method generally estimates the matching score by sliding the image on the template, enumerating all situations, and estimating the similarity of each sliding position; the feature-based registration method mainly extracts features such as points, lines, and surfaces to obtain all matching feature pairs, and estimates the homography matrix of the image to template transformation based on the feature pairs to complete image registration; the transform domain-based registration method generally converts the spatial domain image into the frequency domain image through mathematical transformation, and calculates the translation, rotation, and scaling between the image in the frequency domain and the template to complete image registration.
[0003] About camera calibration technology: Camera calibration is the process of converting from the world coordinate system to the camera coordinate system, and then from the camera coordinate system to the image coordinate system, that is, the process of finding the final projection matrix. The world coordinate system is a user-defined three-dimensional coordinate system used to describe the position of the target in the real world; the camera coordinate system is a coordinate system established on the camera, used to describe the target position from the camera's perspective; the image coordinate system is used to describe the projection relationship from the camera coordinate system to the image coordinate system. When the circuit board is photographed at a fixed shooting position, the position of the image in the world coordinate system is determined, and only the camera coordinate system and the image coordinate system need to be considered. At this time, the camera calibration information can be used to solve the translation matrix, rotation matrix and scaling ratio of the distortion, and the effect of correcting the image distortion can be achieved through inverse transformation, thereby completing the registration of the image and the template. In order to obtain higher registration accuracy, the registration method based on camera calibration technology requires the accurate placement of the circuit board. In reality, this requirement is difficult to achieve. On the one hand, each circuit board has a different shape, and it is difficult to design a good placement. On the other hand, in order to meet the production requirements on the assembly line, the circuit board stays in a fixed position for a very short time, and it is difficult to ensure the correct placement. Excessive pursuit of precise placement will affect the production efficiency of the assembly line. The registration method based on camera calibration technology also has a fatal problem, which is that it is difficult to ensure a high registration accuracy. Slight jitters of the camera or the shooting platform will cause ghosting in the registered image.
[0004] About image registration technology: Traditional image registration mainly includes grayscale-based, feature-based and transform domain-based registration methods.
[0005] Grayscale information registration method is to directly align pixels based on the grayscale information of the image. The main idea of this method is to directly minimize the difference in grayscale information of the image. Grayscale-based registration methods generally grayscale the image and template first, and then slide the grayscale image to align it with the template. When the grayscale information difference in the overlapping area is the smallest and below a certain threshold, it means that the image and template are matched. According to the different judgment criteria of grayscale information difference, it can usually be divided into cross-correlation method, mutual information method and error square sum method.
[0006] The feature registration method is based on the alignment of the key features in the image. The main idea of this method is to find the key features for image and template matching. The feature-based registration method needs to extract the image and template features first, generate feature descriptors, and then complete the matching through the similarity of the feature descriptors. According to the different artificially designed feature extraction operators, it can be divided into point features, line or edge features, surface or texture features.
[0007] The transform domain registration method is a registration method in the frequency domain. The main idea of this method is to use polar coordinates and cross-power spectrum to calculate the translation, rotation and scaling relationship between the image and the template in the frequency domain. The registration method based on the transform domain can overcome correlation noise and frequency noise, and can greatly reduce the impact of geometric distortion on matching performance. The calculation speed is relatively fast. The mainstream algorithms include registration methods based on Fourier-Mellin transform, wavelet transform, etc.
[0008] However, traditional image registration technologies also have their own shortcomings.
[0009] The registration method based on grayscale information aligns the image and the template by translation, rotation, etc., and violently solves the best match. The calculation efficiency is low and it cannot be directly used to correct the nonlinear transformation of the image. In addition, the registration method based on grayscale information requires that the illumination, style, etc. between the image and the template remain small, so the scene applicability is poor.
[0010] The feature-based registration technology needs to extract the features of the corners, edges or textures of the image. These features are shallow features and are greatly affected by resolution, contrast, etc. The registration effect is closely related to the characteristics of the image itself and the operator used to extract the features. The registration based on point features requires obvious corner features, and the registration based on line and edge features requires clear lines and edges. If the generalization of the feature extraction operator is poor, the registration success rate will also be low.
[0011] The registration method based on transform domain is only applicable to image transformations that have corresponding definitions in Fourier transform, and it does not work well when the overlap between the two images is not large enough. Summary of the invention
[0012] In order to solve the above technical problems, the present invention discloses a rigid circuit board registration model and method based on deep corner feature matching. The technical solution of the present invention is implemented as follows:
[0013] A rigid circuit board registration model based on deep corner feature matching, comprising a feature extraction module, a position encoding module, a coarse-grained feature alignment module, a matching point attention module, a fine-grained feature alignment module and an output module;
[0014] The feature extraction module is used to extract pixel features in the circuit board template and the image;
[0015] The position encoding module is used to encode the position of the pixel feature output by the feature extraction module, so that the pixel feature contains position information;
[0016] The coarse-grained feature alignment module is used to align the coarse-grained features of the encoded image with the coarse-grained features of the template and output a coarse-grained feature matching matrix;
[0017] The matching point attention module is used to screen out coarse-grained features with high matching degree in the feature matching matrix, and retain the features on the high-resolution feature map at the corresponding position based on the scale scaling relationship between the coarse-grained features and the fine-grained features;
[0018] The fine-grained feature alignment module performs feature alignment on the fine-grained features of the encoded image and the fine-grained features of the template and outputs a fine-grained feature matching matrix;
[0019] The output module transforms the fine-grained feature matching matrix into a two-dimensional probability matrix and outputs it.
[0020] Preferably, a self-attention module is also included;
[0021] The self-attention module is used to learn the associations between pixels within the feature map and adjust the feature response values based on these associations.
[0022] A rigid circuit board registration method based on deep corner feature matching includes the following steps:
[0023] S1, establishing a neural network model; the model includes a feature extraction module, a position encoding module, a coarse-grained feature alignment module, a matching point attention module, a fine-grained feature alignment module and an output module;
[0024] S2, input the circuit board template and image into the neural network model;
[0025] S3, feature extraction module extracts features of template and image respectively;
[0026] S4, the position encoding module performs coarse-grained feature encoding and fine-grained feature encoding on the coordinate positions of the extracted image features and template features respectively;
[0027] S4, a coarse-grained feature alignment module performs feature alignment on the coarse-grained features of the encoded image and the coarse-grained features of the module and outputs a feature matching matrix;
[0028] S5, matching point attention: The matching point attention module selects the coarse-grained features with high matching degree in the feature matching matrix, and retains the features on the high-resolution feature map at the corresponding position based on the scaling relationship between the coarse-grained features and the fine-grained features;
[0029] S6, fine-grained feature alignment. The fine-grained feature alignment module retains the fine-grained features of the area corresponding to the coarse-grained features with higher scores in the coarse-grained feature matching scores under the guidance of the matching point attention module, and aligns the fine-grained features of the retained image with the fine-grained features of the template, and finally outputs the fine-grained feature matching matrix.
[0030] S7, the output module transforms the fine-grained feature matching matrix into a two-dimensional probability matrix and outputs it.
[0031] Preferably, the S2 includes:
[0032] S2.1, converting two-dimensional image features into one-dimensional sequence features;
[0033] S2.2, arrange each pixel in the one-dimensional sequence feature in order;
[0034] S2.3, performing trigonometric transformation;
[0035] S2.4, adding the transformed arrangement sequence number to the one-dimensional sequence feature in S2.1 to obtain a feature sequence with position information.
[0036] Preferably, it also includes:
[0037] The self-attention module obtains the feature sequence with position information and outputs the feature sequence after self-attention activation to the coarse-grained feature alignment module;
[0038] The coarse-grained feature alignment module performs feature alignment on the coarse-grained features of the image and the coarse-grained features of the module based on the feature sequence after self-attention activation and outputs feature matching data.
[0039] Preferably, in S2, the feature extraction module uses a resnet network and an FPN network as feature extraction networks.
[0040] Preferably, in S3, the position encoding module is Positional Encoding in the transformer model.
[0041] Preferably, the self-attention model is SelfAttention in the transformer model.
[0042] The advantages of the present invention are as follows:
[0043] The present invention innovatively combines deep learning technology with point feature-based registration methods to solve the registration problem in the industrial field. The present invention designs a neural network model that can learn the registration relationship of paired images end-to-end, combines cross attention and matching point attention, and enables the model to focus on the matchable areas in the coarse-grained, high-dimensional feature space, suppress noise interference, and adopt the divide-and-conquer idea to improve the matching accuracy from coarse to fine. The advantages of the present invention are as follows:
[0044] 1. Combine deep learning technology with point feature-based registration methods to solve the registration problem in the industrial field. Extract corner point features under the premise of matching. The entire learning process is end-to-end.
[0045] 2. Combining cross attention and matching point attention, the model focuses on the matchable areas in the coarse-grained, high-dimensional feature space and suppresses noise interference;
[0046] 3. Adopt the divide-and-conquer idea to improve the matching accuracy from coarse to fine. By learning the matching matrices of different resolution features twice, the coarse matching guides the fine matching and gradually optimizes.
[0047] 4. Use weakly supervised learning and use a large number of images of different styles to train the model and improve model accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0049] Figure 1 The invention discloses an algorithm structure of an implementation example of a rigid circuit board registration model based on deep corner feature matching. DETAILED DESCRIPTION
[0050] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention and the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0051] Example
[0052] In a specific embodiment, Figure 1 As shown, a rigid circuit board registration model based on deep corner feature matching includes a feature extraction module, a position encoding module, a coarse-grained feature alignment module, a matching point attention module and a fine-grained alignment module.
[0053] Among them, the feature extraction module is composed of a convolutional neural network model, which is used to extract the features of the template and the image. The template and the image share a convolutional neural network for extracting features. The extracted features include low-resolution features (coarse-grained features) and high-resolution features (fine-grained features).
[0054] The position encoding module is used to encode the coordinate position of the features extracted by the neural network model, that is, to position encode the position of each pixel of the feature map, so as to improve the model's perception of the position. In this embodiment, the position encoding module position encodes the low-resolution and high-resolution features of the template and the image respectively.
[0055] The core component of the coarse-grained feature alignment module (i.e., low-resolution feature alignment module) is the cross-attention network, whose input is the coarse-grained features of the template and the coarse-grained features of the image, and its output is the matching status of all pixels in the two features.
[0056] Since the computational cost of directly matching high-resolution features is large and noisy, the matching point attention module can reduce the computational cost of high-resolution feature learning. The core function of the matching point attention module is to filter out all coarse-grained features with high matching degree, avoiding redundant calculations, and then retain the features on the high-resolution feature map at the corresponding position according to the scale relationship between high-resolution and low-resolution features.
[0057] The core structure of the fine-grained feature alignment module (i.e., high-resolution alignment module) is also a cross-attention network. Its input is the fine-grained features of the template and the fine-grained features of the image, and its output is the feature matching matrix of the two features. The difference is that the fine-grained feature matching matrix will follow the guidance of the matching point attention module and retain the matching features of the areas corresponding to the features with higher scores in the coarse-grained feature matching scores, thereby achieving matching from coarse to fine.
[0058] After position encoding of the low-resolution and high-resolution features of the template and the image, this embodiment first aligns the low-resolution features and then the high-resolution features. The purpose of doing so is to reflect the idea of coarse-to-fine matching. Each pixel of the low-resolution feature represents the content of a larger area at the corresponding position on the original image, while each pixel of the high-resolution feature represents the content of a slightly smaller area at the corresponding position on the original image.
[0059] This embodiment combines deep learning and weakly supervised learning to solve the registration problem of rigid circuit boards, which can greatly improve the accuracy of the point feature-based registration method. Through weakly supervised learning, the generalization ability of the model can be greatly improved, and the manpower for labeling training samples can be saved.
[0060] The specific application of this embodiment is as follows:
[0061] 1. Feature extraction
[0062] This embodiment uses the resnet network and the FPN network as feature extraction networks. Resnet is the most commonly used convolutional neural network. Its core components include convolutional layers, normalization layers, and relu activation layers. By stacking multiple layers of core components, the purpose of deepening the network is achieved. In addition, in order to reduce the resolution of the feature, the feature scale is reduced by multiple downsampling. After the feature extraction module, the low-resolution features and high-resolution features of the template and image are output respectively. In this experiment, we use the features of 1 / 32 (reduced by 1 / 32 according to the side length) of the input image as low-resolution features, and the features of 1 / 8 of the input image as high-resolution features.
[0063] 2. Positional encoding
[0064] This embodiment adopts the Positional Encoding module in the transformer model as the positional encoding module of this method. The transformer model is a very important and classic algorithm model in the field of natural language processing, and the Positional Encoding module is a positional encoding layer of the model.
[0065] Before position coding, this embodiment first performs a feature flattening operation to convert the two-dimensional image features into one-dimensional sequence features, and then performs position coding on the one-dimensional sequence features. The specific method of position coding is to perform trigonometric function transformation (i.e., sine transformation and cosine transformation) on the arrangement number of each pixel on the feature map (because the feature map has been flattened, all pixels of the feature map are arranged in a row at this time). In order to ensure that the values of each position are different, the arrangement number will be divided by a larger value (this value is generally set to a value related to the number of feature channels) before the transformation, and then the trigonometric function transformation is performed. The role of position coding has three aspects: one is to mark the position of each pixel, the second is to adjust the geometric coordinates with large differences to a smaller value range, and the third is that the dot product of two position codes can reflect the distance between the two position codes, so it has good mathematical properties. The output of position coding is also a one-dimensional sequence, which will be added to the one-dimensional feature sequence of the input position coding to obtain a feature sequence with position information.
[0066] 3. Self-Attention
[0067] In this embodiment, the self-attention module is placed after the position encoding module. The purpose is to increase global self-attention in the input feature sequence and adjust the feature response through self-attention so that the model can better focus on the key features in the image.
[0068] The self-attention module converts the correlation between pixels in the feature map into an attention matrix. Multiplying the feature map with the attention matrix can adjust the feature response value.
[0069] This embodiment adopts the SelfAttention module in the transformer model as the self-attention module of this method. The core components of this module are the fully connected layer, the normalization layer and the dot product layer. The self-attention module can learn the association between any elements in the sequence, has the ability to observe the whole, and can improve the learning ability of circuit board features. The input of the self-attention module is: "feature sequence with position information"; the output is: "feature sequence after self-attention activation".
[0070] 4. Feature Alignment
[0071] Feature alignment is constructed using the Cross Attention in the transformer (including coarse-grained feature alignment and fine-grained feature alignment). Its core function is to learn the attention of one sequence to another sequence, and to find the correlation between all feature pixels in the two feature maps. When the correlation between the two pixels is large, the cross attention will output a larger response value. The core calculation units of the cross attention include the fully connected layer, the normalization layer, and the Einstein summation layer.
[0072] The input of feature alignment is the circuit board template feature sequence and the circuit board image feature sequence. After cross-attention, the two feature sequences can form a response relationship from template features to image features. After model training, the corner point features that match between the template and the image will have a higher response, which can intuitively display the matching relationship between the two images. The output of feature alignment is the response matrix (that is, the matching matrix) between the circuit board template and the image features. The coarse-grained feature alignment module outputs the response matrix between low-resolution features, and the fine-grained feature alignment module outputs the response matrix between high-resolution features.
[0073] 5. Matching point attention
[0074] The role of the matching point attention module is to provide a filtering method to screen out highly responsive feature pairs in the response matrix between low-resolution features. These feature pairs represent areas with high matching between high-dimensional, low-resolution circuit board templates and image features.
[0075] This embodiment retains the top few feature pairs with the response probability score (the feature pairs here are similar to pixel pairs. A feature pixel is not a pixel, but an aggregation of a group of pixels on the image. These feature pairs are template features and image features with high similarity) and projects them onto high-resolution features according to the scaling ratio of the feature map. The matching areas in the corresponding high-resolution features are found. In the fine matching process, the model will further accurately identify matching features on the high-resolution features around these high-matching areas, thereby refining the positions of matching points (in this patent, matching and registration have the same meaning). The matching point attention module can control the focus of the model, allowing the model to focus on the areas that can be matched under high-dimensional features and suppress noise interference.
[0076] 6. Output
[0077] After the feature alignment operation, a fine-grained feature matching matrix will be obtained. This feature matching matrix is a two-dimensional matrix with a high number of channels. This embodiment needs to transform this matrix into a two-dimensional probability matrix. The function of the output is to transform the feature matching matrix into a two-dimensional probability matrix. The output layer is a multi-layer perceptron network, which can expand and compress the features in the channel dimension, and finally transform the channel features into probabilities through nonlinear transformation.
[0078] 7. Training process
[0079] The data of this embodiment is divided into two parts, one part is the samples without manual annotation (it can be a non-circuit board image. We obtain the pixel mapping relationship between sample pairs and sample pairs through image perspective transformation and cropping. Each group of sample pairs is a training sample, one image sample is used as a template, and the other image sample is used as the image to be registered), and the other part is the sample pairs with manual annotation (i.e., circuit board template and image, sample pairs have manually annotated feature points, and the corresponding relationship between feature points). After the training samples are input into the neural network model, a two-dimensional probability matrix (including a high-resolution two-dimensional probability matrix and a low-resolution two-dimensional probability matrix) is output, and the cross entropy loss is calculated between the two-dimensional probability matrix and the annotated matching point pairs. The training process is the process of adjusting the parameters of the model in the direction of smaller loss until the loss converges.
[0080] 8. Reasoning process
[0081] In the present embodiment, when inferring, the input is the circuit board template and the image, and the output is the two-dimensional probability matrix between the high-resolution feature map of the template and the image. The probability matrix obtains a Boolean value according to a manually set threshold value. If the value is higher than the threshold value, it means that the two pixels on the high-resolution feature at the corresponding position of the two-dimensional probability matrix are matched with each other. According to the size scaling relationship between the high-resolution feature and the input image, all the mutually matching corner points in the input image can be obtained. According to the coordinates of all the mutually matching corner points, the homography matrix of the registration is obtained through the RANSAC parameter estimation algorithm. Through the homography matrix, the image is transformed in coordinates to achieve registration with the template.
[0082] In summary, the technical problems solved by this embodiment include:
[0083] 1. Flexibility issues. Both the camera calibration-based registration method and the three traditional registration methods (grayscale-based, feature-based, and transform domain-based registration methods) are not flexible enough. The former requires shooting at a fixed position and determining the camera relationship in advance, and the shooting conditions are harsh. Among the traditional image registration methods, the grayscale-based registration method is only applicable to rigid body transformations and cannot cope with non-rigid body transformations such as scaling and affine. The feature-based registration method requires that the feature extraction operator matches the most significant features of the image, otherwise the registration effect will be poor. The transform domain-based registration method requires that the matching area is large enough and supports image transformations with corresponding definitions in the Fourier transform.
[0084] 2. Robustness problem. Traditional image registration methods can overcome the shortcomings of registration methods based on camera calibration, and the requirements for shooting environment are greatly reduced. However, whether it is grayscale-based, feature-based or transform domain-based registration methods, they only involve shallow features of the image, such as grayscale features and point features, which are easily affected by illumination, noise, resolution, etc., and have low robustness. The registration model based on deep corner features in this embodiment is no longer limited to shallow features of the image. Deep corner features are different from ordinary corner features. They are out of the scope of points. Deep corner features are the aggregation of image features of a large field of view around the corners. They are more stable, more reliable, and have a wider field of view than shallow corner features. At the same time, this embodiment adopts a weakly supervised learning method, which can train the model with massive unlabeled image samples to improve the model's registration ability for images of various styles.
[0085] 3. Controllability issues. Traditional image registration methods have poor controllability. Grayscale-based registration methods use brute force to solve matching relationships, and the computational efficiency is uncontrollable. For feature-based registration methods, although the operators for extracting features are manually defined, the feature patterns are complex, and the imaging of real targets is also complex. It is difficult to define a "correct and accurate" feature extraction operator, and the extraction results are not controllable. This embodiment uses supervised learning, and the training data relies on manually marked point feature pairs. The most stable, reliable, and highest matching points seen by the human eye are used as the objects of model learning, and a model that imitates human registration thinking is trained. The training process is more controllable and the reasoning results are more credible.
[0086] 4. Accuracy. Compared with the manually defined corner extraction technology, the corner features extracted by neural network have two significant advantages. One is profundity. The manually defined corner extraction algorithm is difficult to overcome the influence of resolution, noise, etc., but the neural network can easily overcome these problems because the fitting mode of the neural network is more complex and diverse, and can adapt to corners with different characteristics. Moreover, the receptive field of the neural network is adjustable and large enough, and is not sensitive to image resolution. Second, the correlation is taken into account. Learning the matching relationship between the template and the image through the neural network model is based on the matching perspective and premise, and then considering whether it is a corner point. It is essentially different from simply extracting corner points. If the corner points are extracted by the neural network model, if they are unmatched corner points, even if the corner point features are significant, they should be ignored.
[0087] It should be pointed out that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A rigid circuit board registration model based on deep corner feature matching, characterized in that: It includes feature extraction module, position encoding module, coarse-grained feature alignment module, matching point attention module, fine-grained feature alignment module and output module; The feature extraction module is used to extract pixel features in the circuit board template and the image; The position encoding module is used to encode the position of the pixel feature output by the feature extraction module, so that the pixel feature contains position information; The coarse-grained feature alignment module is used to align the coarse-grained features of the encoded image with the coarse-grained features of the template and output a coarse-grained feature matching matrix; The matching point attention module is used to screen out coarse-grained features with high matching degree in the feature matching matrix, and retain the features on the high-resolution feature map at the corresponding position based on the scale scaling relationship between the coarse-grained features and the fine-grained features; The fine-grained feature alignment module performs feature alignment on the fine-grained features of the encoded image and the fine-grained features of the template and outputs a fine-grained feature matching matrix; The output module transforms the fine-grained feature matching matrix into a two-dimensional probability matrix and outputs it.
2. The rigid circuit board registration model based on deep corner feature matching according to claim 1 is characterized in that: It also includes a self-attention module; The self-attention module is used to learn the associations between pixels within the feature map and adjust the feature response values based on these associations.
3. A rigid circuit board registration method based on deep corner feature matching, characterized in that: The steps include: S1, establishing a neural network model; the model includes a feature extraction module, a position encoding module, a coarse-grained feature alignment module, a matching point attention module, a fine-grained feature alignment module and an output module; S2, input the circuit board template and image into the neural network model; S3, feature extraction module extracts features of template and image respectively; S4, the position encoding module performs coarse-grained feature encoding and fine-grained feature encoding on the coordinate positions of the extracted image features and template features respectively; S4, the coarse-grained feature alignment module performs feature alignment on the coarse-grained features of the encoded image and the coarse-grained features of the module and outputs a feature matching matrix; S5, matching point attention: The matching point attention module selects the coarse-grained features with high matching degree in the feature matching matrix, and retains the features on the high-resolution feature map at the corresponding position based on the scaling relationship between the coarse-grained features and the fine-grained features; S6, fine-grained feature alignment. The fine-grained feature alignment module retains the fine-grained features of the area corresponding to the coarse-grained features with higher scores in the coarse-grained feature matching scores under the guidance of the matching point attention module, and aligns the fine-grained features of the retained image with the fine-grained features of the template, and finally outputs the fine-grained feature matching matrix. S7, the output module transforms the fine-grained feature matching matrix into a two-dimensional probability matrix and outputs it.
4. The method according to claim 3, characterized in that The S2 includes: S2.1, converting two-dimensional image features into one-dimensional sequence features; S2.2, arrange each pixel in the one-dimensional sequence feature in order; S2.3, performing trigonometric transformation; S2.4, adding the transformed arrangement sequence number to the one-dimensional sequence feature in S2.1 to obtain a feature sequence with position information.
5. The method according to claim 4, characterized in that Also includes: The self-attention module obtains the feature sequence with position information and outputs the feature sequence after self-attention activation to the coarse-grained feature alignment module; The coarse-grained feature alignment module performs feature alignment on the coarse-grained features of the image and the coarse-grained features of the module based on the feature sequence after self-attention activation and outputs feature matching data.
6. The method according to claim 5, characterized in that In S2, the feature extraction module uses the resnet network and the FPN network as feature extraction networks.
7. The method according to claim 6, characterized in that In S3, the position encoding module is Positional Encoding in the transformer model.
8. The method according to claim 7, characterized in that The self-attention model is the SelfAttention in the transformer model.
Citation Information
Patent Citations
Printed circuit board defect detection method and system based on deep learning
CN114140385A
Method for diagnosing circuit board fault based on visible light image and infrared thermal image sequence
CN116429783A
Inspection method for inspecting an object and machine vision system
US20230138821A1
