Semantic information semi-supervision-based flexible circuit board image cross-domain registration model and method
Through the cross-domain registration model of flexible circuit board images based on semantic information, the deformation field is generated by deep learning algorithm for image correction, which solves the problem of flexible circuit board image registration, and achieves efficient and accurate image alignment, which is suitable for High resolution circuit board.
Patent Information
- Application Number
- CN202411231754.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-11-22
AI Technical Summary
The prior art is difficult to effectively solve the registration problem of flexible circuit board images, especially when there are position deviations, angle deviations and domain differences in the image. Traditional methods rely on key point matching and complex calculations, making it difficult to cope with the telescopic deformation of flexible circuit boards. .
Adopt a cross-domain registration model of flexible circuit board images based on semi-supervised semantic information, through sharding module, linear embedding module, encoding layer module, upsampling module, downsampling module, decoding layer module, multi-layer perceptron module and spatial transformation The module uses deep learning algorithm to learn mutual attention between images, generate deformation fields for image correction, and conduct supervision and training in combination with semantic information to reduce training costs.
It realizes accurate registration of flexible circuit board images, can cope with large differences in image characteristics or grayscale information, reduces computing resource consumption, improves registration efficiency and accuracy, and is suitable for high-resolution circuit board images.
Smart Images

Figure CN119006549A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of circuit board image registration, and in particular to a flexible circuit board image cross-domain registration model and method based on semi-supervision of semantic information. Background Art
[0002] PCB image registration is a technique that aligns multiple PCB images in space for subsequent analysis and processing. This technique is particularly important in the production of PCBs, as images produced at different steps may have positional deviations, angle deviations, and other issues. By registering these images, the accuracy and efficiency of PCB processing can be improved, and by comparing them with standard PCB images, it is possible to detect whether the PCB being produced has defects.
[0003] The current mainstream method is to obtain the perspective transformation relationship between the circuit board image and its reference image by matching the key points of the circuit board image. However, this method usually requires the circuit board to be rigid and not to deform or bend. With the widespread application of flexible circuit boards in the industrial field, there is a lack of effective solutions for the registration of flexible circuit board images.
[0004] Based on this, the present invention proposes a semi-supervised cross-domain registration model and method of flexible circuit board images based on semantic information. The method adopts key point-free alignment technology and realizes accurate registration of flexible circuit board images based on the semantic information and structural information of the image. Summary of the invention
[0005] In order to solve the above technical problems, the present invention discloses a cross-domain registration model and method of flexible circuit board images based on semi-supervision of semantic information. The technical solution of the present invention is implemented as follows:
[0006] The first aspect of the present invention is to disclose a semi-supervised cross-domain registration model of flexible circuit board images based on semantic information, the model includes a slicing module, a linear embedding module, a coding layer module, an upsampling module, a downsampling module, a decoding layer module, a multi-layer perceptron module, and a spatial transformation module;
[0007] The slicing module is used to slice the image into multiple non-overlapping slices according to a grid of fixed size;
[0008] The linear embedding module is used to linearly combine the slice pixels and perform high-dimensional decomposition of the pixel values;
[0009] The coding layer module is used to extract image features and obtain similarity of image features;
[0010] The downsampling module is used to reduce the resolution of image features;
[0011] The upsampling module is used to improve the resolution of image features;
[0012] The decoding layer module is used to fuse image features and perform transformations to obtain similarities of image features;
[0013] The multi-layer perceptron module is used to transform the output image features into a deformation field;
[0014] The spatial transformation module is used to perform elastic transformation on the image according to the deformation field to obtain a corrected image.
[0015] Preferably, the encoding layer module includes a self-attention encoder and at least one cross-attention encoder, and the decoder module includes a self-attention decoder and at least one cross-attention decoder; the self-attention encoder and the self-attention decoder are implemented using window multi-head attention.
[0016] Preferably, the cross-attention encoder and the cross-attention decoder are implemented using sliding window multi-head attention, wherein the input of the sliding window multi-head attention includes image features of the circuit board image and image features of the reference circuit board image.
[0017] Preferably, the up-sampling module adopts a bilinear interpolation method, and the down-sampling module adopts a PatchMerging method.
[0018] Preferably, the multi-layer perceptron module adopts an MLP network; and the spatial transformation module adopts an STN network.
[0019] The second aspect of the present invention is to disclose a method for cross-domain registration of flexible circuit board images based on semi-supervision of semantic information, the method comprising the following steps:
[0020] S1, inputting a circuit board image and a reference circuit board image into a slicing module, and slicing the circuit board image and the reference circuit board image into a plurality of non-overlapping slices respectively;
[0021] S2, inputting the output result of step S1 into a linear embedding module, linearly combining the slice pixels of the circuit board image and the reference circuit board image, and performing high-dimensional decomposition on the pixel values;
[0022] S3, inputting the output result of step S2 into a coding layer module, extracting image features of the circuit board image and the reference circuit board image, and then obtaining a matching relationship between the image features of the two;
[0023] S4, inputting the output result of step S3 into a downsampling module to reduce the resolution of the image features of the circuit board image and the image features of the reference circuit board image;
[0024] S5, inputting the output result of step S4 into an upsampling module to improve the resolution of the image features of the circuit board image and the image features of the reference circuit board image;
[0025] S6, inputting the output result of step S5 into a decoding layer module, fusing the features of the circuit board images with the same resolution, and fusing the features of the reference circuit board images with the same resolution, and then obtaining the image feature similarity between the two;
[0026] Inputting the output result of step S5 into the decoding layer module, fusing the circuit board image features of different resolutions, and fusing the reference circuit board image features of different resolutions, and then obtaining the image feature similarity between the two;
[0027] S7, inputting the output result of step S6 into a multi-layer perceptron module, so that the image features finally output by the decoding layer module are transformed into a deformation field of the circuit board image relative to the reference circuit board image;
[0028] S8, inputting the output result of step S7 into a space transformation module for elastic transformation to obtain a rectified image of the circuit board image.
[0029] Preferably, in step S1, the areas where the boundary is smaller than the slice size are filled with 0 values to increase the area before slice is performed.
[0030] Preferably, the step S8 further includes model loss; the model loss is specifically as follows:
[0031] Loss = L sim +αL smooth +βL dice
[0032] Among them, Loss represents the total loss, L s i m represents the similarity loss, L smooth represents the smoothing loss, L dice represents the semantic shift loss, α and β are adjustment coefficients of the smoothing loss and the semantic shift loss respectively;
[0033] The expression of similarity loss is as follows:
[0034]
[0035] The expression of smooth loss is as follows:
[0036]
[0037] The expression of semantic shift loss is as follows:
[0038]
[0039] In the above formulas, f represents the reference image, p represents the grayscale value of the image pixel, represents the image after spatial transformation according to the deformation field, and represents the local average intensity, represents the difference of the deformation field in different axis directions, X c and Y c represents the mask of the c-th category channel respectively.
[0040] The advantages of the present invention are as follows:
[0041] (1) The present invention adopts a deep learning algorithm to learn the mutual attention between the circuit board image and the reference circuit board image, and generates a deformation field between the image and the reference image based on the mutual attention. The deformation field is used to complete the pixel-by-pixel matching between the image and the reference image, which can address the problem of flexible circuit board image registration.
[0042] (2) The present invention proposes to use semantic information as important information to solve cross-domain circuit board image registration, because semantic information includes the location and category of circuit board components. Alignment of semantic information is a necessary condition for image alignment. Semantic information is manually labeled and is not affected by domain differences. Using semantic information to supervise the effect of deformation field prediction can effectively overcome the problem that the image and the reference image cannot be matched when there are large differences in image features or grayscale information.
[0043] (3) Flexible circuit board image registration requires pixel-level image alignment. The cost of annotating such alignment features is high and often lacks operability. By taking the electronic components in the circuit board image as image elements and annotating the circuit board image element by element, the cost of producing training samples can be greatly reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0045] Figure 1 This is a structural diagram of a semi-supervised cross-domain registration model for flexible circuit board images based on semantic information;
[0046] Figure 2 for Figure 1 The structural diagram of encoding and decoding image features in the model shown;
[0047] Figure 3 for Figure 1 The structural diagram of the coding layer module in the model shown;
[0048] Figure 4 for Figure 1 Flowchart for calculation of model loss in the model shown. DETAILED DESCRIPTION
[0049] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention and the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0050] Unless otherwise defined, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by technicians in the technical field to which the present invention belongs; the terms used in the specific implementation methods are only for the purpose of describing specific embodiments and are not intended to limit the present invention; the terms "including" and "having" in the specification and claims of the present invention and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.
[0051] In the description of the specific implementation methods of the present invention, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present invention, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.
[0052] Reference to "embodiments" in the present invention means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in the present invention may be combined with other embodiments.
[0053] In the description of the embodiments of the present invention, the term "and / or" is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects before and after are in an "or" relationship.
[0054] It should be noted that, for the convenience of description, in the following embodiments, all identical technical features are marked with the same symbols.
[0055] PCB image registration is a technique that aligns multiple PCB images in space for subsequent analysis and processing. This technique is particularly important in the production process of PCBs, because images produced at different steps may have problems such as positional deviation and angle deviation. By registering these images, the accuracy and efficiency of PCB processing can be improved. It is also possible to compare the images with the standard PCB images to determine whether there are defects in the PCB being produced.
[0056] At present, the existing technologies for image registration are grayscale-based image registration and feature-based image registration. The specific details and technical deficiencies are as follows:
[0057] 1. Grayscale-based image registration is an image registration method that does not require explicit feature extraction and matching. It achieves image alignment by directly comparing the grayscale information of two images. Common methods for measuring grayscale information include mutual information, mutual correlation coefficient, and mean square error.
[0058] However, the grayscale-based image registration method involves the grayscale information of the entire image, and the computational complexity is high, especially for high-resolution images. The direct comparison of grayscale information may be affected by image noise, and the optimization process may converge to a local minimum, especially when the similarity metric function is non-convex, and the calculation result is unstable. The grayscale-based image registration method completely relies on the grayscale information between images. When the image is curved or there is a large domain difference with the reference image, the grayscale relationship will be destroyed, resulting in registration failure.
[0059] 2. Feature-based image registration is a widely used image registration method. It mainly relies on features extracted from the image for registration. Common features include point features, line features, and texture features.
[0060] Traditional feature detection algorithms for feature-based image registration include SIFT, SURF, Harris, ORB, etc. Among them, SIFT features are local features of the image, which remain invariant to rotation, scaling, and brightness changes, and also maintain a certain degree of stability to perspective changes, affine transformations, and noise; SURF features also have the advantages of SIFT features. The pyramid image constructed by SURF is very different from SIFT. SIFT uses DOG images, while SURF uses Hessian matrix determinant approximation images. The Hessian matrix is used to generate image-stable edge points (mutation points). Similar to SIFT feature point matching, SURF also determines the matching degree by calculating the Euclidean distance between two feature points. The shorter the Euclidean distance, the better the matching degree of the two feature points. Harris corner detection is the basis of feature point detection. It proposes the concept of applying the grayscale difference of neighboring pixels to determine whether it is a corner, edge, or smooth area. The principle of Harris corner detection is to use a moving window to calculate the grayscale change value in the image. The key processes include conversion to grayscale image, calculation of differential image, Gaussian smoothing, calculation of local extreme values, and confirmation of corner points. ORB solves the scale invariance and rotation invariance through pyramid and centroid calibration based on fast corner features, and solves the problem of difficulty in distinguishing rotation characteristics and feature points by introducing the rotation angle of corner features and machine learning.
[0061] The image registration algorithm based on deep features optimizes the key point feature extraction algorithm and matching algorithm. The more classic algorithm is SuperPoint. SuperPoint proposes a self-supervised framework that can be used to train feature point detection and representation for a large number of multi-view geometric problems in computer vision.
[0062] The matching of two images can be completed by obtaining feature points based on the feature detection algorithm and identifying feature point pairs through algorithms such as nearest neighbor matching and FLANN.
[0063] However, the feature-based registration method relies on being able to extract enough features through the established feature extraction mode. When fewer features are extracted, the matching effect is poor. At the same time, the process of extracting features is also easily affected by noise. In the process of feature matching, if there is a large domain difference between the image and the reference image, the extracted features will also be very different, and the matching effect will be very poor. Another fatal problem is that the purpose of the feature-based image registration method is to obtain the homography matrix between the image and the reference image through matching feature pairs, and then calibrate the position and posture of the image through the homography matrix. It can only deal with the image mismatch problem caused by projective transformation.
[0064] Based on this, the present invention proposes a semi-supervised cross-domain registration model and method of flexible circuit board images based on semantic information. The method adopts key point-free alignment technology and realizes accurate registration of flexible circuit board images based on the semantic information and structural information of the image.
[0065] The embodiments of the present invention will be described in more detail below through examples and comparative examples. It should be noted that the embodiments of the present invention are not limited to these examples.
[0066] Example
[0067] In a specific embodiment, Figure 1 As shown, a semi-supervised cross-domain registration model of flexible circuit board images based on semantic information includes a slicing module, a linear embedding module, a coding layer module, an upsampling module, a downsampling module, a decoding layer module, a multi-layer perceptron module, and a spatial transformation module;
[0068] The specific functions of each module are as follows:
[0069] 1. Slicing module: Slicing is to divide the image into multiple non-overlapping slices according to a fixed-size grid. The purpose of slicing is to reduce computing power consumption and improve reasoning efficiency. Since a large amount of attention is used in the encoding layer and decoding layer, if slicing is not performed and pixel-level attention is directly used, calculating these attentions will occupy higher computing resources and reduce reasoning efficiency, and will not be able to meet the requirements of high-resolution circuit board image registration. To this end, slicing is performed first, and slice-level attention calculation can reduce computing resource consumption exponentially.
[0070] 2. Linear embedding module: Linear embedding is an operation that linearly combines slice pixels and decomposes pixel values into high dimensions.
[0071] 3. Coding layer module: The function of the coding layer is to extract image features and obtain the feature similarity between the circuit board image and the reference circuit board image. Each coding layer module includes multiple encoders, which are divided into self-attention encoders and cross-attention encoders. The function of the self-attention encoder is to perform autocorrelation learning on image features and obtain the feature association of the image itself. The cross-attention encoder is to obtain the similarity between the circuit board image and the reference circuit board image, that is, to obtain the feature matching relationship between the circuit board image and the reference circuit board image. Both the self-attention encoder and the cross-attention encoder are learners of slice-level feature association. Figure 3 As shown, FA is input into the self-attention encoder, while FA and FB are input into the cross-attention encoder, and FA' is obtained after a series of algorithms. Similarly, FB is input into the self-attention encoder, while FA and FB are input into the cross-attention encoder, and FB' is obtained after a series of algorithms.
[0072] 4. Downsampling module: The purpose of downsampling is to reduce the resolution of image features, reduce noise, and enhance the robustness of the model to changes in image scale. At the same time, it can also reduce computing resource consumption and deepen the network depth.
[0073] 5. Upsampling module: Upsampling is the inverse operation of downsampling, and its purpose is to improve the resolution of image features.
[0074] 6. Decoding layer module: The structure of the decoding layer is the same as that of the encoding layer. Figure 3 As shown in Figure 1, its main components are the self-attention decoder and the cross-attention decoder. The function of the decoding layer is to fuse the encoding layer features of the same resolution and the decoding layer features of the low resolution, and transform the fused features to obtain the feature similarity between the circuit board image and the reference circuit board image.
[0075] 7. Multi-layer Perceptron module (MLP): used to transform the features output by the last decoding layer into a deformation field of the circuit board image relative to the reference circuit board image.
[0076] 8. Spatial transformation module: elastically transform the image according to the deformation field to obtain a corrected image of the circuit board image.
[0077] Among them, Figure 2 As shown in the figure, the model encodes and decodes image features through the encoding layer module and the decoding layer module, constructing a bottom-to-top and top-to-bottom U-shaped structure. The left part is a bottom-to-top structure, which is the encoding structure of image features. There are multiple encoding layers inside. The purpose is to extract image features of different resolutions through the network layer. The right part is a top-to-bottom structure, which is the decoding structure of image features. There are multiple decoding layers inside. The purpose is to fuse features of different resolutions through the feature pyramid network.
[0078] The specific application process of the model in this embodiment is as follows:
[0079] The model adopts the idea of bidirectional registration. The so-called bidirectional registration means that for a pair of images (set as Figure A and Figure B), Figure A is assumed to be the reference circuit board image, Figure B is the circuit board image, and Figure A corrects Figure B, or Figure B is assumed to be the reference circuit board image, Figure A is the circuit board image, and Figure B corrects Figure A.
[0080] S1, input the circuit board image and the reference circuit board image into the slicing module, and use the Patch Partition method in the literature "Swin Transformer V2" (Liu Z, Hu H, Lin Y, et al. Swin Transformer V2: Scaling Up Capacity and Resolution [J]. 2021. DOI: 10.48550 / arXiv.2111.09883.) to cut the circuit board image and the reference circuit board image into multiple non-overlapping slices;
[0081] In this embodiment, in step S1, the areas where the boundary is smaller than the slice size are filled with 0 values to increase the area before slice is performed.
[0082] S2, input the output result of step S1 into the linear embedding module, linearly combine the slice pixels of the circuit board image and the reference circuit board image, and perform high-dimensional decomposition on the pixel values.
[0083] In this embodiment, linear embedding adopts the LinearEmbeding operation in "SwinTranformerV2", which is specifically implemented through a convolution operation. Through this convolution, feature combination and channel dimension improvement are achieved for each slice.
[0084] S3, inputting the output result of step S2 into the coding layer module, extracting the image features of the circuit board image and the reference circuit board image, and then obtaining the matching relationship between the image features of the two.
[0085] In this embodiment, the encoding layer module includes a self-attention encoder and at least one cross-attention encoder, and the decoder module includes a self-attention decoder and at least one cross-attention decoder; in the actual application of this embodiment, the self-attention encoder and the self-attention decoder are implemented using window multi-head attention, and the cross-attention encoder and the cross-attention decoder are implemented using sliding window multi-head attention, wherein the input of the sliding window multi-head attention includes image features of the circuit board image and the reference circuit board image.
[0086] S4, inputs the output result of step S3 into the downsampling module to reduce the image feature resolution of the circuit board image and the benchmark circuit board image, reduce noise, and enhance the robustness of the model to image scale changes.
[0087] In this embodiment, the downsampling module adopts the PatchMerging method in "Swin TranformerV2".
[0088] S5, inputting the output result of step S4 into an upsampling module to improve the image feature resolution of the circuit board image and the reference circuit board image.
[0089] In this embodiment, the up-sampling module adopts a bilinear interpolation method.
[0090] S6, inputting the output result of step S5 into the decoding layer module, fusing the features of the circuit board images with the same resolution, and fusing the features of the reference circuit board images with the same resolution, and then obtaining the image feature similarity between the two;
[0091] The output result of step S5 is input into the decoding layer module, the circuit board image features of different resolutions are fused, and the reference circuit board image features of different resolutions are fused, and then the image feature similarity between the two is obtained.
[0092] S7, inputting the output result of step S6 into the multi-layer perceptron module, so that the image features finally output by the decoding layer module are transformed into the deformation field of the circuit board image relative to the reference circuit board image.
[0093] In this embodiment, the multi-layer perceptron module adopts an MLP network, and the multi-layer perceptron layer is implemented using the MLP network in "SwinTranformerV2".
[0094] S8, inputting the output result of step S7 into the space transformation module for elastic transformation to obtain a rectified image of the circuit board image (Figure A' and Figure B').
[0095] In this embodiment, the spatial transformation module adopts the STN network proposed in the document "Spatial Transformer Networks" (MoK.Spatial Transformer Network[J].[2024-07-18]).
[0096] Step S8 in the specific application process of the model in this embodiment also includes model loss. The model loss includes three parts, namely similarity loss, smoothness loss and semantic offset loss. The supervisory role of semantic information for cross-domain circuit board image registration is mainly reflected by semantic offset loss. The semantic segmentation images of the circuit board image and the reference circuit board image contain their semantic information. When the semantic segmentation map of the corrected circuit board image is aligned with the semantic segmentation map of the reference circuit board image, the corrected circuit board image and the reference circuit board image must also be aligned. Therefore, when there is no semantic offset loss, the positions of the electronic components on the circuit board image and the electronic components on the reference circuit board image are aligned, indicating that the deformation field predicted by the model is basically reliable. Through this deformation field, the registration of the circuit board image and the reference circuit board image can be achieved. Similarity loss and smoothness loss can further optimize the prediction effect of the deformation field. Similarity loss is mainly used to measure the deviation between the corrected circuit board image and the reference circuit board image. The mismatch between the circuit board image and the reference circuit board image is mainly caused by the deformation caused by the ductility and softness of the flexible circuit board. The displacement between each pixel of the circuit board image and the reference circuit board image is smooth, and the smoothness loss is to add smoothness constraints to the deformation field.
[0097] The above model loss application process is as follows Figure 4 As shown, the details are as follows:
[0098] Figure A is spatially transformed according to its deformation field to obtain the corrected Figure A, and the similarity loss is calculated between the corrected Figure A and Figure B. Figure B is spatially transformed according to its deformation field to obtain the corrected Figure B, and the similarity loss is also calculated between the corrected Figure B and Figure A. The deformation field of Figure A and the deformation field of Figure B can each calculate the smoothness loss through differential operation. The semantic segmentation image of Figure A is spatially transformed according to its deformation field to obtain the corrected semantic segmentation image of Figure A, and the semantic offset loss is calculated between the corrected semantic segmentation image of Figure A and the semantic segmentation image of Figure B. The semantic segmentation image of Figure B is spatially transformed according to its deformation field to obtain the corrected semantic segmentation image of Figure B, and the semantic offset loss is calculated between the corrected semantic segmentation image of Figure B and the semantic segmentation image of Figure A.
[0099] In this embodiment, the total loss, similarity loss, smoothness loss and semantic shift loss are implemented by the following functions:
[0100] Loss = L sim +αL smooth +βL dice
[0101] Among them, Loss represents the total loss, L sim represents the similarity loss, L smooth represents the smoothing loss, L dicerepresents the semantic shift loss, α and β are the adjustment coefficients of the smoothing loss and the semantic shift loss respectively;
[0102] L sim , L smooth and L dice The expression is as follows:
[0103]
[0104] Where f represents the reference image, p represents the grayscale value of the image pixel, represents the image after spatial transformation according to the deformation field, and represents the local average intensity, represents the difference of the deformation field in different axis directions, X c and Y c They represent the masks of the c-th category channel respectively.
[0105] It should be pointed out that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A semi-supervised cross-domain registration model for flexible circuit board images based on semantic information, characterized by: Including slicing module, linear embedding module, encoding layer module, upsampling module, downsampling module, decoding layer module, multi-layer perceptron module, and spatial transformation module; The slicing module is used to divide the image into multiple non-overlapping slices according to a fixed-size grid; The linear embedding module is used to linearly combine the slice pixels and perform high-dimensional decomposition of the pixel values; The coding layer module is used to extract image features and obtain similarity of image features; The downsampling module is used to reduce the resolution of image features; The upsampling module is used to improve the resolution of image features; The decoding layer module is used to fuse image features and perform transformations to obtain similarity of image features; The multi-layer perceptron module is used to convert the output image features into a deformation field; The spatial transformation module is used to perform elastic transformation on the image according to the deformation field to obtain a corrected image.
2. A semi-supervised cross-domain registration method for flexible circuit board images based on semantic information, characterized in that: The steps are as follows: S1, inputting a circuit board image and a reference circuit board image into a slicing module, and slicing the circuit board image and the reference circuit board image into a plurality of non-overlapping slices respectively; S2, inputting the output result of step S1 into a linear embedding module, linearly combining the slice pixels of the circuit board image and the reference circuit board image, and performing high-dimensional decomposition on the pixel values; S3, inputting the output result of step S2 into the coding layer module, extracting image features of the circuit board image and the reference circuit board image, and then obtaining a matching relationship between the image features of the two; S4, inputting the output result of step S3 into a downsampling module to reduce the resolution of the image features of the circuit board image and the image features of the reference circuit board image; S5, inputting the output result of step S4 into an upsampling module to improve the resolution of the image features of the circuit board image and the image features of the reference circuit board image; S6, inputting the output result of step S5 into a decoding layer module, fusing the features of the circuit board images with the same resolution, and fusing the features of the reference circuit board images with the same resolution, and then obtaining the image feature similarity between the two; Inputting the output result of step S5 into the decoding layer module, fusing the circuit board image features of different resolutions, and fusing the reference circuit board image features of different resolutions, and then obtaining the image feature similarity between the two; S7, inputting the output result of step S6 into a multi-layer perceptron module, so that the image features finally output by the decoding layer module are converted into a deformation field of the circuit board image relative to the reference circuit board image; S8, inputting the output result of step S7 into a spatial transformation module for elastic transformation to obtain a rectified image of the circuit board image.
3. The semi-supervised cross-domain registration model for flexible circuit board images based on semantic information according to claim 1, characterized in that: The encoding layer module includes a self-attention encoder and at least one cross-attention encoder, and the decoder module includes a self-attention decoder and at least one cross-attention decoder; the self-attention encoder and the self-attention decoder are implemented using window multi-head attention.
4. The semi-supervised cross-domain registration model for flexible circuit board images based on semantic information according to claim 3 is characterized in that: The cross-attention encoder and the cross-attention decoder are implemented using sliding window multi-head attention, wherein the input of the sliding window multi-head attention includes image features of the circuit board image and image features of the reference circuit board image.
5. The semi-supervised cross-domain registration model for flexible circuit board images based on semantic information according to claim 1, characterized in that: The up-sampling module adopts a bilinear interpolation method, and the down-sampling module adopts a PatchMerging method.
6. The semi-supervised cross-domain registration model for flexible circuit board images based on semantic information according to claim 1, characterized in that: The multi-layer perceptron module adopts an MLP network; the spatial transformation module adopts an STN network.
7. The method for cross-domain registration of flexible circuit board images based on semi-supervised semantic information according to claim 2, characterized in that: In step S1, the areas where the boundary is smaller than the slice size are filled with 0 values to increase the area before the slice is performed.
8. The method for cross-domain registration of flexible circuit board images based on semi-supervised semantic information according to claim 2, characterized in that: The step S8 further includes model loss; the model loss is specifically as follows: Loss=L sim +αL smooth +βL dice Among them, Loss represents the total loss, L sim represents the similarity loss, L smooth represents the smoothing loss, L dice represents the semantic shift loss, α and β are the adjustment coefficients of the smoothing loss and the semantic shift loss respectively; The expression of similarity loss is as follows: The expression of smoothing loss is as follows: The expression of semantic shift loss is as follows: In the above formulas, f represents the reference image, p represents the grayscale value of the image pixel, represents the image after spatial transformation according to the deformation field, and represents the local average intensity, Represents the difference of the deformation field in different axis directions, X c and Y c They represent the masks of the c-th category channel respectively.
Citation Information
Cited By
Multi-mode dMRI image distortion correction system based on unsupervised deep learning
CN119762398A
