A high-precision Mark positioning algorithm based on diffusion model

Abstract

The present invention is applicable to the field of machine vision positioning technology, and provides a high-precision Mark positioning algorithm based on a diffusion model, the method comprising: data set construction, construction of a diffusion model neural network, diffusion model neural network training, diffusion model neural network verification, diffusion model neural network evaluation, diffusion model neural network reasoning and prediction. Compared with the traditional machine vision Mark positioning method, the high-precision Mark positioning algorithm based on the diffusion model is used to perform high-precision positioning of the Mark point, which has the characteristics of high positioning accuracy, good generalization, and strong robustness. The algorithm of the present invention can be used to uniformly position common Mark shape algorithms, and has the characteristics of strong anti-interference ability. In the actual detection process, the method of the present invention can optimize the network, and adopt the parallel computing method provided by the GPU to improve the computing efficiency and overall processing speed.

Description

Technical Field

[0001] The invention belongs to the technical field of machine vision positioning, and in particular relates to a high-precision Mark positioning algorithm based on a diffusion model. Background Art

[0002] In recent years, with the rapid development of Industry 4.0, machine vision has played an important role in industrial product inspection, assembly, identification, positioning and dimension measurement due to its advantages of non-contact, high precision and good real-time performance. As a unit technology, machine vision positioning is equivalent to the eyes of smart devices, completing the positioning of target workpieces; it has been widely used in the processing and production of OGS touch panels, IC chips, PCB boards and HDI boards. The basic principle of machine vision positioning is to use image sensors to simulate the vision of the human eye, extract the features of the target workpiece from the image of the detection object, and process the features in real time, and finally achieve the positioning of the target workpiece.

[0003] Mark point positioning is the most common machine vision positioning technology and is widely used in the electronic manufacturing and electronic printing industries. Mark points, also called reference points, are common positioning measurement points provided for all steps in the IC industry assembly process. They are used to ensure that during the assembly process, each device and each component can be accurately positioned to the originally designed position. As the integration of ICs and circuit boards increases, the requirements for inter-layer positioning accuracy increase accordingly. Therefore, improving the positioning accuracy of mark points has become an important basis for the technological advancement of electronic manufacturing equipment. Mark points have a variety of shapes, usually including common shapes such as circles, squares, triangles, diamonds, and crosshairs.

[0004] Mark positioning technology is used in all production processes of the PCB industry and directly determines the quality of the PCB. For example, the high-precision forming of PCBs directly determines the quality of electronic equipment. Some PCBs with complex forming shapes and very high internal and external matching precision requirements or some PCBs that need to engage with other sockets require high-precision PCB contour forming. High-precision forming is highly dependent on the high-precision positioning technology of Marks in the forming process.

[0005] In recent years, many researchers at home and abroad have proposed various Mark point positioning algorithms, mainly including two categories based on template matching and Hough transform. The core process of these algorithms can be divided into three stages: first, the original image is preprocessed including binarization and filtering; then, feature extraction algorithms such as Sift, HOG, and DPM are used to extract features; finally, the Mark point is positioned using a positioning algorithm. However, the existing Mark point positioning algorithms rely heavily on preprocessing and manually selected target features, so they are insufficient in anti-interference ability, generalization ability, and robustness.

[0006] The key concept of operators such as Sift and HOG is to describe the gradient or edge direction distribution of the mark point edge. However, when the mark point edge contrast is not high, preprocessing operations such as binarization and filtering become difficult. To solve this problem, researchers need to continue to explore more superior mark point positioning algorithms.

[0007] In addition, when the Mark image is interfered by noise, the manually extracted features will be seriously affected, resulting in reduced accuracy or even failure of the Mark point positioning algorithm. After setting the algorithm parameters, if the Mark point graphics are incomplete or have defects, the algorithm cannot be adjusted dynamically, which may eventually lead to Mark point positioning failure. In addition, the existing algorithms mainly locate specific types of Mark logos, which makes it inconvenient to switch to other types of Mark logos during runtime. These problems have brought huge challenges to the design of high-precision Mark point positioning algorithms.

[0008] There are two main ways to form a mark on a PCB: one is to form a mark of the corresponding shape on the PCB by drilling; the other is to form a mark in the image transfer process by laser irradiation on the photosensitive material covering the PCB. Regardless of which method is used to generate the mark, defects such as burrs on the edge of the mark point will have a significant impact on the positioning accuracy during high-precision positioning.

[0009] With the development of generative models, image generation algorithms based on diffusion models have gradually attracted widespread attention. The diffusion model defines a Markov chain of diffusion steps to slowly add random noise to the data, and then learns to reverse the diffusion process to construct the required data samples from the noise. The diffusion model has been successfully applied in multiple image processing applications, especially in the field of multimodal image generation. Compared with traditional image processing algorithms, image processing algorithms based on diffusion models have more powerful feature expression capabilities, strong generalization capabilities, and better robustness.

[0010] Compared with the high-precision Mark positioning algorithm based on deep learning, the generative model does not require a large number of sample data sets containing the actual Mark point locations. This can well solve the problem of images collected in the production process of the IC industry, where the actual location of the Mark point is usually unknown. Summary of the invention

[0011] Since the existing Mark positioning methods fail to achieve satisfactory accuracy in actual production, especially in the fields of industry and manufacturing, it is necessary to deal with the Mark logos obtained by industrial cameras in complex environments such as low edge contrast, incomplete Mark points and defects. In view of this situation, in order to meet the demand for strong robustness and good generalization ability of high-precision Mark point positioning algorithms, the algorithm described in the present invention introduces the currently developed mature diffusion model, which is used in the reconstruction of clear Mark logos from Mark logos with low edge contrast, and uses the generation effect of the generation model to guide the reconstruction of the Mark logo, so as to obtain the precise positioning of the original Mark logo in the image. The algorithm has excellent robustness and superior generalization performance. In addition to being able to quickly and accurately locate standard Mark points, it can also perform high-precision positioning of low-contrast, incomplete and defective Mark points under different imaging effects and lighting conditions. The application of this algorithm will maximize the positioning accuracy of Mark points and increase the stability of positioning results.

[0012] The present invention is implemented as follows: a high-precision Mark positioning algorithm based on a diffusion model, the method comprising:

[0013] S1. Build a dataset containing training set, test set and validation set;

[0014] S2. Collect and annotate a dataset of images containing Mark signs, and construct a U-Net architecture. Add a fully connected layer at the end of the U-Net to output the predicted position coordinates of the Mark signs, and obtain a diffusion model neural network.

[0015] S3, inputting the images in the training set into the constructed diffusion model neural network to train the diffusion model neural network;

[0016] S4, input the data in the validation set into the trained diffusion model neural network, obtain the predicted Mark position output by the model, calculate the difference between the actual position and the predicted position in the validation data set, and if the error is greater than the set threshold, adjust the model's hyperparameters, return to S3, and continue training the model; if the error is less than the set threshold, stop training;

[0017] S5. Input the data in the test set into the verified diffusion model neural network to obtain the predicted Mark position output by the model , calculate the difference between the actual position and the predicted position in the test data set. If the error is greater than the set threshold, return to S3 and continue training the model; if the error is less than the set threshold, stop training and the model training ends;

[0018] S6. Deploy the diffusion model neural network into the application environment, input the image obtained from the industrial camera into the diffusion model neural network, and output the position of the Mark in the image.

[0019] As a further solution of the present invention, the method for constructing the data set is:

[0020] The size of the Mark mark is obtained from the PCB exposure production process. Pictures, including is the width of the image, is the height of the picture, in pixels;

[0021] According to the process of Mark identification generation, obtain the position coordinates of the center of the Mark mark in each picture. ;

[0022] Before model training begins, the dataset is divided into training set, test set, and validation set in a ratio of 7:2:1.

[0023] As a further solution of the present invention, the backbone network of the diffusion model neural network is U-Net;

[0024] The decoder initial layer of the U-Net network contains two parallel decoders, one and two:

[0025] The decoder 1 is used to extract the global features of the entire image;

[0026] The second decoder is used to extract the local features of the Mark logo;

[0027] The outputs of decoder 1 and decoder 2 are concatenated as vectors and fed into the rest of the U-Net decoder layers as a unified vector.

[0028] After the U-Net network structure completes the decoding process of the image features, a fully connected layer is added to output the predicted position coordinates of the Mark mark. .

[0029] As a further solution of the present invention, the second decoder first performs an initial positioning of the Mark mark, and crops the sub-image initially including the Mark according to the initial positioning, and performs feature extraction on the sub-image.

[0030] The beneficial effects of the present invention are:

[0031] Compared with the traditional machine vision Mark positioning method, the high-precision Mark positioning algorithm based on the diffusion model is used to perform high-precision positioning of the Mark point, which has the characteristics of high positioning accuracy, good generalization, and strong robustness. The algorithm of the present invention can be used to uniformly position the common Mark shape algorithm, and has the characteristics of strong anti-interference ability. In the actual detection process, the method of the present invention can optimize the network, and adopt the parallel computing method provided by the GPU, which can improve the computing efficiency and overall processing speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A Mark identification image with low edge contrast provided by the present invention;

[0033] Figure 2 The ideal Mark logo provided by the present invention;

[0034] Figure 3 The image from the ideal Mark to the real Mark obtained by the industrial camera can be regarded as the forward diffusion process diagram of the diffusion model;

[0035] Figure 4 Diagram of the reverse diffusion process of obtaining the ideal Mark logo image by denoising the real Mark logo image obtained by the industrial camera. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] A high-precision Mark positioning algorithm based on a diffusion model, the method comprising:

[0038] S1. Dataset construction: Construct a dataset including training set, test set and validation set. The dataset construction method is: obtain the Mark mark from the PCB exposure production process. Pictures, including is the width of the image, is the height of the image, in pixels. Typical The value is 1600. The value is 1200. According to the process of generating the Mark mark, obtain the position coordinates of the center of the Mark mark in each picture. Before model training begins, the dataset is divided into training set, test set, and validation set in a ratio of 7:2:1.

[0039] S2. Construct a diffusion model neural network. The backbone network of the diffusion model neural network is U-Net. In order to make the diffusion model more suitable for the positioning of the Mark mark, the present invention adjusts the decoder after the MiddleBlock in the U-Net network.

[0040] The present invention designs the initial decoder layer of the U-Net network to include two parallel decoders: Decoder 1 and Decoder 2:

[0041] The decoder 1 is used to extract the global features of the entire image. The decoder is the same as the decoder in the ordinary U-Net network and needs to be upsampled;

[0042] The second decoder is used to extract the local features of the Mark. The decoder first needs to perform preliminary positioning of the Mark using an algorithm such as Yolo, and then cut out the sub-image containing the Mark according to the preliminary positioning, and perform feature extraction on the sub-image.

[0043] The outputs of decoder 1 and decoder 2 are concatenated as vectors and fed into the rest of the U-Net decoder layers as a unified vector.

[0044] After the U-Net network structure completes the decoding process of the image features, a fully connected layer is added to output the predicted position coordinates of the Mark mark. .

[0045] S3. Diffusion model neural network training. The images in the training set are input into the diffusion model neural network constructed in the second step. The output of the diffusion model neural network is a function of the hyperparameters. The image obtained after denoising the original input image, and then the Mark predicted coordinates output by the fully connected layer The loss function for model training can be written as

[0046] ;

[0047] S4. Diffusion model neural network verification. The data in the verification set is input into the diffusion model neural network trained in step 3, and the model output is the predicted mark position. , calculate the difference between the actual position and the predicted position in the validation data set. If the error is greater than the set threshold, adjust the model's hyperparameters and return to step 3 to continue training the model. If the error is less than the set threshold, stop training and go to step 5.

[0048] S5. Evaluation of the diffusion model neural network. The data in the test set is input into the diffusion model neural network verified in step 4, and the model output is the predicted mark position. , calculate the difference between the actual position and the predicted position in the test data set. If the error is greater than the set threshold, return to step 3 and continue training the model. If the error is less than the set threshold, stop training and the model training ends.

[0049] S6. Diffusion model neural network reasoning and prediction. After the diffusion model neural network training is completed, it can be deployed in the application environment. For the pictures obtained from the industrial camera, they are input into the diffusion model neural network, and the network output is the position of the Mark mark in the picture.

[0050] The present invention represents the high-precision Mark identification positioning problem as a reverse diffusion process of a diffusion model, and the image with blurred edges is considered to be a forward diffusion process of the diffusion model.

[0051] For machine vision positioning problems, Figure 1 The low edge contrast of the Mark logo shown can be considered as an image obtained by adding noise to the original clear logo image. By learning the reverse diffusion process, we can get a noisy image such as Figure 1 The Mark logo shown in the figure is constructed as follows Figure 2 Clear Mark identification as shown in.

[0052] The present invention takes a circular Mark as an example. The present invention believes that whether the Mark is formed by drilling a hole on the PCB board by a drill or by irradiating a photosensitive material covered on the PCB board by a laser, the Mark will be formed like this in an ideal state. Figure 2 However, during the processing, the drill or laser will introduce noise, and the industrial camera will also introduce noise during the shooting process due to the environment, light source, photosensitive devices, etc., which will eventually form a noisy actual shooting. Figure 1 The Mark logo is shown.

[0053] The present invention believes that Figure 2 to Figure 1 The image change can actually be regarded as Figure 3 The noise adding process shown is equivalent to the forward diffusion process of the diffusion model.

[0054] In the diffusion model, the noise added in each step above is determined by the hyperparameter controlled.

[0055] In order to Figure 1 The blurred picture shown is obtained as Figure 2 The ideal Mark logo position in the picture shown in the figure needs to be denoised from the blurred picture containing the Mark logo to reconstruct the original ideal Mark logo image. This corresponds to the reverse diffusion process in the diffusion model, such as Figure 4 shown.

[0056] Figure 4 In is unknown, which is determined by the hyperparameter controlled by the hyperparameters in the forward diffusion process The difference between the position of the Mark predicted by the model and the actual position of the Mark is used as the loss function. Through model training, the parameters of each step of the diffusion model can be determined. , thus determining the entire model. After the model is determined, a real noisy Mark image can be input and its position in the image can be output.

[0057] As mentioned above, the images containing the Mark logo obtained from the industrial camera are usually 1600×1200 pixel images, while the images input by the diffusion model are usually 128×128 pixels. Directly applying the diffusion model to the positioning of the Mark logo, in order to obtain high-precision Mark logo positioning, usually brings about problems such as high computational complexity, excessive resource consumption of model training, or even inability to train. In order to solve this problem, the present invention first preprocesses the image; secondly, the original diffusion model is partially modified. The original image first uses the Yolo algorithm to obtain the initial approximate positioning of the Mark logo, and the original image is cropped according to the positioning to obtain a cropped image containing the Mark logo, and the cropped image is used as the input for local feature extraction of the diffusion model. Secondly, the decoder of the diffusion model is partially modified, and an encoder for local feature extraction is added. In the original decoder, the input is the original image, which is used to extract global features; the input of the added decoder is the cropped image, which is used to extract local features.

[0058] In addition, the output of the diffusion model is a denoised image. The purpose of the present invention is to obtain the high-precision position of the Mark in the image. Therefore, the present invention adds a fully connected layer to the output part of the diffusion model for predicting the position of the Mark in the image.

[0059] Specifically, the forward diffusion process of the diffusion model is first trained through the Mark signs and the actual Mark position information collected by the training data set; then the reverse diffusion process is trained based on the pictures containing Mark signs obtained by the industrial camera to restore the actual position represented by the original clear Mark sign from the noisy pictures.

[0060] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0061] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0062] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0063] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A high-precision Mark positioning algorithm based on a diffusion model, characterized in that: The algorithm includes: S1. Build a dataset containing training set, test set and validation set; S2. Collect and annotate a dataset of images containing Mark signs, and construct a U-Net architecture. Add a fully connected layer at the end of the U-Net to output the predicted position coordinates of the Mark signs, and obtain a diffusion model neural network. S3, inputting the images in the training set into the constructed diffusion model neural network to train the diffusion model neural network; S4, input the data in the validation set into the trained diffusion model neural network, obtain the predicted Mark position output by the model, calculate the difference between the actual position and the predicted position in the validation data set, and if the error is greater than the set threshold, adjust the model's hyperparameters, return to S3, and continue training the model; if the error is less than the set threshold, stop training; S5. Input the data in the test set into the verified diffusion model neural network to obtain the predicted position coordinates of the predicted Mark output by the model. , calculate the difference between the actual position and the predicted position in the test data set. If the error is greater than the set threshold, return to S3 and continue training the model; if the error is less than the set threshold, stop training and the model training ends; S6. Deploy the diffusion model neural network into the application environment, input the image obtained from the industrial camera into the diffusion model neural network, and output the position of the Mark in the image; Wherein, the backbone network of the diffusion model neural network is U-Net; The decoder initial layer of the U-Net network contains two parallel decoders, one and two: The decoder 1 is used to extract the global features of the entire image; The second decoder is used to extract the local features of the Mark logo; The outputs of decoder 1 and decoder 2 are concatenated as vectors and fed into the rest of the U-Net decoder layers as a unified vector. After the U-Net network structure completes the decoding process of the image features, a fully connected layer is added to output the predicted position coordinates of the Mark mark. ; The diffusion model neural network is trained, and the diffusion model neural network output is determined by the hyperparameter The image obtained after denoising the original input image and the predicted position coordinates of the Mark mark output by the fully connected layer ; in, is the parameter of each step of the diffusion model, A hyperparameter for controlling the amount of noise added to the real image at each step; The difference between the position of the Mark predicted by the model and the actual position of the Mark is used as the loss function. The loss function of the diffusion model neural network training is: , It is the position coordinate of the center of the Mark in each picture.

2. The high-precision Mark positioning algorithm based on the diffusion model according to claim 1 is characterized in that: The method for constructing the dataset is as follows: The size of the Mark mark is obtained from the PCB exposure production process. Pictures, including is the width of the image, is the height of the picture, in pixels; According to the process of Mark identification generation, obtain the position coordinates of the center of the Mark mark in each picture. ; Before model training begins, the dataset is divided into training set, test set, and validation set in a ratio of 7:2:

1.

3. The high-precision Mark positioning algorithm based on the diffusion model according to claim 1 is characterized in that: The second decoder first performs an initial positioning of the Mark mark, and cuts out a sub-image containing the Mark mark according to the initial positioning, and performs feature extraction on the sub-image.

Images