Image capturing window path planning method in automatic optical detection system
Through the improved cross-iteration self-organization algorithm and a hybrid framework of 2-opt and improved brainstorming algorithm, the image acquisition window and path of the automatic optical detection system are optimized, and the problem of slow detection speed of large-area printed circuit boards is solved, achieving more efficient image acquisition and detection efficiency.
Patent Information
- Application Number
- CN202410840677.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-27
- Publication Date
- 2024-11-19
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
When existing automatic optical detection systems detect large-area printed circuit boards, the image splicing time is long, resulting in a low detection speed. The field of view of industrial cameras is limited and it needs to traverse the entire area to obtain the overall image, which increases data throughput and splicing time.
Using an improved cross-iteration self-organization algorithm and a hybrid framework of 2-opt and improved brainstorming algorithm, optimize the image-taking window division and movement path of industrial cameras, reduce the number of image-taking windows and optimize the path sequence, and establish an image-taking window division model. and move the path model to achieve minimum image pickup window and optimal path planning.
It improves detection efficiency, shortens image acquisition time, reduces data throughput, and improves overall detection speed and accuracy.
Smart Images

Figure CN118984424A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of automatic optical detection, and in particular to an imaging window path planning method in an automatic optical detection system. Background Art
[0002] In the field of electronic manufacturing, the automatic optical inspection (AOI) system is the key to ensuring the quality of printed circuit board components (PCBA); the AOI system based on optical principles combines image analysis, computer technology and automatic control to detect and process defects encountered in the production process. Generally, AOI is used before and after reflow soldering and before electrical testing to improve the pass rate of electrical processing or functional testing stages. It should be pointed out that the application of AOI has greatly improved the inspection efficiency and reduced the errors and instability of manual inspection. With the widespread adoption of surface mount technology (SMT), the component density in PCBA has increased significantly, making the detection of various welding defects (such as missing patches, short circuits, leaks, etc.) more and more important.
[0003] In the AOI system, component inspection is performed by an industrial camera driven by a motion mechanism. When working, the PCBA to be inspected is first fixed at a specified position on the equipment, and then the host computer sends a command to the motion control unit, which controls the mechanical structure unit and the image acquisition unit to obtain the PCBA image. Due to the strict requirements of the inspection system on size and position accuracy, high-resolution industrial cameras are usually used for image capture. However, due to the limited field of view of industrial cameras and the fact that PCBA inspection on the production line usually consists of multiple circuit boards, the industrial camera can only obtain a partial image of the entire circuit board in one shot, and the camera needs to traverse the entire PCBA area to obtain the overall image.
[0004] At present, the image acquisition method adopted by most AOI systems is still a sequential image acquisition method based on image stitching, that is, the mobile industrial camera or PCBA board captures partial images of the PCBA from left to right and from top to bottom, and then uses the regional stitching algorithm to stitch these partial images together to create a global image containing all the inspection components for inspection. When the area of the PCBA to be inspected is large, this method will result in a large amount of data throughput, resulting in a long stitching time, thereby reducing the overall inspection speed. Therefore, in addition to improving the image processing speed, improving the image acquisition speed by optimizing the inspection path is also the key to improving the efficiency of AOI inspection. Summary of the invention
[0005] The purpose of the present invention is to provide an imaging window path planning method in an automatic optical detection system, which can establish an imaging window division model and a moving path model, and solve the imaging window planning model based on an improved cross-iterative self-organizing algorithm to obtain the minimum imaging window of an industrial camera; and also solve the imaging path model through a hybrid framework based on 2-opt and an improved brainstorming algorithm to obtain the optimal imaging window planning path to improve detection efficiency.
[0006] A method for imaging window path planning in an automatic optical detection system includes the following steps, specifically:
[0007] Step S1: firstly establish an industrial camera imaging window division model for the components to be inspected on the printed circuit board;
[0008] Step S2: Based on the industrial camera imaging window division model in step S1, a moving path of the industrial camera between the imaging windows is modeled, and an imaging window moving path model is established;
[0009] Step S3: for the industrial camera imaging window division model, an improved cross-iterative self-organizing algorithm is used to determine the minimum number of imaging windows for the industrial camera;
[0010] Step S4: Design a hybrid algorithm framework for the above two-layer optimization model of the moving path. The framework uses an improved brainstorming algorithm to optimize the imaging position points of the industrial camera and a 2-opt algorithm to optimize the path sequence between the imaging position points.
[0011] Step S5: Based on the imaging path solved by the hybrid framework, a neighborhood adjustment strategy is adopted to obtain the optimal imaging path.
[0012] The overall model of the industrial camera imaging window division model in step S1 is:
[0013] O = {1, 2, ..., m};
[0014] A={1,2,...,n};
[0015] Wherein, O is the set of components to be detected, and A is the set of imaging windows after planning;
[0016] The objective function of the industrial camera imaging window division model is Min n, where n is the number of imaging windows;
[0017] Introduce a Boolean variable, which is defined as:
[0018]
[0019] The constraints of the industrial camera imaging window division model include:
[0020]
[0021] In the constraint conditions of the industrial camera imaging window partition model, W j represents the width of the detection window j, H j represents the height of the detection window j, W FOV Indicates the width of the industrial camera's field of view, H FOV Indicates the height of the industrial camera's field of view.
[0022] The overall model of the moving path model in step S2 is:
[0023] Let G = (V, E) be a weighted graph, where the point set V consisting of the image position points of the industrial camera in each image window is expressed as: V = {1, 2, ..., n}; E is the set of edges;
[0024] Introduce the Boolean variable Y pq , Y pq Defined as:
[0025]
[0026] The objective function of the mobile path model is:
[0027]
[0028] The constraints of the mobile path model include:
[0029]
[0030] In the constraints of the mobile path model, S is a subset of the set of point set V, and |S| is the number of elements in S; Represents the horizontal position coordinates of the industrial camera imaging point p, Represents the vertical position coordinate of the industrial camera imaging point p; is the lower limit of the movable range of the horizontal coordinate of position point p, It is the upper limit of the movable range of the horizontal coordinate of the position point p; is the lower limit of the movable range of the vertical coordinate of position point p, It is the upper limit of the movable range of the vertical coordinate of position point p.
[0031] The specific process of the cross-iteration self-organizing algorithm in step S3 is as follows:
[0032] Step S3.1, parsing the Gerber file of the printed circuit board to be inspected to obtain the two-dimensional data of the component to be inspected; accepting initial parameters, which include the width W of the imaging window FOV and height H FOV , number of individuals p, individual crossover probability Cr, number of individual selections in the tournament N, number of iteration terminations i, fitness function;
[0033] Step S3.2, initialize the population individuals, and for each individual X, randomly select the two-dimensional centers of m components to be detected as the centers of the imaging windows, where X is an m*2 matrix, which respectively represents the imaging position coordinates of the m imaging windows;
[0034] Step S3.3, using an iterative self-organizing algorithm to optimize the initial solution;
[0035] Step S3.4: for each individual, divide the detection elements, that is, divide the parts of the elements covered by multiple windows into a single window;
[0036] Step S3.5, performing selection and crossover operations on the population to enhance information exchange between individuals;
[0037] Step S3.6, merging the imaging windows, which is consistent with the merging logic of the iterative self-organizing algorithm;
[0038] Step S3.7, repeat steps S3.5 to S3.6 until no optimal solution is generated;
[0039] Step S3.8, outputting the optimal imaging window group;
[0040] Wherein, the step S3.5 specifically includes:
[0041] Step S3.5.1, the selection operation uses the tournament method to retain excellent individuals, and uses the elite retention strategy to directly retain the best individuals of the previous generation to the next generation;
[0042] Step S3.5.2: In the crossover operation, two individuals are randomly selected for gene recombination according to the crossover probability Cr to generate two new individuals;
[0043] Step S3.5.3: Repeat steps S3.5.1 to S3.5.2 until the preset number of iterations is reached.
[0044] The iterative self-organizing algorithm in step S3.3 is specifically as follows:
[0045] Step S3.3.1, analyzing the spatial position relationship between the imaging position point and the component to be detected, removing the imaging window that does not contain the component to be detected in each body, and completing the imaging window for the uncovered component; then judging whether the imaging windows of each body can be merged by the following formula;
[0046]
[0047] in, and is the coordinate of the upper left corner of the actual detection area in window j, and is the coordinate of the lower right corner of the actual detection area in window j; F j is the set of components in window j; and is the upper left corner coordinate of component i in window j, and is the lower right corner coordinate of component i in window j; and is the coordinate of the upper left corner of the limit search area of window j, and is the coordinate of the lower right corner of the limit search area of window j;
[0048] Step S3.3.2: If the actual detection area of imaging window u is covered by the limit search area of imaging window j, it means that the two imaging windows can be merged; the geometric center of the newly merged imaging window is determined by the following formula:
[0049]
[0050] in, and is the upper left corner coordinate of the new imaging window after the windows j and u are merged. and The lower right corner coordinate of the new imaging window after the windows j and u are merged;
[0051] Step S3.3.3: After the windows are merged, determine the component information covered by each imaging window. If there is an imaging window whose covered components are a subset of the covered component set of the remaining windows, delete the redundant window.
[0052] Wherein, the step S3.5.2 specifically includes:
[0053] Step S3.5.2.1, randomly generate an imaging window index in individual 1, and select all detection elements contained in the window;
[0054] Step S3.5.2.2, searching in individual 2 for the detection elements selected in individual 1, including the remaining elements belonging to the same window of these detection elements;
[0055] Step S3.5.2.3, return to individual 1, search for the detection elements selected in individual 2, including the remaining elements belonging to the same window of these detection elements;
[0056] Step S3.5.2.4, repeat steps S2 and 3 until all the elements selected in the two individuals are the same;
[0057] Step S3.5.2.5: Exchange the selected windows and components between the two individuals to form two new individuals.
[0058] Wherein, the step S4 specifically includes:
[0059] Step S4.1, obtaining input parameters, the input parameters including two-dimensional data of the component to be detected, and step S3 solving the obtained imaging window position information;
[0060] Step S4.2: Since the area of each imaging window is greater than or equal to its actual detection area, the adjustable area of each imaging window is calculated by the following formula;
[0061]
[0062] in, is the central horizontal coordinate of the adjustable area of window j, is the center ordinate of the adjustable area of window j, is the width of the adjustable area of window j, is the height of the adjustable area of window j;
[0063] Step S4.3, since the shortest imaging path is based on the imaging of the adjustable domain boundary, a mapping method is used to concentrate the imaging position points of each imaging window on the adjustable domain boundary to avoid the internal space and improve the search efficiency. The mapping formula is:
[0064]
[0065] Given an adjustment domain, take clockwise as an example, where v i represents the i-th vertex in the adjustment domain, v k represents the vertex before the image position point x, and P represents the perimeter of the entire adjustment domain; through the above mapping formula, the position point x is encoded as the ratio of the length from the first vertex v1 along the boundary to x to the perimeter of the entire adjustment domain; each point on the boundary is represented by a single variable in [0,1), and any value in [0,1) corresponds to a unique image position point on the boundary;
[0066] Step S4.4: Use the improved brainstorming optimization algorithm to solve the imaging position points on the adjustment domain of each imaging window, and use the 2-opt algorithm to find the access order between these position points based on the above imaging position points.
[0067] The improved brainstorming optimization algorithm in step S4.4 is specifically as follows:
[0068] In the process of improving the global search and local search of the brainstorming optimization algorithm, an adaptive adjustment strategy is introduced to conduct extensive global search in the early stage of the search process and focus on local search to accelerate the convergence of the algorithm in the later stage. The formula of the adaptive adjustment strategy is:
[0069] P1=P 1low +P range *(FEs / MaxFEs);
[0070] N k =floor(N kup -N range *(FEs / MaxFEs));
[0071] Among them, P 1low and P range are the lowest value and linear variation range of the probability of discussion within the group, N k N is the number of clustering categories used by the k-means clustering algorithm. kup and N range is the maximum number of categories and the linear range of the number of categories of the k-means clustering algorithm; MaxFEs is the maximum number of function evaluations, and FEs is the current number of function evaluations. floor() is a floor rounding function;
[0072] In addition to selecting the class center and a random individual, the group discussion method introduces a method of randomly selecting two individuals to merge and generate a new individual to ensure that the algorithm fully explores the local area;
[0073] Introduce the dynamic step size strategy, the dynamic step size strategy formula is: ξ d is the step size of the dth dimension, and is the value of the dth dimension of two individuals randomly selected from the population;
[0074] The k-means clustering operation is transferred from the decision space to the one-dimensional target space to solve the problem that the computation time of the clustering algorithm grows exponentially with the dimension of the problem;
[0075] An information interaction strategy is introduced to enhance the performance of the algorithm's dynamic search; the information interaction strategy is to perform crossover between the newly generated individuals after the discussion operation and the original index individuals, resulting in two offspring individuals, and the greedy mechanism is applied to retain the best individual among the four individuals; the method of individual crossover is single-point crossover, that is, a crossover point is randomly generated within the dimension range, and the values after the crossover point of the two individuals are exchanged to generate two offspring individuals.
[0076] Wherein, the step S4.4 specifically includes:
[0077] Step S4.4.1, randomly initialize the population individuals, each individual is a 1*n vector, where each latitude value is between 0 and 1;
[0078] Step S4.4.2, reversely solve the 1*2n position information vector according to the mapping formula;
[0079] Step S4.4.3, using the 2-opt algorithm to calculate the path length according to the position information vector, and use it as the fitness of each individual;
[0080] Step S4.4.4: Update the intra-group discussion parameter P1 and the k-means category number parameter N k ;
[0081] Step S4.4.5, clustering the population in the target space using k-means clustering;
[0082] Step S4.4.6, generating a random number to determine whether to randomly initialize a cluster center;
[0083] Step S4.4.7, for each individual, generate a random number to determine whether the new individual is updated based on the intra-group discussion or the inter-group discussion;
[0084] Step S4.4.8, according to the information interaction strategy, cross the original index individual and the newly generated individual to generate two offspring individuals, calculate the individual fitness according to the 2-opt algorithm and apply greedy selection to retain the best individual among the four individuals;
[0085] Step S4.4.9, repeat steps S4.5.4-4.5.8 until the preset number of function evaluations is reached;
[0086] Step S4.4.10: Output the global optimal solution and iteration curve.
[0087] The neighborhood adjustment strategy of step S5 specifically includes:
[0088] Step S5.1, starting from the Mark point, consider a group of multiple adjacent imaging windows on the path as a neighborhood window group;
[0089] Step S5.2, evaluating whether the imaging task can be completed by connecting the line segment of the first and last imaging windows in the group, so as to further optimize the imaging path;
[0090] Step S5.3, repeat step S5.2 until all neighboring window groups are judged;
[0091] Step S5.4, repeating steps S5.2 to S5.3 until the imaging path cannot be optimized any further;
[0092] Step S5.5: output the final imaging path.
[0093] Compared with the prior art, the present invention has the following beneficial effects. Specifically, the imaging window path planning method in the automatic optical inspection system can establish an imaging window division model and a moving path model, and solve the imaging window planning model based on an improved cross-iterative self-organizing algorithm to obtain the minimum imaging window for the industrial camera; and the imaging path model is solved through a hybrid framework based on 2-opt and an improved brainstorming algorithm to obtain the optimal imaging window planning path to improve the inspection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0094] The present invention is further described below with the aid of the accompanying drawings. However, the embodiments in the accompanying drawings do not constitute any limitation to the present invention.
[0095] Figure 1 The present invention is a flow chart of an imaging window path planning algorithm in an automatic optical inspection system.
[0096] Figure 2 Flowchart of the imaging window for solving the iterative self-organizing clustering algorithm.
[0097] Figure 3 It is a schematic diagram of the actual detection area of the detection element in the imaging window.
[0098] Figure 4 Schematic diagram of the limit search area of the imaging window.
[0099] Figure 5 Schematic diagram of the adjustable area of the imaging window.
[0100] Figure 6 Schematic diagram of boundary mapping strategy.
[0101] Figure 7 Schematic diagram of the nearest neighbor search strategy.
[0102] Figure 8 Flowchart for solving the imaging path optimization problem.
[0103] Fig. 9 The final imaging path. DETAILED DESCRIPTION
[0104] The present invention is described below in conjunction with specific implementation methods.
[0105] A method for imaging window path planning in an automatic optical inspection system, such as Figure 1 As shown, it includes the following steps, specifically:
[0106] Step S1: firstly establish an industrial camera imaging window division model for the components to be inspected on the printed circuit board;
[0107] Step S2: Based on the industrial camera imaging window division model in step S1, a moving path of the industrial camera between the imaging windows is modeled, and an imaging window moving path model is established;
[0108] Step S3: Figure 2 As shown, for the industrial camera imaging window division model, an improved cross-iterative self-organizing algorithm is used to determine the minimum number of industrial camera imaging windows;
[0109] Step S4: Design a hybrid algorithm framework for the above two-layer optimization model of the moving path. The framework uses an improved brainstorming algorithm to optimize the imaging position points of the industrial camera and a 2-opt algorithm to optimize the path sequence between the imaging position points.
[0110] Step S5: Based on the imaging path solved by the hybrid framework, a neighborhood adjustment strategy is adopted to obtain the optimal imaging path.
[0111] Furthermore, the overall model of the industrial camera imaging window division model in step S1 is:
[0112] O = {1, 2, ..., m};
[0113] A={1,2,...,n};
[0114] Wherein, O is the set of components to be detected, and A is the set of imaging windows after planning;
[0115] The objective function of the industrial camera imaging window division model is Min n, where n is the number of imaging windows;
[0116] Introduce a Boolean variable, which is defined as:
[0117]
[0118] The constraints of the industrial camera imaging window division model include:
[0119]
[0120] In the constraint conditions of the industrial camera imaging window partition model, W j represents the width of the detection window j, H j represents the height of the detection window j, W FOV Indicates the width of the industrial camera's field of view, H FOV Indicates the height of the industrial camera's field of view.
[0121] Furthermore, the overall model of the moving path model in step S2 is:
[0122] Let G = (V, E) be a weighted graph, where the point set V consisting of the image position points of the industrial camera in each image window is expressed as: V = {1, 2, ..., n}; E is the set of edges;
[0123] Introduce the Boolean variable Y pq , Y pq Defined as:
[0124]
[0125] The objective function of the mobile path model is:
[0126]
[0127] The constraints of the mobile path model include:
[0128]
[0129] In the constraints of the mobile path model, S is a subset of the set of point set V, and |S| is the number of elements in S; Represents the horizontal position coordinates of the industrial camera imaging point p, Represents the vertical position coordinate of the industrial camera imaging point p; is the lower limit of the movable range of the horizontal coordinate of position point p, It is the upper limit of the movable range of the horizontal coordinate of the position point p; is the lower limit of the movable range of the vertical coordinate of position point p, It is the upper limit of the movable range of the vertical coordinate of position point p.
[0130] It should be noted that the specific process of the cross-iteration self-organizing algorithm in step S3 is as follows:
[0131] Step S3.1, parsing the Gerber file of the printed circuit board to be inspected to obtain the two-dimensional data of the component to be inspected; accepting initial parameters, which include the width W of the imaging window FOV and height H FOV, number of individuals p, individual crossover probability Cr, number of individual selections in the tournament N, number of iteration terminations i, fitness function;
[0132] Step S3.2, initialize the population individuals, and for each individual X, randomly select the two-dimensional centers of m components to be detected as the centers of the imaging windows, where X is an m*2 matrix, which respectively represents the imaging position coordinates of the m imaging windows;
[0133] Step S3.3, using an iterative self-organizing algorithm to optimize the initial solution;
[0134] Step S3.4: for each individual, divide the detection elements, that is, divide the parts of the elements covered by multiple windows into a single window;
[0135] Step S3.5, performing selection and crossover operations on the population to enhance information exchange between individuals;
[0136] Step S3.6, merging the imaging windows, which is consistent with the merging logic of the iterative self-organizing algorithm;
[0137] Step S3.7, repeat steps S3.5 to S3.6 until no optimal solution is generated;
[0138] Step S3.8, outputting the optimal imaging window group;
[0139] Wherein, the step S3.5 specifically includes:
[0140] Step S3.5.1, the selection operation uses the tournament method to retain excellent individuals, and uses the elite retention strategy to directly retain the best individuals of the previous generation to the next generation;
[0141] Step S3.5.2: In the crossover operation, two individuals are randomly selected for gene recombination according to the crossover probability Cr to generate two new individuals;
[0142] Step S3.5.3: Repeat steps S3.5.1 to S3.5.2 until the preset number of iterations is reached.
[0143] Specifically, Figures 2 to 4 As shown, the iterative self-organizing algorithm in step S3.3 is as follows:
[0144] Step S3.3.1, Figure 3 and Figure 4 As shown, the spatial position relationship between the imaging position point and the component to be detected is analyzed, the imaging windows that do not contain the component to be detected in each body are removed, and the imaging windows of the uncovered components are completed; then the following formula is used to determine whether the imaging windows of each body can be merged;
[0145]
[0146] in, and is the coordinate of the upper left corner of the actual detection area in window j, and is the coordinate of the lower right corner of the actual detection area in window j; F j is the set of components in window j; and is the upper left corner coordinate of component i in window j, and is the lower right corner coordinate of component i in window j; and is the coordinate of the upper left corner of the limit search area of window j, and is the coordinate of the lower right corner of the limit search area of window j;
[0147] Step S3.3.2: If the actual detection area of imaging window u is covered by the limit search area of imaging window j, it means that the two imaging windows can be merged; the geometric center of the newly merged imaging window is determined by the following formula:
[0148]
[0149]
[0150] in, and is the upper left corner coordinate of the new imaging window after the windows j and u are merged. and The lower right corner coordinate of the new imaging window after the windows j and u are merged;
[0151] Step S3.3.3: After the windows are merged, determine the component information covered by each imaging window. If there is an imaging window whose covered components are a subset of the covered component set of the remaining windows, delete the redundant window.
[0152] It should be further pointed out that step S3.5.2 specifically includes:
[0153] Step S3.5.2.1, randomly generate an imaging window index in individual 1, and select all detection elements contained in the window;
[0154] Step S3.5.2.2, searching in individual 2 for the detection elements selected in individual 1, including the remaining elements belonging to the same window of these detection elements;
[0155] Step S3.5.2.3, return to individual 1, search for the detection elements selected in individual 2, including the remaining elements belonging to the same window of these detection elements;
[0156] Step S3.5.2.4, repeat steps S2 and 3 until all the elements selected in the two individuals are the same;
[0157] Step S3.5.2.5: Exchange the selected windows and components between the two individuals to form two new individuals.
[0158] For the imaging window path planning method in the automatic optical detection system of the present invention, Figure 8 As shown, step S4 specifically includes:
[0159] Step S4.1, obtaining input parameters, the input parameters including two-dimensional data of the component to be detected, and step S3 solving the obtained imaging window position information;
[0160] Step S4.2: Figure 5 As shown, since each imaging window is greater than or equal to its actual detection area, the adjustable area of each imaging window is calculated by the following formula;
[0161]
[0162] in, is the central horizontal coordinate of the adjustable area of window j, is the center ordinate of the adjustable area of window j, is the width of the adjustable area of window j, is the height of the adjustable area of window j;
[0163] Step S4.3, since the shortest imaging path is based on the imaging of the adjustable domain boundary, a mapping method is used to concentrate the imaging position points of each imaging window on the adjustable domain boundary to avoid the internal space and improve the search efficiency. The mapping formula is:
[0164]
[0165] Given an adjustment domain, such as Figure 6 As shown, taking clockwise as an example, where v i represents the i-th vertex in the adjustment domain, v k represents the vertex before the image position point x, and P represents the perimeter of the entire adjustment domain; through the above mapping formula, the position point x is encoded as the ratio of the length from the first vertex v1 along the boundary to x to the perimeter of the entire adjustment domain; each point on the boundary is represented by a single variable in [0,1), and any value in [0,1) corresponds to a unique image position point on the boundary; this mapping method also reduces the dimension of the decision variables of the subsequent algorithm, simplifies the calculation amount, and speeds up the optimization process;
[0166] Step S4.4: Use the improved brainstorming optimization algorithm to solve the imaging position points on the adjustment domain of each imaging window, and use the 2-opt algorithm to find the access order between these position points based on the above imaging position points.
[0167] For the improved brainstorming optimization algorithm in step S4.4, the details are as follows:
[0168] In the process of improving the global search and local search of the brainstorming optimization algorithm, an adaptive adjustment strategy is introduced to conduct extensive global search in the early stage of the search process and focus on local search to accelerate the convergence of the algorithm in the later stage. The formula of the adaptive adjustment strategy is:
[0169] P1=P 1low +P range *(FEs / MaxFEs);
[0170] N k =floor(N kup -N range *(FEs / MaxFEs));
[0171] Among them, P 1low and P range are the lowest value and linear variation range of the probability of discussion within the group, N k N is the number of clustering categories used by the k-means clustering algorithm. kup and N range is the maximum number of categories and the linear range of the number of categories of the k-means clustering algorithm; MaxFEs is the maximum number of function evaluations, and FEs is the current number of function evaluations. floor() is a floor rounding function;
[0172] In addition to selecting the class center and a random individual, the group discussion method introduces a method of randomly selecting two individuals to merge and generate a new individual to ensure that the algorithm fully explores the local area;
[0173] Introduce the dynamic step size strategy, the dynamic step size strategy formula is: ξ d is the step size of the dth dimension, and is the value of the dth dimension of two individuals randomly selected from the population;
[0174] The k-means clustering operation is transferred from the decision space to the one-dimensional target space to solve the problem that the computation time of the clustering algorithm grows exponentially with the dimension of the problem;
[0175] An information interaction strategy is introduced to enhance the performance of the algorithm's dynamic search; the information interaction strategy is to perform crossover between the newly generated individuals after the discussion operation and the original index individuals, resulting in two offspring individuals, and the greedy mechanism is applied to retain the best individual among the four individuals; the method of individual crossover is single-point crossover, that is, a crossover point is randomly generated within the dimension range, and the values after the crossover point of the two individuals are exchanged to generate two offspring individuals.
[0176] For the 2-opt algorithm in step S4.4, the details are as follows:
[0177] The nearest neighbor search is used to construct the initial loop, that is, each image location point is taken as the initial city, the city closest to the current city is selected from the remaining cities as the next visited city, and finally the initial city is returned to construct the initial solution set;
[0178] Execute the 2-opt strategy on the existing paths, that is, take edge exchange to locally modify each path in the initial solution set to further explore the solution;
[0179] Returns the optimal imaging path for the current imaging position.
[0180] In addition, the step S4.4 specifically includes:
[0181] Step S4.4.1, randomly initialize the population individuals, each individual is a 1*n vector, where each latitude value is between 0 and 1;
[0182] Step S4.4.2, reversely solve the 1*2n position information vector according to the mapping formula;
[0183] Step S4.4.3, using the 2-opt algorithm to calculate the path length according to the position information vector, and use it as the fitness of each individual;
[0184] Step S4.4.4: Update the intra-group discussion parameter P1 and the k-means category number parameter N k ;
[0185] Step S4.4.5, clustering the population in the target space using k-means clustering;
[0186] Step S4.4.6, generating a random number to determine whether to randomly initialize a cluster center;
[0187] Step S4.4.7, for each individual, generate a random number to determine whether the new individual is updated based on the intra-group discussion or the inter-group discussion;
[0188] Step S4.4.8, according to the information interaction strategy, cross the original index individual and the newly generated individual to generate two offspring individuals, calculate the individual fitness according to the 2-opt algorithm and apply greedy selection to retain the best individual among the four individuals;
[0189] Step S4.4.9, repeat steps S4.5.4-4.5.8 until the preset number of function evaluations is reached;
[0190] Step S4.4.10: Output the global optimal solution and iteration curve.
[0191] It should be emphasized that the neighborhood adjustment strategy of step S5 specifically includes:
[0192] Step S5.1, starting from the Mark point, consider a group of multiple adjacent imaging windows on the path as a neighborhood window group;
[0193] Step S5.2: Figure 7 As shown, it is evaluated whether the imaging task can be completed by connecting the line segment of the first and last imaging windows in the group, so as to further optimize the imaging path;
[0194] Step S5.3, repeat step S5.2 until all neighboring window groups are judged;
[0195] Step S5.4, repeating steps S5.2 to S5.3 until the imaging path cannot be optimized any further;
[0196] Step S5.5: output the final imaging path.
[0197] The imaging window path planning method in the automatic optical inspection system of the present invention is specifically described below in conjunction with a specific embodiment:
[0198] 1. The printed circuit board targeted by step S1 is composed of two PCBA boards spliced together, with a total width of 340 mm, a height of 220 mm, and 212 components to be tested;
[0199] 2. Step S3 specifically includes:
[0200] Step S3.1, parsing the Gerber file of the printed circuit board to be inspected to obtain the two-dimensional data of the components to be inspected. There are 212 components on the printed circuit board to be inspected. The initial parameters are as follows: the width W of the imaging window FOV is 40mm, and the height H FOV is 30mm, the number of individuals p is 50, the individual crossover probability Cr is 0.95, the number of individual selections N in the tournament is 5, and the number of iteration terminations i is 5;
[0201] Step S3.2, initialize the population individuals. For each individual X, randomly select the two-dimensional centers of 30 components to be detected as the imaging window centers. X is a 30*2 matrix, which represents the imaging position coordinates of the 30 imaging windows respectively.
[0202] Step S3.3: For the above initialization solution, use Figure 2 The iterative self-organizing algorithm shown performs initial solution optimization;
[0203] Step S3.4, for each individual, detection component division is performed, that is, some components covered by multiple windows are uniformly divided into a single window, which is not included in other window detection tasks;
[0204] Step S3.5, performing selection and crossover operations on the population to enhance information exchange between individuals;
[0205] Step S3.6, merging the imaging windows, which is consistent with the merging logic of the iterative self-organizing algorithm;
[0206] Step S3.7, repeat steps S3.5 to S3.6 until no optimal solution is generated;
[0207] Step S3.8: output the optimal imaging window group, which has 49 imaging windows in total.
[0208] 3. In the improved brainstorming optimization algorithm in step S4.4, P 1low is 0.2, P range is 0.6, N kup is 5, N range is 3, MaxFEs is 10000;
[0209] 4. Step S4.4 specifically includes:
[0210] Step S4.4.1, randomly initialize the population individuals. Since the optimal imaging window group obtained in step S3 has 49 imaging windows, based on this, each individual is a 1*49 vector, where each latitude value is between 0 and 1;
[0211] Step S4.4.2, reversely solve the position information vector of 1*2n, i.e. 1*98, according to the mapping formula;
[0212] Step S4.4.3, using the 2-opt algorithm to calculate the path length according to the position information vector, and use it as the fitness of each individual;
[0213] Step S4.4.4: Update the intra-group discussion parameter P1 and the k-means category number parameter N k ;
[0214] Step S4.4.5, clustering the population in the target space using k-means clustering;
[0215] Step S4.4.6, generating a random number to determine whether to randomly initialize a cluster center;
[0216] Step S4.4.7, for each individual, generate a random number to determine whether the new individual is updated based on the intra-group discussion or the inter-group discussion;
[0217] Step S4.4.8, according to the information interaction strategy, cross the original index individual and the newly generated individual to generate two offspring individuals, calculate the individual fitness according to the 2-opt algorithm and apply greedy selection to retain the best individual among the four individuals;
[0218] Step S4.4.9, repeat steps S4.5.4-4.5.8 until the preset number of function evaluations is reached;
[0219] Step S4.4.10, outputting the global optimal solution and iteration curve;
[0220] Step S5: Based on the imaging path solved by the hybrid framework, a neighborhood adjustment strategy is adopted to obtain an optimal imaging path;
[0221] The neighborhood adjustment strategy of step S5 specifically includes:
[0222] Step S5.1, starting from the Mark point, consider a group of multiple adjacent imaging windows on the path as a neighborhood window group;
[0223] Step S5.2: Figure 7 As shown, it is evaluated whether the imaging task can be completed by connecting the line segment of the first and last imaging windows in the group, so as to further optimize the imaging path;
[0224] Step S5.3, repeat step S5.2 until all neighboring window groups are judged;
[0225] Step S5.4, repeating steps S5.2 to S5.3 until the imaging path cannot be optimized any further;
[0226] Step S5.5, such as Fig. 9 As shown, the final imaging path is output, and the length of the final imaging path is 1551.6585 mm.
[0227] The imaging window path planning method in the automatic optical inspection system of the present invention has the following advantages, specifically:
[0228] 1. The present invention establishes two planning models, namely, the industrial camera imaging window division model and the moving path model, so as to optimize the problem in steps, reduce the complexity of the problem, improve the solving efficiency, and is not easy to fall into the local optimum, which is conducive to obtaining the global optimum;
[0229] 2. Compared with the prior art, the present invention firstly adopts an iterative self-organizing clustering algorithm to divide the imaging windows, and then combines the clustering algorithm with the swarm intelligence algorithm to propose an improved cross-iterative self-organizing clustering algorithm to minimize the number of imaging windows in order to solve the problem that the traditional clustering algorithm is prone to fall into the local optimum;
[0230] 3. Based on the AOI detection path planning problem, the present invention adopts a hybrid framework to simultaneously optimize the image acquisition position points and the access sequence, which is more precise and targeted;
[0231] 4. Considering that the optimization efficiency of the traditional brainstorming algorithm needs to be improved and it is difficult to find the optimal solution, in the improved brainstorming optimization algorithm, the probability of intra-group discussion and inter-group discussion of the original algorithm and the number of clustering categories are adaptively adjusted, so that the algorithm focuses on global search in the initial stage and local search in the later stage. In view of the shortcomings of specific post-processing strategies and traditional BSO algorithms, an improved step strategy is introduced to make individual evolution smoother and more efficient. In order to solve the problem that the original algorithm is slow in solving high-dimensional problems, the decision space clustering in the original algorithm is converted into target space clustering, which improves the solution speed without losing the search accuracy; in the greedy selection stage, a crossover operator is added to increase population diversity;
[0232] 5. In view of the fact that the optimal solution appears at the boundary of the neighborhood, the mapping method is used to significantly reduce the search space. The calculation process is simplified, and the algorithm converges to the optimal solution more focused and faster. By adopting this mapping strategy, the dimension of the decision variable is halved, which speeds up the optimization process and enhances the ability of the algorithm to quickly converge to a high-quality solution;
[0233] 6. Adopt the adjacent neighborhood search strategy based on the shortest path to further optimize the final imaging path and improve detection efficiency.
[0234] The above contents are only preferred embodiments of the present invention. For ordinary technicians in this field, according to the concept of the present invention, there will be changes in the specific implementation methods and application scopes. The content of this specification should not be understood as limiting the present invention.
Claims
1. A method for imaging window path planning in an automatic optical inspection system, characterized in that: The following steps are included, specifically: Step S1: firstly establish an industrial camera imaging window division model for the components to be inspected on the printed circuit board; Step S2: Based on the industrial camera imaging window division model in step S1, a moving path of the industrial camera between the imaging windows is modeled, and an imaging window moving path model is established; Step S3: for the industrial camera imaging window division model, an improved cross-iterative self-organizing algorithm is used to determine the minimum number of imaging windows for the industrial camera; Step S4: Design a hybrid algorithm framework for the above two-layer optimization model of the moving path. The framework uses an improved brainstorming algorithm to optimize the imaging position points of the industrial camera and a 2-opt algorithm to optimize the path sequence between the imaging position points. Step S5: Based on the imaging path solved by the hybrid framework, a neighborhood adjustment strategy is adopted to obtain the optimal imaging path.
2. The imaging window path planning method in an automatic optical inspection system according to claim 1, characterized in that: The overall model of the industrial camera imaging window division model in step S1 is: O = {1, 2, ..., m}; A={1,2,...,n}; Wherein, O is the set of components to be detected, and A is the set of imaging windows after planning; The objective function of the industrial camera imaging window division model is Min n, where n is the number of imaging windows; Introduce a Boolean variable, which is defined as: The constraints of the industrial camera imaging window division model include: In the constraint conditions of the industrial camera imaging window partition model, W j represents the width of the detection window j, H j represents the height of the detection window j, W FOV Indicates the width of the industrial camera's field of view, H FOV Indicates the height of the industrial camera's field of view.
3. The imaging window path planning method in an automatic optical inspection system according to claim 1, characterized in that: The overall model of the moving path model in step S2 is: Let G = (V, E) be a weighted graph, where the point set V consisting of the image position points of the industrial camera in each image window is expressed as: V = {1, 2, ..., n}; E is the set of edges; Introduce the Boolean variable Y pq , Y pq Defined as: The objective function of the mobile path model is: The constraints of the mobile path model include: In the constraints of the mobile path model, S is a subset of the set of point set V, and |S| is the number of elements in S; Represents the horizontal position coordinates of the industrial camera imaging point p, Represents the vertical position coordinate of the industrial camera imaging point p; is the lower limit of the movable range of the horizontal coordinate of position point p, It is the upper limit of the movable range of the horizontal coordinate of the position point p; is the lower limit of the movable range of the vertical coordinate of position point p, It is the upper limit of the movable range of the vertical coordinate of position point p.
4. The imaging window path planning method in an automatic optical inspection system according to claim 1, characterized in that: The specific process of the cross-iteration self-organizing algorithm in step S3 is as follows: Step S3.1, parsing the Gerber file of the printed circuit board to be inspected to obtain the two-dimensional data of the component to be inspected; accepting initial parameters, which include the width W of the imaging window FOV and height H FOV , number of individuals p, individual crossover probability Cr, number of individual selections in the tournament N, number of iteration terminations i, fitness function; Step S3.2, initialize the population individuals, and for each individual X, randomly select the two-dimensional centers of m components to be detected as the centers of the imaging windows, where X is an m*2 matrix, which respectively represents the imaging position coordinates of the m imaging windows; Step S3.3, using an iterative self-organizing algorithm to optimize the initial solution; Step S3.4: for each individual, divide the detection elements, that is, divide the parts of the elements covered by multiple windows into a single window; Step S3.5, performing selection and crossover operations on the population to enhance information exchange between individuals; Step S3.6, merging the imaging windows, which is consistent with the merging logic of the iterative self-organizing algorithm; Step S3.7, repeat steps S3.5 to S3.6 until no optimal solution is generated; Step S3.8, outputting the optimal imaging window group; Wherein, the step S3.5 specifically includes: Step S3.5.1, the selection operation uses the tournament method to retain excellent individuals, and uses the elite retention strategy to directly retain the best individuals of the previous generation to the next generation; Step S3.5.2: In the crossover operation, two individuals are randomly selected for gene recombination according to the crossover probability Cr to generate two new individuals; Step S3.5.3: Repeat steps S3.5.1 to S3.5.2 until the preset number of iterations is reached.
5. The imaging window path planning method in an automatic optical inspection system according to claim 4, characterized in that: The iterative self-organizing algorithm in step S3.3 is specifically as follows: Step S3.3.1, analyzing the spatial position relationship between the imaging position point and the component to be detected, removing the imaging window that does not contain the component to be detected in each body, and completing the imaging window for the uncovered component; then judging whether the imaging windows of each body can be merged by the following formula; in, and is the coordinate of the upper left corner of the actual detection area in window j, and is the coordinate of the lower right corner of the actual detection area in window j; F j is the set of components in window j; and is the upper left corner coordinate of component i in window j, and is the lower right corner coordinate of component i in window j; and is the coordinate of the upper left corner of the limit search area of window j, and is the coordinate of the lower right corner of the limit search area of window j; Step S3.3.2: If the actual detection area of imaging window u is covered by the limit search area of imaging window j, it means that the two imaging windows can be merged; the geometric center of the newly merged imaging window is determined by the following formula: in, and is the upper left corner coordinate of the new imaging window after the windows j and u are merged. and The lower right corner coordinate of the new imaging window after the windows j and u are merged; Step S3.3.3: After the windows are merged, determine the component information covered by each imaging window. If there is an imaging window whose covered components are a subset of the covered component set of the remaining windows, delete the redundant window.
6. The imaging window path planning method in an automatic optical inspection system according to claim 4, characterized in that: The step S3.5.2 specifically includes: Step S3.5.2.1, randomly generate an imaging window index in individual 1, and select all detection elements contained in the window; Step S3.5.2.2, searching in individual 2 for the detection elements selected in individual 1, including the remaining elements belonging to the same window of these detection elements; Step S3.5.2.3, return to individual 1, search for the detection elements selected in individual 2, including the remaining elements belonging to the same window of these detection elements; Step S3.5.2.4, repeat steps S2 and 3 until all the elements selected in the two individuals are the same; Step S3.5.2.5: Exchange the selected windows and components between the two individuals to form two new individuals.
7. The imaging window path planning method in an automatic optical inspection system according to claim 1, characterized in that: The step S4 specifically includes: Step S4.1, obtaining input parameters, the input parameters including two-dimensional data of the component to be detected, and step S3 solving the obtained imaging window position information; Step S4.2: Since the area of each imaging window is greater than or equal to its actual detection area, the adjustable area of each imaging window is calculated by the following formula; in, is the central horizontal coordinate of the adjustable area of window j, is the center ordinate of the adjustable area of window j, is the width of the adjustable area of window j, is the height of the adjustable area of window j; Step S4.3, since the shortest imaging path is based on the imaging of the adjustable domain boundary, a mapping method is used to concentrate the imaging position points of each imaging window on the adjustable domain boundary to avoid the internal space and improve the search efficiency. The mapping formula is: Given an adjustment domain, take clockwise as an example, where v i represents the i-th vertex in the adjustment domain, v k represents the vertex before the image position point x, and P represents the perimeter of the entire adjustment domain; through the above mapping formula, the position point x is encoded as the ratio of the length from the first vertex v1 along the boundary to x to the perimeter of the entire adjustment domain; each point on the boundary is represented by a single variable in [0,1), and any value in [0,1) corresponds to a unique image position point on the boundary; Step S4.4: Use the improved brainstorming optimization algorithm to solve the imaging position points on the adjustment domain of each imaging window, and use the 2-opt algorithm to find the access order between these position points based on the above imaging position points.
8. The imaging window path planning method in an automatic optical inspection system according to claim 7, characterized in that: The improved brainstorming optimization algorithm in step S4.4 is specifically as follows: In the process of improving the global search and local search of the brainstorming optimization algorithm, an adaptive adjustment strategy is introduced to conduct extensive global search in the early stage of the search process and focus on local search to accelerate the convergence of the algorithm in the later stage. The formula of the adaptive adjustment strategy is: P1=P 1low +P range *(FEs / MaxFEs); N k =floor(N kup -N range *(FEs / MaxFEs)); Among them, P 1low and P range are the lowest value and linear variation range of the probability of discussion within the group, N k N is the number of clustering categories used by the k-means clustering algorithm. kup and N range is the maximum number of categories and the linear range of the number of categories of the k-means clustering algorithm; MaxFEs is the maximum number of function evaluations, and FEs is the current number of function evaluations. floor() is a floor rounding function; In addition to selecting the class center and a random individual, the group discussion method introduces a method of randomly selecting two individuals to merge and generate a new individual to ensure that the algorithm fully explores the local area; Introduce the dynamic step size strategy, the dynamic step size strategy formula is: ξ d is the step size of the dth dimension, and is the value of the dth dimension of two individuals randomly selected from the population; The k-means clustering operation is transferred from the decision space to the one-dimensional target space to solve the problem that the computation time of the clustering algorithm grows exponentially with the dimension of the problem; An information interaction strategy is introduced to enhance the performance of the algorithm's dynamic search; the information interaction strategy is to perform crossover between the newly generated individuals after the discussion operation and the original index individuals, resulting in two offspring individuals, and the greedy mechanism is applied to retain the best individual among the four individuals; the method of individual crossover is single-point crossover, that is, a crossover point is randomly generated within the dimension range, and the values after the crossover point of the two individuals are exchanged to generate two offspring individuals.
9. The imaging window path planning method in an automatic optical inspection system according to claim 7, characterized in that: The step S4.4 specifically includes: Step S4.4.1, randomly initialize the population individuals, each individual is a 1*n vector, where each latitude value is between 0 and 1; Step S4.4.2, reversely solve the 1*2n position information vector according to the mapping formula; Step S4.4.3, using the 2-opt algorithm to calculate the path length according to the position information vector, and use it as the fitness of each individual; Step S4.4.4: Update the intra-group discussion parameter P1 and the k-means category number parameter N k ; Step S4.4.5, clustering the population in the target space using k-means clustering; Step S4.4.6, generating a random number to determine whether to randomly initialize a cluster center; Step S4.4.7, for each individual, generate a random number to determine whether the new individual is updated based on the intra-group discussion or the inter-group discussion; Step S4.4.8, according to the information interaction strategy, cross the original index individual and the newly generated individual to generate two offspring individuals, calculate the individual fitness according to the 2-opt algorithm and apply greedy selection to retain the best individual among the four individuals; Step S4.4.9, repeat steps S4.5.4-4.5.8 until the preset number of function evaluations is reached; Step S4.4.10: Output the global optimal solution and iteration curve.
10. The imaging window path planning method in an automatic optical inspection system according to claim 7, characterized in that: The neighborhood adjustment strategy of step S5 specifically includes: Step S5.1, starting from the Mark point, consider a group of multiple adjacent imaging windows on the path as a neighborhood window group; Step S5.2, evaluating whether the imaging task can be completed by connecting the line segment of the first and last imaging windows in the group, so as to further optimize the imaging path; Step S5.3, repeat step S5.2 until all neighboring window groups are judged; Step S5.4, repeating steps S5.2 to S5.3 until the imaging path cannot be optimized any further; Step S5.5: output the final imaging path.
Citation Information
Patent Citations
Cloud window autonomous perceptual decision-making method for effective planning of imaging missions
CN109284904A
AUV path planning method based on improved brainstorm optimization algorithm
CN112925317A
Part contour visual detection method based on local tracking path planning
CN115170517A
Rotor unmanned aerial vehicle autonomous path planning method based on Tiancattle swarm algorithm
CN115903888A
Scene detection view field generation and path planning method and application thereof
CN116861796A
Cited By
Detection camera path planning method and device, computer equipment and storage medium
CN121007845A
