Positioning method for lamination of multi-layer flexible material based on machine vision

Abstract

The invention relates to the technical field of image processing, in particular to a positioning method for lamination of multi-layer flexible materials based on machine vision, which comprises the following steps: acquiring images of a first flexible material to be laminated and a second flexible material serving as a reference; contour geometry and surface texture features are extracted in parallel for the image, and a first feature map and a second feature map with the features as nodes and the spatial topological relation and gradient mutual information as weighted edges are constructed; calculating an initial nonlinear deformation field of the first flexible material; utilizing the initial nonlinear deformation field guide graph matching algorithm to obtain rough matching feature point pairs; and further solving to obtain a final non-rigid transformation model, and generating positioning parameters including non-uniform scaling and local distortion to guide fitting operation. According to the invention, the quality and efficiency of the flexible material laminating process can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, in particular to a positioning method for laminating multi-layer flexible materials based on machine vision. Background Art

[0002] In high-tech fields such as flexible electronics, smart wearables, and composite materials manufacturing, the precise bonding of multiple layers of flexible materials is a critical process. Unlike rigid materials, flexible materials (such as films, fabrics, and flexible circuit boards) are prone to non-rigid deformations such as stretching, wrinkling, and curling during production, transportation, and processing due to their inherent physical properties and external forces. This deformation is highly nonlinear, uncertain, and localized, making it difficult for traditional positioning methods to meet the high-precision alignment requirements at the micron level. Currently, some industries rely on mechanical positioning methods, such as pinholes, dowel pins, or fixtures. However, these methods are not only inefficient and lack universal applicability, but are also prone to physical damage to the material surface, limiting their application in high-end precision manufacturing. Therefore, the use of machine vision for non-contact, high-precision automated positioning has become a mainstream technological development.

[0003] Early machine vision positioning methods were mostly based on rigid or affine transformation models, calculating the position and posture of an object through template matching or extracting global features (such as outlines and center points). These methods assume that the target maintains a fixed overall geometric structure during deformation. They are only applicable to rigid objects or scenes with minimal deformation. They cannot accurately describe the non-uniform scaling and local distortion of flexible materials, which can lead to severe positioning errors in practical applications. To address this issue, researchers have begun exploring feature point-based non-rigid registration techniques. These methods typically extract local feature points on the material surface (such as SIFT and SURF) and fit a non-rigid transformation model (such as thin plate spline functions) based on the matching relationship between feature points. However, existing methods still have significant shortcomings: First, they often rely on a single type of feature. When the material surface texture is sparse, repetitive, or missing, feature extraction is unstable and matching robustness is poor. Second, when faced with large-scale deformations, due to the lack of prior constraints on the material's physical properties, the feature matching search range is large, which is prone to mismatching, resulting in model solution failure or poor accuracy. Finally, the precision of matching points is insufficiently optimized, making it difficult to achieve sub-pixel accuracy, and general transformation models cannot accurately express the inherent and complex local deformation patterns of materials. Therefore, developing a positioning method that can integrate multi-source features, introduce physical priors, and achieve high-precision matching is crucial to improving the quality and efficiency of flexible material bonding processes. Summary of the Invention

[0004] In order to solve the technical problem of how to improve the quality and efficiency of the flexible material lamination process, the present invention provides the following solution.

[0005] A positioning method for laminating multi-layer flexible materials based on machine vision, comprising the following steps: S1, acquiring images of a first flexible material to be laminated and a second flexible material as a reference; S2, extracting contour geometry and surface texture features from the images in parallel, and constructing first and second feature maps with features as nodes and spatial topological relationships and gradient mutual information as weighted edges; S3, calculating the initial nonlinear deformation of the first flexible material using a non-rigid transformation prior model constructed based on material constitutive relationships and historical deformation data, combined with global structural differences of the feature maps. field; S4, using the initial nonlinear deformation field to guide the graph matching algorithm to obtain coarse matching feature point pairs; S5, for any coarse matching feature point pair, calculating the sub-pixel level phase correlation spectrum in its neighborhood, when the peak signal-to-noise ratio and the peak point curvature radius of the spectrum are greater than the preset threshold, it is determined to be a fine matching point pair; S6, based on all fine matching point pairs, with the initial nonlinear deformation field as the initial value, using a non-rigid optimization algorithm with regularization constraints, solve the final non-rigid transformation model, and generate positioning parameters including non-uniform scaling and local distortion to guide the fitting operation.

[0006] Furthermore, contour geometry and surface texture features are extracted in parallel for the image, and a first and a second feature map are constructed with features as nodes and spatial topological relationships and gradient mutual information as weighted edges, including: using an edge detection operator to extract the contour curve of the material, and locating the contour corners of the contour curve through a Harris corner detection algorithm or a curve fitting algorithm to obtain contour geometric features; using a local feature descriptor to extract key points of the material surface and their texture information as surface texture features; using the contour corners of the contour curve and the key points of the surface texture features as map nodes, and defining their spatial topological relationships by constructing a Delaunay triangulation between the nodes.

[0007] Furthermore, the edge detection operator adopts the Canny operator.

[0008] Furthermore, local feature descriptors are used to extract key points on the material surface and texture information thereof as surface texture features, including: using a SIFT algorithm to extract surface texture features.

[0009] Furthermore, the non-rigid transformation prior model is a mass-spring model based on the finite element method to describe the physical properties of the flexible material.

[0010] Furthermore, the initial nonlinear deformation field of the first flexible material is calculated, including: taking the parameters of the material's constitutive relationship as input, and adopting a machine learning model based on offline training of historical deformation data; quantifying the global structural difference of the graph by comparing the spectral characteristics of the graph Laplacian matrices of the first and second characteristic graphs; taking the global structural difference as the query input of the machine learning model, and outputting a probabilistic displacement vector field as the initial nonlinear deformation field.

[0011] Furthermore, the machine learning model is a Gaussian process regression model.

[0012] Furthermore, the step of using the initial nonlinear deformation field to guide the graph matching algorithm to obtain coarse matching feature point pairs includes: applying the initial nonlinear deformation field to the node coordinates of the first feature map for pre-alignment; using a reweighted random walk graph matching algorithm to match the first feature map and the second feature map after pre-alignment, and when the matching confidence between the nodes is higher than a preset matching threshold, establishing coarse matching feature point pairs.

[0013] Furthermore, determining a fine matching point pair includes: extracting local neighborhood image blocks on the images of the first and second flexible materials respectively with the corresponding point in any coarse matching point pair as the center; calculating the sub-pixel phase correlation spectrum between the two image blocks, and judging based on quality evaluation indicators such as the signal-to-noise ratio and curvature of the spectrum peak; when the quality evaluation indicator is better than a preset quality threshold, upgrading the coarse matching point pair to a fine matching point pair.

[0014] Furthermore, the step of solving the final non-rigid transformation model includes: adopting a probabilistic point set registration algorithm as the non-rigid optimization algorithm; constructing an objective function including a data term and a regularization term, wherein the data term is used to minimize the registration error between all pairs of precise matching points, and the regularization term is used to constrain the smoothness of the deformation field to maintain the physical coherence of the motion; solving the objective function through iterative optimization until the convergence condition is met, thereby obtaining the final non-rigid transformation model.

[0015] Compared with the existing technology, the present invention effectively overcomes the defect of unstable positioning of a single feature when the material texture is sparse or the contour is simple by extracting contour geometry and surface texture features in parallel and constructing a feature map, which significantly enhances the universality and robustness of the method. More importantly, the present invention innovatively introduces a non-rigid transformation prior model based on the material constitutive relationship and historical deformation data. This model provides strong physical constraints and precise initial guidance for feature matching under large-scale deformation, avoiding the matching failure or model distortion caused by the lack of prior knowledge in traditional methods. In addition, through a coarse and fine two-level matching strategy, especially the use of sub-pixel phase correlation spectrum for precise positioning, combined with strict screening criteria, the high precision of the matching points is guaranteed. The non-rigid transformation model finally solved can accurately describe the complex non-uniform scaling and local distortion of flexible materials, thereby generating high-precision positioning parameters, providing a reliable technical guarantee for achieving high-quality bonding of multi-layer flexible materials. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic diagram schematically illustrating step S2 according to the present invention; Figure 2 is a schematic diagram schematically illustrating step S4 according to the present invention; Figure 3 is a schematic diagram schematically illustrating step S5 according to the present invention; Figure 4 FIG. 1 is a schematic diagram schematically illustrating the final non-rigid transformation model according to the present invention. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0018] A positioning method for laminating multi-layer flexible materials based on machine vision comprises the following steps: S1, acquiring images of a first flexible material to be bonded and a second flexible material as a reference; Specifically, an industrial CCD camera with more than 5 million pixels can be used to capture high-resolution, non-reflective images of the first flexible material, such as a flexible circuit board FPC, and the second flexible material, such as a functional film, on the bonding table under an annular shadowless dome light source.

[0019] S2, extracting contour geometry and surface texture features of the image in parallel, and constructing first and second feature maps with the features as nodes and spatial topological relationships and gradient mutual information as weighted edges; In an optional embodiment, contour geometry and surface texture features are extracted in parallel for the image, and a first and a second feature map are constructed with features as nodes and spatial topological relationships and gradient mutual information as weighted edges, including: using an edge detection operator to extract the contour curve of the material, and locating contour corner points such as endpoints, corner points or high curvature points of the contour curve through a Harris corner detection algorithm or a curve fitting algorithm to obtain contour geometric features; using a local feature descriptor to extract key points on the material surface and their texture information as surface texture features.

[0020] The contour corners of the contour curve and key points on the material surface are used as nodes, and their spatial topological relationships are defined by constructing a Delaunay triangulation between the nodes. The Delaunay triangulation algorithm is used to construct the connection relationship between the nodes to define the spatial topological relationship. For each edge connecting two nodes, the gradient direction histogram of the neighboring image blocks is calculated, and the normalized gradient mutual information value between the two histograms is solved as the edge weight, ultimately forming a weighted feature map.

[0021] The edge detection operator may be the Canny operator.

[0022] Local feature descriptors are used to extract key points on the material surface and their texture information as surface texture features, including: using a SIFT algorithm to extract surface texture features.

[0023] Specifically, for a piece of fabric material image with a size of 100 mm x 100 mm, the Canny edge detection operator is used to identify the contour curve of the fabric. Figure 1 The “feature extraction” shown in the figure extracts the contour corner points of the contour curve and the key points of the material surface (such as Figure 1 The "surface texture points" (shown in the figure) are used as nodes. A B-spline curve is fitted to the extracted boundary pixels to obtain the corner points of the contour curve, including four endpoints and 20 curvature extreme points. Furthermore, local feature descriptors such as SIFT or SURF are used to detect the unique texture patterns on the fabric surface. For example, 250 stable key points and their corresponding 128-dimensional feature vectors are identified.

[0024] The 4 endpoints of the contour curve, 20 extreme curvature points, and 250 key points of the surface texture obtained in the above steps, a total of 274 points, are collectively defined as nodes of the graph. Delaunay triangulation is performed on the 274 nodes to generate a mesh consisting of approximately 500 triangles, as shown in Figure 1The "Feature Map Construction" shown in the figure. The triangular mesh not only defines the adjacency relationships between nodes but also characterizes the geometric structure and topology of the material in its initial state, thereby constructing the first feature map. This process is repeated for the deformed material image to construct the second feature map for subsequent comparison and registration.

[0025] S3, calculating the initial nonlinear deformation field of the first flexible material using a non-rigid transformation prior model constructed based on the material constitutive relationship and historical deformation data, and combining the global structural difference of the characteristic map; Specifically, a mass-spring model describing the physical properties of flexible materials (i.e., the material constitutive relations) is pre-established based on the finite element method as the non-rigid transformation prior model. Parameters of this non-rigid transformation prior model, such as Young's modulus and Poisson's ratio, are statistically calibrated using historical deformation image data. A macroscopic affine transformation is obtained by comparing the differences in the minimum bounding rectangle dimensions of the point sets formed by all nodes in the first and second feature maps. This affine transformation is applied as an external load to the boundary nodes of the mass-spring model. After the mass-spring model relaxes through energy minimization, the displacements generated by each node constitute the initial nonlinear deformation field.

[0026] In an optional embodiment, the initial nonlinear deformation field of the first flexible material is calculated, including: taking the parameters of the constitutive relationship of the material as input, and adopting a machine learning model trained offline based on historical deformation data; quantifying the global structural difference of the atlas by comparing the spectral characteristics of the graph Laplacian matrices of the first and second characteristic maps; taking the global structural difference as the query input of the machine learning model, and outputting a probabilistic displacement vector field as the initial nonlinear deformation field.

[0027] Specifically, the machine learning model can be a Gaussian process regression model. During the offline training phase, finite element simulation is used to generate thousands of sets of deformation data for flexible materials under different tensile and torsional loads. Constitutive parameters such as the material's Young's modulus of 1.5 gigapascals and Poisson's ratio of 0.4 are used as model inputs, and the simulated displacement field is used as output. The Gaussian process regression model is trained to establish a priori model that can predict the deformation behavior of the material based on its properties.

[0028] Specifically, during actual measurement, the graph Laplacian matrices of the first and second characteristic maps are calculated respectively. The first 10 smallest eigenvalues ​​of the two matrices are extracted, and the difference between them is calculated to form a 10-dimensional difference vector, which describes the global structural difference of the material from the initial state to the deformed state. The 10-dimensional difference vector is input into a pre-trained Gaussian process regression model. The Gaussian process regression model outputs an initial displacement field covering all map nodes. For example, it predicts that a node may move from coordinates 100,150 to around 105,162, and gives the confidence of the prediction. This vector field containing displacement and uncertainty information is the initial nonlinear deformation field.

[0029] S4, using the initial nonlinear deformation field guided graph matching algorithm to obtain coarse matching feature point pairs; like Figure 2 As shown in the figure, where Graph 1 refers to the first feature graph and Graph 2 refers to the second feature graph. Specifically, a reweighted random walk graph matching algorithm (RRWM) is employed. For any node in the first feature graph, the initial nonlinear deformation field is used to predict its predicted position in the second flexible material image. Then, in the second feature graph, matching candidate points are searched only within a small search window at the predicted position, rather than performing a global search. This is used to construct a matching affinity matrix, thereby guiding the RRWM algorithm to efficiently find reliable coarse matching feature point pairs.

[0030] In an optional embodiment, the step of using the initial nonlinear deformation field to guide the graph matching algorithm to obtain a coarse matching feature point pair includes: applying the initial nonlinear deformation field to the node coordinates of the first feature map for pre-alignment; using a reweighted random walk graph matching algorithm to match the pre-aligned first feature map and the second feature map, and establishing a coarse matching feature point pair when the matching confidence between the nodes is higher than a preset matching threshold. Specifically, the initial nonlinear deformation field provides a predicted displacement for each node in the first feature map. For example, for all 274 nodes in the map, the deformation field gives their approximate target position in the second image. All node coordinates of the first map are moved according to this predicted displacement so that they are spatially aligned with the second map. This pre-alignment step greatly narrows the search range of the subsequent matching algorithm and improves the efficiency and accuracy of matching.

[0031] Specifically, after pre-alignment, a reweighted random walk graph matching algorithm is used to find a one-to-one correspondence between the nodes of the two graphs. This algorithm simulates a random walk process performed simultaneously on the two graphs and iteratively updates the matching probability based on the similarity of the features and the consistency of the geometric structure between the nodes. After, for example, 20 iterations, a matching confidence score between 0 and 1 is generated for each pair of possible matching nodes. A matching threshold of 0.8 is set; when the matching confidence of a pair of nodes exceeds this threshold, they are considered to be a reliable match. 210 pairs of coarse matching feature points are screened from the 274 nodes.

[0032] S5, for any coarse matching feature point pair, calculate the sub-pixel phase correlation spectrum in its neighborhood, and determine it as a fine matching point pair when the peak signal-to-noise ratio and the peak point curvature radius of the spectrum are greater than a preset threshold; Specifically, combined Figure 3 For each coarse matching point pair, a neighborhood image block of, for example, 64x64 pixels is extracted from the original image with it as the center. The two image blocks are Fourier transformed, their cross-power spectrum is calculated, and then the inverse Fourier transform is performed to obtain the phase correlation spectrum matrix. By fitting a two-dimensional Gaussian surface to the peak area of ​​the phase correlation spectrum matrix, the vertex coordinates of the fitted Gaussian function are used to determine the precise displacement at the sub-pixel level. At the same time, the ratio of the peak height to the mean value of the spectrum plane noise is calculated as the peak signal-to-noise ratio, and the principal curvature of the peak point is calculated. Only when the signal-to-noise ratio is greater than 10 and the minimum curvature radius is greater than 2 pixels, the point pair is confirmed as a precise matching point pair.

[0033] In an optional embodiment, determining a precise matching point pair includes: extracting local neighborhood image blocks on the images of the first and second flexible materials, centered on the corresponding points in any coarse matching point pair; calculating the sub-pixel phase correlation spectrum between the two image blocks, and judging the quality based on quality evaluation indicators such as the signal-to-noise ratio and curvature of the spectrum peak; and upgrading the coarse matching point pair to a precise matching point pair when the quality evaluation indicator exceeds a preset quality threshold. Specifically, for any of the 210 pairs of coarse matching point pairs obtained in the previous step, the system will capture an image block of, for example, 64 pixels by 64 pixels, centered on the coordinates of the pair of matching points in their respective images. These two image blocks contain rich local image information around the corresponding feature points, providing a basis for more accurate positioning.

[0034] S6, based on all the precise matching point pairs, taking the initial nonlinear deformation field as the initial value, adopting a non-rigid optimization algorithm with regularization constraints to solve the final non-rigid transformation model, and generating positioning parameters including non-uniform scaling and local distortion to guide the fitting operation.

[0035] Specifically, the coherent point drift algorithm, namely the CPD algorithm, is used as a non-rigid optimization algorithm. All the precise matching points on the first flexible material are used as the source point set, and the corresponding precise matching points on the second flexible material are used as the target point set. The initial nonlinear deformation field calculated in step three is used as the displacement field for initializing the Gaussian mixture model in the CPD algorithm. During the iterative solution process of the algorithm, a smooth regularization term is introduced to constrain the spatial gradient of the displacement field to ensure the physical continuity of the transformation. After the algorithm converges, a dense displacement vector field that can describe the deformation of the entire material surface is obtained. This vector field is the final non-rigid transformation model. During fitting, according to the coordinates of any point on the fitting tool, the high-precision corresponding coordinates on the target material can be obtained by querying the displacement vector field, thereby generating positioning parameters containing local stretching and distortion. As Figure 4 The final non-rigid transformation model is shown.

[0036] In an optional embodiment, the step of solving the final non-rigid transformation model includes: employing a probabilistic point set registration algorithm as the non-rigid optimization algorithm; constructing an objective function comprising a data term and a regularization term, wherein the data term is used to minimize the registration error between all pairs of precisely matched points, and the regularization term is used to constrain the smoothness of the deformation field to maintain physical coherence of the motion; and solving the objective function through iterative optimization until convergence conditions are met, thereby obtaining the final non-rigid transformation model. Specifically, this step employs the Coherent Point Drift algorithm, a powerful probabilistic point set registration method. The algorithm constructs an objective function consisting of two parts. The data term is the sum of the squared Euclidean distances of all 185 pairs of precisely matched points after transformation, and the goal is to minimize this error term. The regularization term constrains the smoothness of the deformation field, for example, by modeling the space using Gaussian radial basis functions, penalizing discontinuous or drastic deformations to ensure that the final transformation conforms to the physical properties of continuous deformation of flexible materials.

[0037] Specifically, the objective function is minimized through iterative optimization strategies such as expectation maximization. In each iteration, the corresponding probabilities of the point pairs and the parameters of the non-rigid transformation are updated alternately. The iterative process continues until the value of the objective function converges, for example, the change between two iterations is less than one ten-thousandth, or a preset maximum number of iterations, such as 100, is reached. When the optimization process converges, the obtained set of optimal transformation parameters constitutes the final non-rigid transformation model. This model can accurately describe the spatial mapping relationship of the material from the initial state to the deformed state, and can be used for subsequent deformation analysis or robot grasping and other applications.

Claims

1. A positioning method for laminating multi-layer flexible materials based on machine vision, characterized in that: The following steps are involved: S1, acquiring images of a first flexible material to be bonded and a second flexible material as a reference; S2, extracting contour geometry and surface texture features of the image in parallel, and constructing first and second feature maps with the features as nodes and spatial topological relationships and gradient mutual information as weighted edges; S3, calculating the initial nonlinear deformation field of the first flexible material using a non-rigid transformation prior model constructed based on the material constitutive relationship and historical deformation data, and combining the global structural difference of the characteristic map; S4, using the initial nonlinear deformation field guided graph matching algorithm to obtain coarse matching feature point pairs; S5, for any coarse matching feature point pair, calculate the sub-pixel phase correlation spectrum in its neighborhood, and determine it as a fine matching point pair when the peak signal-to-noise ratio and the peak point curvature radius of the spectrum are greater than a preset threshold; S6, based on all the precise matching point pairs, taking the initial nonlinear deformation field as the initial value, adopting a non-rigid optimization algorithm with regularization constraints to solve the final non-rigid transformation model, and generating positioning parameters including non-uniform scaling and local distortion to guide the fitting operation.

2. The method according to claim 1, characterized in that Extracting contour geometry and surface texture features from the image in parallel, and constructing first and second feature maps with the features as nodes and spatial topological relationships and gradient mutual information as weighted edges, including: An edge detection operator is used to extract the contour curve of the material, and the contour corners of the contour curve are located through the Harris corner detection algorithm or the curve fitting algorithm to obtain the contour geometric features; a local feature descriptor is used to extract the key points of the material surface and its texture information as surface texture features; the contour corners of the contour curve and the key points of the surface texture features are used as graph nodes, and their spatial topological relationships are defined by constructing a Delaunay triangulation between the nodes.

3. The method according to claim 2, characterized in that The edge detection operator uses the Canny operator.

4. The method according to claim 2, characterized in that Local feature descriptors are used to extract key points on the material surface and their texture information as surface texture features, including: using a SIFT algorithm to extract surface texture features.

5. The method according to claim 1, wherein The non-rigid transformation prior model is a mass-spring model based on the finite element method to describe the physical properties of the flexible material.

6. The method according to claim 5, characterized in that Calculating the initial nonlinear deformation field of the first flexible material includes: taking parameters of the material's constitutive relationship as input and adopting a machine learning model trained offline based on historical deformation data; quantifying the global structural difference of the graphs by comparing the spectral characteristics of the graph Laplacian matrices of the first and second characteristic graphs; taking the global structural difference as a query input of the machine learning model, and outputting a probabilistic displacement vector field as the initial nonlinear deformation field.

7. The method according to claim 6, characterized in that The machine learning model is a Gaussian process regression model.

8. The method according to claim 1, characterized in that The step of obtaining a pair of coarse matching feature points by using the initial nonlinear deformation field guided graph matching algorithm comprises: The initial nonlinear deformation field is applied to the node coordinates of the first feature map for pre-alignment; a reweighted random walk graph matching algorithm is used to match the pre-aligned first feature map with the second feature map, and when the matching confidence between the nodes is higher than a preset matching threshold, a coarse matching feature point pair is established.

9. The method according to claim 1, characterized in that Determined as an exact matching point pair, including: Taking the corresponding point in any coarse matching point pair as the center, local neighborhood image blocks are extracted from the images of the first and second flexible materials respectively. The sub-pixel phase correlation spectrum between the two image blocks is calculated, and the quality evaluation indicators such as the signal-to-noise ratio and curvature of the spectrum peak are used to judge. When the quality evaluation indicators are better than the preset quality threshold, the coarse matching point pair is upgraded to a fine matching point pair.

10. The method according to claim 1, characterized in that The step of solving the final non-rigid transformation model includes: A probabilistic point set registration algorithm is adopted as the non-rigid optimization algorithm; an objective function including a data term and a regularization term is constructed, wherein the data term is used to minimize the registration error between all pairs of precisely matched points, and the regularization term is used to constrain the smoothness of the deformation field to maintain the physical coherence of the motion; the objective function is solved by iterative optimization until the convergence condition is met, thereby obtaining the final non-rigid transformation model.

Images