Method for positioning of multi-layer flexible material for lamination based on machine vision

Abstract

The present application relates to the technical field of image processing, and particularly relates to a positioning method for multi-layer flexible material lamination based on machine vision, which comprises the following steps: acquiring images of a first flexible material to be laminated and a second flexible material as a reference; extracting contour geometry and surface texture features in parallel for the images, and constructing first and second feature maps with features as nodes and spatial topological relationship and gradient mutual information as weighted edges; calculating an initial nonlinear deformation field of the first flexible material; guiding a graph matching algorithm by using the initial nonlinear deformation field to obtain a coarse matching feature point pair; and then solving a final non-rigid transformation model and generating positioning parameters containing non-uniform scaling and local distortion to guide the lamination operation. The present application can improve the quality and efficiency of the flexible material lamination process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a positioning method for multi-layer flexible material bonding based on machine vision. BACKGROUND

[0002] In high-end fields such as flexible electronics, smart wear, and composite material manufacturing, the accurate bonding of multi-layer flexible materials is a key process. Unlike rigid materials, flexible materials (such as films, fabrics, and flexible circuit boards) are prone to stretching, wrinkling, and curling during production, transportation, and processing due to their physical properties and external forces. This deformation is highly nonlinear, uncertain, and localized, making it difficult for traditional positioning methods to meet the high-precision alignment requirements of microns. Currently, some industries use mechanical positioning methods, such as pinholes, positioning pins, or clamps, but these methods are inefficient, have poor universality, and can cause physical damage to the material surface, limiting their application in high-end precision manufacturing. Therefore, non-contact, high-precision automated positioning using machine vision has become the mainstream direction of technological development.

[0003] Early machine vision positioning methods are mostly based on rigid or affine transformation models, using template matching or extracting global features (such as contours and center points) to calculate the position and attitude of the object. This method assumes that the target maintains a fixed geometric structure during deformation, which is only suitable for rigid objects or scenarios with minimal deformation, and cannot accurately describe the non-uniform scaling and local distortion of flexible materials, leading to significant positioning deviations in practical applications. To address this issue, researchers have begun to explore non-rigid registration techniques based on feature points. These methods typically extract local feature points (such as SIFT and SURF) from the material surface and fit a non-rigid transformation model (such as thin plate splines) through the matching relationship between feature points. However, existing methods still have significant shortcomings: first, they rely on a single type of feature, and when the material surface has sparse, repetitive, or missing textures, feature extraction is unstable and matching is not robust; second, when facing large-scale deformation, the search range for feature matching is large due to the lack of prior constraints on material physical properties, making it prone to false matches and resulting in model solving failures or poor accuracy; finally, the precision optimization of matching points is insufficient, making it difficult to achieve sub-pixel level, and the general transformation model cannot accurately express the inherent and complex local deformation patterns of materials. Therefore, developing a positioning method that can integrate multiple features, introduce physical priors, and achieve high-precision matching is crucial for improving the quality and efficiency of flexible material bonding processes. SUMMARY

[0004] To solve the technical problem of how to improve the quality and efficiency of flexible material bonding processes, the present application provides the following solutions.

[0005] A positioning method for multi-layer flexible material bonding based on machine vision, comprising the following steps: S1, acquiring images of a first flexible material to be bonded and a second flexible material as a reference; S2, extracting contour geometry and surface texture features in parallel for the images, and constructing first and second feature graphs with features as nodes and spatial topological relationship and gradient mutual information as weighted edges; S3, using a non-rigid transformation prior model constructed based on material constitutive relationship and historical deformation data, and combining global structural differences of the feature graphs, calculating an initial non-linear deformation field of the first flexible material; S4, using the initial non-linear deformation field to guide a graph matching algorithm, obtaining a coarse matching feature point pair; S5, for any coarse matching feature point pair, calculating a sub-pixel level phase correlation spectrum in its neighborhood, and when the peak signal-to-noise ratio and peak point curvature radius of the spectrum are greater than a preset threshold, determining it as a fine matching point pair; S6, based on all fine matching point pairs, taking the initial non-linear deformation field as an initial value, using a non-rigid optimization algorithm with regularization constraints, solving to obtain a final non-rigid transformation model, and generating positioning parameters containing non-uniform scaling and local distortion to guide the bonding operation.

[0006] Further, contour geometry and surface texture features are extracted in parallel for the images, and first and second feature graphs are constructed with features as nodes and spatial topological relationship and gradient mutual information as weighted edges, comprising: using an edge detection operator to extract the contour curve of the material, and locating the contour corner points of the contour curve through a Harris corner point detection algorithm or through a curve fitting algorithm to obtain contour geometry features; using a local feature descriptor to extract key points and texture information of the surface of the material as surface texture features; taking the contour corner points of the contour curve and the key points of the surface texture features as graph nodes, and defining the spatial topological relationship thereof by constructing the Delaunay triangulation between the nodes.

[0007] Further, the edge detection operator uses a Canny operator.

[0008] Further, the key points and texture information of the surface of the material are extracted as surface texture features using a local feature descriptor, comprising: using a SIFT algorithm to extract surface texture features.

[0009] Further, the non-rigid transformation prior model is a particle spring model describing the physical properties of the flexible material based on a finite element method.

[0010] Further, the initial non-linear deformation field of the first flexible material is calculated by taking the parameters of the constitutive relation of the material as input, using a machine learning model trained offline based on historical deformation data; quantifying the global structural difference between the first and second feature maps by comparing the spectral characteristics of the graph Laplacian matrices of the feature maps; taking the global structural difference as the query input of the machine learning model, and outputting a probabilistic displacement vector field as the initial non-linear deformation field.

[0011] Further, the machine learning model is a Gaussian process regression model.

[0012] Further, the step of guiding the graph matching algorithm using the initial non-linear deformation field to obtain the coarse matching feature point pairs includes: applying the initial non-linear deformation field to the node coordinates of the first feature map to perform pre-alignment; using a re-weighted random walk graph matching algorithm to perform matching between the pre-aligned first feature map and the second feature map, and establishing a coarse matching feature point pair when the matching confidence between nodes is higher than a preset matching threshold.

[0013] Further, determining the fine matching point pairs includes: taking the corresponding points in any coarse matching point pair as the center to extract local neighborhood image blocks on the images of the first and second flexible materials, respectively; calculating the sub-pixel level phase correlation spectrum between the two image blocks, and judging according to the quality evaluation indicators such as signal-to-noise ratio and curvature of the spectrum peak; when the quality evaluation indicators are better than the preset quality threshold, upgrading the coarse matching point pair to a fine matching point pair.

[0014] Further, the step of solving the final non-rigid transformation model includes: using a probabilistic point set registration algorithm as the non-rigid optimization algorithm; constructing an objective function including a data term and a regularization term, wherein the data term is used to minimize the registration error between all fine matching point pairs, and the regularization term is used to constrain the smoothness of the deformation field to maintain the physical coherence of the motion; solving the objective function by iterative optimization until the convergence condition is met, to obtain the final non-rigid transformation model.

[0015] Compared with the prior art, the present application effectively overcomes the defect that a single feature is not stable in positioning when the material texture is sparse or the contour is simple by extracting contour geometry and surface texture features in parallel and constructing feature graphs, and significantly enhances the universality and robustness of the method. More importantly, the present application creatively introduces a non-rigid transformation prior model based on the material constitutive relation and historical deformation data, which provides strong physical constraints and accurate initial guidance for feature matching under large-scale deformation, avoiding the matching failure or model distortion caused by the lack of prior knowledge in traditional methods. In addition, through a coarse-fine two-level matching strategy, especially using sub-pixel level phase correlation spectrum for accurate positioning, and combining with strict screening criteria, the high precision of the matching points is ensured. The finally solved non-rigid transformation model can accurately describe the complex non-uniform scaling and local distortion of flexible materials, thereby generating high-precision positioning parameters, providing reliable technical support for realizing high-quality lamination of multi-layer flexible materials. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a schematic diagram illustrating step S2 according to the present application;

[0017] Figure 2 is a schematic diagram illustrating step S4 according to the present application;

[0018] Figure 3 is a schematic diagram illustrating step S5 according to the present application;

[0019] Figure 4 is a schematic diagram illustrating the final non-rigid transformation model according to the present application. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0021] A positioning method for multi-layer flexible material lamination based on machine vision, comprising the following steps:

[0022] S1, acquiring images of a first flexible material to be laminated and a second flexible material as a reference;

[0023] Specifically, an industrial CCD camera with more than 5 million pixels can be used to collect high-resolution, non-reflective images of the first flexible material, such as flexible circuit board FPC, and the second flexible material, such as functional film, on the lamination table under a ring-shaped shadowless dome light source.

[0024] S2, extracting contour geometry and surface texture features in parallel for the images, and constructing first and second feature graphs with features as nodes and spatial topological relationship and gradient mutual information as weighted edges;

[0025] In an optional embodiment, contour geometry and surface texture features are extracted in parallel from the image to construct first and second feature maps with features as nodes and spatial topological relationships and gradient mutual information as weighted edges. This includes: extracting the contour curve of the material using an edge detection operator, and locating the endpoints, corners, or high curvature points of the contour curve using a Harris corner detection algorithm or a curve fitting algorithm to obtain contour geometry features; and extracting key points and their texture information on the material surface using a local feature descriptor as surface texture features.

[0026] The contour corners of the contour curve and key points on the material surface are used as nodes, and their spatial topological relationships are defined by constructing Delaunay triangulation between nodes. The connection relationships between nodes are constructed using the Delaunay triangulation algorithm to define the spatial topological relationships. For each edge connecting two nodes, the gradient direction histogram of its neighboring image patch is calculated, and the normalized gradient mutual information value between the two histograms is used as the edge weight, ultimately forming a weighted feature map.

[0027] The edge detection operator can be the Canny operator.

[0028] The key points and their texture information on the material surface are extracted using local feature descriptors as surface texture features, including: extracting surface texture features using the SIFT algorithm.

[0029] Specifically, for an image of a fabric material measuring 100 mm by 100 mm, the Canny edge detection operator is used to identify the fabric's contour curves. For example... Figure 1 The "feature extraction" shown refers to the extraction of the contour corner points of the contour curve and key points on the material surface (such as...). Figure 1 The extracted boundary pixels (represented as "surface texture points") are used as nodes. B-spline curves are fitted to obtain the corner points of the contour curve, including 4 endpoints and 20 curvature extrema. Simultaneously, local feature descriptors such as SIFT or SURF are used to detect unique texture patterns on the fabric surface, for example, identifying 250 stable keypoints and their corresponding 128-dimensional feature vectors.

[0030] The four endpoints, 20 curvature extrema, and 250 key points of the surface texture obtained in the preceding steps, totaling 274 points, are collectively defined as nodes in the atlas. Delaunay triangulation is performed on these 274 nodes to generate a mesh consisting of approximately 500 triangles, as shown below. Figure 1The illustrated "feature map construction". The grid composed of triangles not only defines the adjacency relationship between nodes, but also represents the geometric structure and topological morphology of the material in the initial state, thereby constructing the first feature map. Repeat the above process for the deformed material image to construct the second feature map for subsequent comparison and registration.

[0031] S3, using the non-rigid transformation prior model constructed based on the material constitutive relationship and historical deformation data, and combining the global structural difference of the feature map, calculating the initial nonlinear deformation field of the first flexible material;

[0032] Specifically, a particle spring model describing the physical properties of the flexible material (i.e. material constitutive relationship) is established in advance based on the finite element method as the non-rigid transformation prior model, and the Young's modulus, Poisson's ratio and other parameters of the non-rigid transformation prior model are calibrated by historical deformation image data statistics. By comparing the size difference of the minimum circumscribed rectangle of the point set composed of all nodes in the first and second feature maps, an affine transformation is obtained. Apply this affine transformation as an external load to the boundary nodes of the particle spring model. After the particle spring model is relaxed by energy minimization, the displacement of each node constitutes the initial nonlinear deformation field.

[0033] In an optional embodiment, calculating the initial nonlinear deformation field of the first flexible material comprises: taking the parameters of the material constitutive relationship as input, using a machine learning model trained offline based on historical deformation data; quantifying the global structural difference of the feature map by comparing the spectral properties of the graph Laplacian matrix of the first and second feature maps; taking the global structural difference as the query input of the machine learning model, and outputting a probabilistic displacement vector field as the initial nonlinear deformation field.

[0034] Specifically, the machine learning model can be a Gaussian process regression model. In the offline training phase, thousands of sets of deformation data of flexible materials under different tensile and torsional loads are generated by finite element simulation. Taking the material constitutive parameters such as Young's modulus of 1.5 gigapascal and Poisson's ratio of 0.4 as model input, the displacement field obtained by simulation as output, the Gaussian process regression model is trained, thereby establishing a prior model that can predict the deformation behavior of the material according to its properties.

[0035] Specifically, during actual measurements, the graph Laplacian matrices of the first and second feature maps are calculated separately. The top 10 smallest eigenvalues ​​of the two matrices are extracted, and the difference between them is calculated to form a 10-dimensional difference vector, describing the global structural difference of the material from its initial state to its deformed state. This 10-dimensional difference vector is then input into a pre-trained Gaussian process regression model. The Gaussian process regression model outputs an initial displacement field covering all map nodes; for example, it predicts that a node may move from coordinates 100,150 to around 105,162, and provides the confidence level of this prediction. This vector field containing displacement and uncertainty information is the initial nonlinear deformation field.

[0036] S4. Using the initial nonlinear deformation field guided map matching algorithm, coarse matching feature point pairs are obtained;

[0037] like Figure 2 As shown, graph 1 refers to the first feature graph, and graph 2 refers to the second feature graph. Specifically, the reweighted random walk graph matching algorithm, i.e., the RRWM algorithm, is used. For any node in the first feature graph, its predicted position in the second flexible material image is predicted using the initial nonlinear deformation field. Then, in the second feature graph, matching candidate points are searched only within a small search window at the predicted position, rather than performing a global search, to construct a matching affinity matrix, thereby guiding the RRWM algorithm to efficiently find reliable coarse matching feature point pairs.

[0038] In an optional embodiment, the step of obtaining coarse-matching feature point pairs using the initial nonlinear deformation field-guided graph matching algorithm includes: applying the initial nonlinear deformation field to the node coordinates of the first feature map for pre-alignment; employing a reweighted random walk graph matching algorithm to match between the pre-aligned first and second feature maps; and establishing coarse-matching feature point pairs when the matching confidence between nodes is higher than a preset matching threshold. Specifically, the initial nonlinear deformation field provides a predicted displacement for each node in the first feature map. For example, for all 274 nodes in the map, the deformation field gives their approximate target positions in the second image. All node coordinates in the first map are moved according to this predicted displacement to align them spatially with the second map. This pre-alignment step significantly narrows the search range of subsequent matching algorithms, improving matching efficiency and accuracy.

[0039] Specifically, after pre-alignment, a re-weighted random walk graph matching algorithm is employed to find the one-to-one correspondence between two atlas nodes. By simulating a random walk process on both atlases simultaneously, and iteratively updating the matching probability according to the similarity of the features and the consistency of the geometric structure between nodes, a matching confidence score between 0 and 1 is generated for each pair of possible matching nodes after, for example, 20 iterations. A matching threshold of 0.8 is set, and when the matching confidence of a pair of nodes is higher than this threshold, they are considered as a reliable matching pair. 210 pairs of coarse matching feature points are selected from the 274 nodes.

[0040] S5, for any coarse matching feature point pair, a sub-pixel level phase correlation spectrum is calculated in its neighborhood, and when the peak signal-to-noise ratio and the peak point curvature radius of the spectrum are greater than a preset threshold, it is determined as a fine matching point pair;

[0041] Specifically, in combination with Figure 3 For each coarse matching point pair, a neighborhood image block of, for example, 64x64 pixels is extracted on the original image with it as the center. Fourier transform is performed on the two image blocks, the mutual power spectrum is calculated, and then inverse Fourier transform is performed to obtain the phase correlation spectrum matrix. The peak region of the phase correlation spectrum matrix is fitted by two-dimensional Gaussian surface fitting, and the vertex coordinates of the fitted Gaussian function are used to determine the sub-pixel level accurate displacement. At the same time, the ratio of the peak height to the average noise of the spectrum plane is calculated as the peak signal-to-noise ratio, and the principal curvature of the peak point is calculated. Only when the signal-to-noise ratio is greater than 10 and the minimum curvature radius is greater than 2 pixels, the point pair is confirmed as a fine matching point pair.

[0042] In an optional embodiment, determining a fine matching point pair includes: extracting local neighborhood image blocks on the images of the first and second flexible materials with the corresponding points in any coarse matching point pair as the center; calculating the sub-pixel level phase correlation spectrum between the two image blocks, and judging according to the quality evaluation indicators such as the signal-to-noise ratio and the curvature of the spectrum peak; when the quality evaluation indicators are better than the preset quality threshold, the coarse matching point pair is upgraded to a fine matching point pair. Specifically, for any one of the 210 pairs of coarse matching point pairs obtained in the previous step, the system will extract an image block of, for example, 64x64 pixels in size with the coordinates of the matching points in the respective images as the center. The two image blocks contain rich local image information around the corresponding feature points, providing a basis for higher precision positioning.

[0043] S6, based on all fine matching point pairs, taking the initial nonlinear deformation field as the initial value, using a non-rigid optimization algorithm with regularization constraint, solving to obtain the final non-rigid transformation model, and generating positioning parameters containing non-uniform scaling and local distortion to guide the fitting operation.

[0044] In particular, the coherent point drift (CPD) algorithm is used as the non-rigid optimization algorithm. All the fine matching points on the first flexible material are used as the source point set, and the corresponding fine matching points on the second flexible material are used as the target point set. The initial non-linear deformation field calculated in step three is used as the displacement field for the Gaussian mixture model initialization in the CPD algorithm. During the iterative solving process of the algorithm, a smoothing regularization term is introduced to constrain the spatial gradient of the displacement field, ensuring the physical continuity of the transformation. After the algorithm converges, a dense displacement vector field is obtained, which describes the deformation of the entire material surface. This vector field is the final non-rigid transformation model. During the fitting process, the high-precision corresponding coordinates of any point on the fitting tool on the target material can be obtained by querying the displacement vector field, and the positioning parameters containing local stretching and twisting can be generated. Figure 4 The final non-rigid transformation model is shown.

[0045] In an optional embodiment, the step of solving the final non-rigid transformation model includes: using a probabilistic point set registration algorithm as the non-rigid optimization algorithm; constructing an objective function including a data term and a regularization term, wherein the data term is used to minimize the registration error between all fine matching point pairs, and the regularization term is used to constrain the smoothness of the deformation field to maintain the physical coherence of the motion; solving the objective function through iterative optimization until the convergence condition is met, obtaining the final non-rigid transformation model. In particular, the coherent point drift (CPD) algorithm is used in this step, which is a powerful probabilistic point set registration method. The algorithm constructs an objective function consisting of two parts. The data term is the sum of the squared Euclidean distances of all 185 fine matching point pairs after transformation, and the goal is to minimize this error term. The regularization term constrains the smoothness of the deformation field, for example, by modeling the space with a Gaussian radial basis function, penalizing discontinuous or highly variable deformations, and ensuring that the final transformation meets the physical properties of continuous deformation of flexible materials.

[0046] In particular, the objective function is minimized through iterative optimization strategies such as expectation maximization. In each iteration, the corresponding probabilities of the point pairs and the parameters of the non-rigid transformation are updated alternately. The iteration process continues until the value of the objective function converges, for example, the change between two iterations is less than one ten-thousandth, or the maximum number of iterations is reached, such as 100 times. When the optimization process converges, the set of optimal transformation parameters obtained constitutes the final non-rigid transformation model. This model can accurately describe the spatial mapping relationship of the material from the initial state to the deformed state, and can be used for subsequent deformation analysis or robot grasping applications.

Claims

1. A machine vision based positioning method for multi-layer flexible material tacking, characterized in that, The method comprises the following steps: S1, acquiring images of a first flexible material to be fitted and a second flexible material as a reference; S2, extracting contour geometry and surface texture features from the images in parallel, and constructing first and second feature graphs with features as nodes and spatial topological relations and gradient mutual information as weighted edges; S3, calculating an initial nonlinear deformation field of the first flexible material by using a non-rigid transformation prior model based on material constitutive relations and historical deformation data, and combining global structural differences of the feature graphs, including: taking parameters of the material constitutive relations as input, and adopting a machine learning model trained offline based on historical deformation data; quantifying the global structural differences of the graphs by comparing spectral characteristics of graph Laplacian matrices of the first and second feature graphs; taking the global structural differences as query input of the machine learning model, and outputting a probabilistic displacement vector field as the initial nonlinear deformation field; S4, guiding a graph matching algorithm by using the initial nonlinear deformation field to obtain a coarse matching feature point pair; S5, for any coarse matching feature point pair, calculating a sub-pixel level phase correlation spectrum in a neighborhood thereof, and determining a fine matching point pair when a peak signal-to-noise ratio of the spectrum and a peak point curvature radius are greater than preset thresholds; S6, based on all fine matching point pairs, taking the initial nonlinear deformation field as an initial value, and adopting a non-rigid optimization algorithm with a regularization constraint to solve a final non-rigid transformation model, and generating positioning parameters containing non-uniform scaling and local distortion to guide a fitting operation.

2. The method of claim 1, wherein, The method comprises the following steps: extracting contour curves of the materials by using an edge detection operator, and locating contour corner points of the contour curves by using a Harris corner point detection algorithm or a curve fitting algorithm to obtain contour geometry features; extracting key points and texture information of surfaces of the materials as surface texture features by using a local feature descriptor; and taking the contour corner points of the contour curves and the key points of the surface texture features as graph nodes, and defining spatial topological relations among the nodes by constructing Delaunay triangulation among the nodes.

3. The method of claim 2, wherein, The edge detection operator is a Canny operator.

4. The method of claim 2, wherein, The method comprises the following steps:

5. The method of claim 1, wherein, extracting key points and texture information of surfaces of the materials as surface texture features by using a local feature descriptor, including: extracting surface texture features by using a SIFT algorithm.

6. The method of claim 1, wherein, The non-rigid transformation prior model is a particle spring model describing physical properties of the flexible materials based on a finite element method.

7. The method of claim 1, wherein, The machine learning model is a Gaussian process regression model. The step of guiding a graph matching algorithm by using the initial nonlinear deformation field to obtain a coarse matching feature point pair comprises the following steps:

8. The method of claim 1, wherein, applying the initial nonlinear deformation field to node coordinates of the first feature graph to perform pre-alignment; and matching the pre-aligned first feature graph and the second feature graph by using a re-weighted random walk graph matching algorithm, and establishing a coarse matching feature point pair when matching confidence among nodes is higher than a preset matching threshold. The method comprises the following steps: Taking the corresponding point in any one of the coarse matching point pairs as the center, local neighborhood image blocks are extracted on the images of the first and second flexible materials respectively; a sub-pixel level phase correlation spectrum between the two image blocks is calculated, and a judgment is made according to quality evaluation indexes such as signal-to-noise ratio and curvature of the spectrum peak; when the quality evaluation indexes are better than preset quality threshold values, the coarse matching point pair is upgraded to a fine matching point pair.

9. The method of claim 1, wherein, The step of solving the final non-rigid transformation model comprises: A probabilistic point set registration algorithm is adopted as the non-rigid optimization algorithm; a target function comprising a data item and a regularization item is constructed, wherein the data item is used for minimizing registration errors between all fine matching point pairs, and the regularization item is used for constraining the smoothness of a deformation field to maintain the physical coherence of motion; the target function is solved by iterative optimization until a convergence condition is met, and a final non-rigid transformation model is obtained.

Images