Paper-based material displacement field estimation and image correction method in industrial visual inspection

Abstract

The invention relates to a paper-based material displacement field estimation and image correction method in industrial visual inspection, which comprises the following implementation steps of: constructing a deformation acquisition platform based on pure vision, acquiring deformation-related data of a paper-based material, and marking the data; constructing a first model for estimating the displacement field of the paper-based material and correcting the image; carrying out the training of the constructed first model; and deploying the trained first model as a paper-based material displacement field estimation and image correction model. According to the invention, the elastic deformation displacement of paper of different specifications generated by external force under the influence of different environmental parameters can be effectively estimated, and fine adjustment can be carried out according to actual image data acquired by a production line to obtain a more accurate result. The displacement field is used for solving geometric correction mapping, and a two-dimensional image after the paper-based material is stressed and deformed is mapped into a naturally tiled non-stressed image, so that the accuracy of visual inspection and positioning of the paper-based material is improved.

Description

Technical Field

[0001] This invention relates to the field of image correction technology, specifically to a method for displacement field estimation and image correction of paper-based materials in industrial visual inspection. Background Technology

[0002] Image-based intelligent defect detection and localization methods have been widely applied in industries such as aerospace, instrumentation manufacturing, and high-end equipment manufacturing. The core of industrial vision inspection and error localization lies in acquiring image data through machine vision systems, and then identifying defects and locating errors through geometric feature template matching or end-to-end deep learning models.

[0003] However, paper-based flexible materials are susceptible to nonlinear deformations such as stretching, wrinkling, and warping during processing due to mechanical tension and production line vibrations. This significantly reduces the reliability of visual algorithms or even causes them to fail. Furthermore, material deformation interference can amplify the actual positioning accuracy error by more than an order of magnitude compared to rigid scenarios.

[0004] Currently, there is a lack of image detection and localization methods in existing technologies that can effectively solve the above problems.

[0005] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to overcome the above-mentioned defects and provide a method for displacement field estimation and image correction of paper-based materials in industrial visual inspection, so as to solve the problem that under the influence of different environmental parameters, paper-based flexible materials will produce nonlinear deformation during processing, which will lead to a significant reduction in the reliability of visual algorithms or even failure.

[0007] To solve the above-mentioned technical problems, the technical solution provided by the present invention is as follows: a method for estimating the displacement field and correcting the image of paper-based materials in industrial visual inspection, comprising the following steps: S100, constructing a deformation acquisition platform based on pure vision, acquiring data related to the deformation of paper-based materials, and labeling these data; S200, based on the acquired data related to the deformation of paper-based materials, constructing a first model for estimating the displacement field of paper-based materials and correcting the image; S300, using a meta-learning training framework to train the constructed first model; S400, deploying the trained first model as a paper-based material displacement field estimation and image correction model, thereby estimating the displacement field of paper-based materials to achieve correction of the image features of paper-based materials.

[0008] Optionally, a vision-based deformation displacement acquisition platform is constructed to collect data related to deformation of paper-based materials and to annotate this data, including: S110, simulating the vibration of the production line using a frequency-adjustable precision vibration table; S120, acquiring the external force on the paper-based material sample using a fixture with a force gauge; S130, acquiring image data using a ring-shaped light source with an aperture and a high-frequency industrial camera, and acquiring environmental variables using a temperature and humidity sensor; S140, printing random speckle images on the paper-based flexible material sample; and S150, annotating the true value of the displacement field and establishing a training set using three-dimensional digital image technology.

[0009] Optionally, the step of constructing a first model for estimating the displacement field of the paper-based material and image correction based on the collected data related to the deformation of the paper-based material includes: S210, using a dynamic GNN framework to discretize the material surface into a mass-spring system, thereby constructing a dynamic graph of the deformation of the paper-based material; S220, based on the physical characteristics of the paper-based material during the deformation process, adjusting the elements related to the deformation of the paper-based material in the dynamic graph in real time to maintain the consistency between the graph structure and the deformation state of the paper-based material; S230, based on Newton's second law and the nodal motion state during the deformation process of the paper-based material, adjusting the core mechanical parameters of the mass-spring system in real time; S240, capturing the spatiotemporal characteristics of the simulated mass-spring system, constructing a ConvLSTM decoder, thereby converting the physical characteristics of the mass-spring system into displacement field prediction; S250, adjusting the predicted displacement field based on a preset multi-constraint loss function.

[0010] Optionally, the dynamic GNN framework is used to discretize the material surface into a mass-spring system, thereby constructing a dynamic graph of the deformation of the paper-based material, including: S211, defining the graph structure as... , where the nodes of the graph Characterize material particles and store location information and quality attributes ,side Characterizes the spring connection and stores constitutive parameter stiffness coefficients. and initial length , It is a dynamic adjacency matrix. It is a time variable;

[0011] S212, Stiffness coefficient modeled using physical quantities ,quality Damping It is expressed by the following formula:

[0012]

[0013]

[0014] in, For the cross-sectional area, For density, For thickness, For the damping ratio, It is the elastic modulus;

[0015] Among these methods, the elastic modulus is obtained by measuring the stress-strain curve of a standard material. The influence of environmental parameters on material properties is learned through a material parameter encoder, which is expressed by the following formula:

[0016] in, This represents a multilayer sensor, with parameters including temperature. ,humidity ; S213. Material particles are feature points of the image. For images with unknown patterns, adaptive grid spacing is set. Segment the image, To determine the image brightness, after obtaining the grid image, extract the point with the strongest feature within each grid as a node of the GNN. The displacement and velocity of the nodes are encoded into high-dimensional feature vectors;

[0017] S214. After obtaining the key point information of the image, the Delaunay triangulation network is constructed using the Bowyer-Watson algorithm. The triangulation network satisfies the empty circle property, which is expressed by the following formula:

[0018] In this case, the sides of the triangle are called sides. ,in It is a circumcircle. Given the corner points of a triangle, i.e., the mass points, construct the edge set accordingly. And the adjacency matrix A.

[0019] Optionally, the step of adjusting the elements related to the deformation of the paper-based material in the dynamic graph in real time based on the physical properties of the paper-based material during the deformation process to maintain the consistency between the graph structure and the deformation state of the paper-based material includes:

[0020] S221. Optimize real-time maintenance of edge sets through dynamic connections. Dynamic adjacency matrix When updating the graph, the edge is determined based on the strain energy. Whether to retain it or not is expressed by the following formula:

[0021] in, It is the minimum strain energy threshold. It is a node and nodes The current distance between them; S222, When the strain energy of the spring At this point, if the contribution of the edge to the system strain is considered negligible, the low-energy edge can be removed. And set the corresponding adjacency matrix item to 0; for isolated nodes without connections, remove the corresponding node. S223, If the velocity of a certain node If the threshold is exceeded, it indicates a drastic change in its motion state, and a new connection needs to be established around the node.

[0022] if The search radius is then dynamically adjusted based on the current speed, as expressed by the following formula:

[0023] in, Let the initial radius be , This is the preset adjustment coefficient;

[0024] If a node is found satisfy

[0025] Then add a new edge .

[0026] If the acceleration of a certain node Exceeding the threshold, along the direction of acceleration will The grid is split, new feature points are extracted, and new nodes are inserted. And by constructing a Delaunay triangulation network, a search radius is established. Connections to other nodes within the range; For a three-dimensional surface, a particle The curvature is shown in the following formula:

[0027] In the discrete model of the point mass spring model, it can be approximated by two-dimensional projection calculation, which is expressed by the following formula:

[0028] The numerator is the approximate second derivative, and the denominator is the product of the lengths of adjacent sides as a normalization factor.

[0029] Optionally, the real-time adjustment of the core mechanical parameters of the mass-spring system based on Newton's second law and the nodal motion state during the deformation process of the paper-based material includes:

[0030] The dynamic equations of the nodes are established based on Newton's second law, and are expressed by the following equation:

[0031] When the speed increases, it needs to be improved. To suppress high-frequency oscillations, it is expressed by the following formula:

[0032] At the same time, according to the adjustment edge stiffness It is expressed by the following formula:

[0033] in, It is an inertial force. For damping force, The elastic force calculated according to Hooke's Law. As an external force, among which, These are preset material dependence parameters. It is a characteristic velocity used to normalize the dynamic response of materials.

[0034] Optionally, capturing the spatiotemporal features of the simulated mass-spring system and constructing a ConvLSTM decoder to convert the physical features of the mass-spring system into displacement field prediction includes: The physical characteristics of nodal displacements and velocities simulated by the mass-spring model are spatiotemporal sequences. ,in For time steps, For the number of nodes, To capture both spatial locality and temporal dependence in the physical quantity dimension, a ConvLSTM is designed based on a convolutional gating mechanism. Multiple layers are stacked to reconstruct local strain details in the shallow layer and model global motion patterns in the deep layer. Hooke's law is embedded as a constraint into the LSTM's state update equation, and its output state... The recursive formula is:

[0035] in, This indicates element-wise multiplication. This represents the convolution operation. and For convolution kernel, Elastic force calculated for Hooke's Law The strain coefficient is... This represents the forget gate weights that are dynamically adjusted based on the node speed.

[0036] in, and It is the convolution kernel.

[0037] Optionally, adjusting the predicted displacement field based on a preset multi-constraint loss function includes:

[0038] The loss function is shown in the following equation:

[0039] in, To balance the weights, parameters are dynamically adjusted using a multi-task learning strategy;

[0040] in, As a task loss, the displacement field regression task is designed to account for the prediction error of displacement.

[0041] in, Represents a node The predicted displacement, The true value measured for DIC; The physical constraint loss is used to output a dynamic model that satisfies the mass-spring dynamics model through the loss constraint network:

[0042] in, and Through the Differential calculations; This represents the energy conservation loss, used to constrain damping dissipation to a reasonable value;

[0043] The kinetic energy is:

[0044] The elastic potential energy is:

[0045] Input function is:

[0046] This value is minimized, and the total energy loss is close to the damping dissipation. ; For regularization loss:

[0047] For sparsity constraints:

[0048] in, Wherein represents the indicator function.

[0049] Optionally, the step of using a meta-learning training framework to train the constructed first model includes: using a meta-learning training framework to learn the temporal changes of the dynamic graph, treating each time step as an independent task, learning shared parameters across tasks through historical information, optimizing global shared parameters in the outer loop, and fine-tuning local parameters in the inner loop for a single snapshot; the training process is divided into pre-training and fine-tuning processes, the network output is the predicted displacement field, the goal of pre-training is to understand the elastic deformation displacement law of paper of different specifications under external force under the current environmental parameters, and fine-tuning is to fine-tune the network using real data from the production line to improve the prediction accuracy for specific process links.

[0050] Optionally, deploying the trained first model as a paper-based material displacement field estimation and image correction model, and then estimating the displacement field of the paper-based material to correct the image features of the paper-based material, includes:

[0051] The pixel position relationship of the paper under force and without force is obtained from the displacement field predicted by the network, as expressed by the following formula:

[0052]

[0053] in, and The mapping relationship is achieved for the pixel positions of the naturally tiled paper using bilinear interpolation. By solving this problem, the stress image can be directly corrected to a naturally flat image, avoiding the influence of paper elastic deformation on the detection features and improving the accuracy of visual defect detection and positioning.

[0054] The advantages of this invention are: This invention presents a method for estimating the paper displacement field using a dynamic graph neural network. This method can effectively estimate the elastic deformation displacement of paper of different specifications under the influence of different environmental parameters and external forces. Furthermore, it can be fine-tuned based on actual image data collected from the production line to obtain more accurate results. By utilizing this displacement field solution geometric correction mapping, the two-dimensional image of the paper-based material after stress deformation is mapped to a naturally flat, stress-free image, thereby improving the accuracy of visual detection and positioning of the paper-based material. Attached Figure Description

[0055] Figure 1This is a flowchart of a method for estimating the displacement field and correcting the image of paper-based materials in industrial visual inspection, provided by an embodiment of the present invention. Figure 2 This is a sub-flowchart of step S100 provided in the embodiment of the present invention; Figure 3 This is a sub-flowchart of step S200 provided in the embodiment of the present invention; Figure 4 This is a schematic diagram of a paper-based material deformation acquisition platform based on pure vision provided in an embodiment of the present invention; Detailed Implementation

[0056] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses consistent with some aspects of this disclosure as detailed in the appended claims.

[0057] As mentioned earlier, image-based intelligent defect detection and localization methods have been widely applied in industries such as aerospace, instrumentation manufacturing, and high-end equipment manufacturing. The core of industrial vision inspection and error localization lies in acquiring image data through machine vision systems and then identifying defects and locating errors through geometric feature template matching or end-to-end deep learning models. However, paper-based flexible materials are susceptible to nonlinear deformations such as stretching, wrinkling, and warping during processing due to mechanical tension and production line vibrations, leading to a significant reduction in the reliability of vision algorithms or even their failure. Furthermore, material deformation interference can cause the actual positioning accuracy error to be magnified by more than an order of magnitude compared to rigid scenarios.

[0058] To address this issue, this invention proposes a method for estimating the displacement field and correcting images of paper-based materials in industrial visual inspection. By using a dynamic graph neural network to estimate the paper displacement field, this method can effectively estimate the elastic deformation displacement of paper of different specifications under various environmental parameters caused by external forces. Furthermore, it can be fine-tuned based on actual image data collected from the production line to obtain more accurate results. Using this displacement field solution geometric correction mapping, the two-dimensional image of the paper-based material after deformation under stress is mapped to a naturally flat, stress-free image, thereby improving the accuracy of visual inspection and positioning of paper-based materials.

[0059] The present invention solves the problem in the following way.

[0060] Example 1:

[0061] Please refer to the instruction manual appendix. Figures 1 to 4As shown in the figure, this embodiment provides a method for displacement field estimation and image correction of paper-based materials in industrial visual inspection. This method can be executed by an actuator and specifically includes the following steps:

[0062] S100. Construct a deformation acquisition platform based on pure vision to collect data related to deformation of paper-based materials and label these data.

[0063] In this embodiment, step S100 may further include: S110, using a frequency-adjustable precision vibration table to simulate the vibration of the production line.

[0064] S120. Obtain the external force on the paper-based material sample through a clamp equipped with a force gauge.

[0065] The S130 uses a ring light source with an aperture and a high-frequency industrial camera to acquire image data, and a temperature and humidity sensor to acquire environmental variables.

[0066] S140. Print random speckle images on paper-based flexible material samples.

[0067] S150. The true values ​​of the displacement field are labeled using three-dimensional digital image technology, and a training set is established.

[0068] S200. Based on the collected data related to the deformation of the paper-based material, a first model is constructed for estimating the displacement field of the paper-based material and for image correction.

[0069] In this first embodiment, step S200 may further include:

[0070] S210. Using a dynamic GNN framework, the material surface is discretized into a mass-spring system, thereby constructing a dynamic diagram of the deformation of the paper-based material.

[0071] In this first embodiment, step S210 may further include:

[0072] S211, Define the graph structure as follows , where the nodes of the graph Characterize material particles and store location information and quality attributes ,side Characterizes the spring connection and stores constitutive parameter stiffness coefficients. and initial length , It is a dynamic adjacency matrix. It is a time variable.

[0073] S212, Stiffness coefficient modeled using physical quantities ,quality Damping It is expressed by the following formula: (1) (2) (3) in, For the cross-sectional area, For density, For thickness, For the damping ratio (e.g., initialized to 0.05), It is the elastic modulus.

[0074] Among these methods, the elastic modulus is obtained by measuring the stress-strain curve of a standard material. The influence of environmental parameters on material properties is learned through a material parameter encoder, which is expressed by the following formula: (4) in, This represents a multilayer sensor, with parameters including temperature. ,humidity And other parameters that affect material properties.

[0075] S213. Material particles are the feature points of an image. For images with known patterns, feature points can be found through known patterns, such as the center position of the speckles in a random speckle pattern. For images with unknown patterns, adaptive grid spacing can be set. Segment the image, To determine the image brightness, after obtaining the grid image, extract the point with the strongest feature within each grid as a node of the GNN. The displacement and velocity of the nodes are encoded into high-dimensional feature vectors.

[0076] S214. After obtaining the key point information of the image, the Delaunay triangulation network is constructed using the Bowyer-Watson algorithm. The triangulation network satisfies the empty circle property, which is expressed by the following formula: (5) In this case, the sides of the triangle are called sides. ,in It is a circumcircle. Given the corner points of a triangle, i.e., the mass points, construct the edge set accordingly. And the adjacency matrix A.

[0077] S220. Based on the physical properties of the paper-based material during deformation, adjust the elements related to the deformation of the paper-based material in the dynamic diagram in real time to maintain the consistency between the diagram structure and the deformation state of the paper-based material.

[0078] In this first embodiment, step S220 may further include:

[0079] S221. Optimize real-time maintenance of edge sets through dynamic connections. Dynamic adjacency matrix When updating the graph, the edge is determined based on the strain energy. Whether to retain it or not is expressed by the following formula: (6) in, It is the minimum strain energy threshold (e.g., 0.1% to 1% of the total strain energy of the system). It is a node and nodes The current distance between them.

[0080] S222, When the strain energy of the spring At this point, if the contribution of the edge to the system strain is considered negligible, the low-energy edge can be removed. Then set the corresponding adjacency matrix item to 0, and remove the corresponding node for isolated nodes without connections.

[0081] S223, If the velocity of a certain node If the threshold is exceeded, it indicates a drastic change in its motion state, and a new connection needs to be established around the node.

[0082] if The search radius is then dynamically adjusted based on the current speed, as expressed by the following formula: (7)

[0083] in, Let the initial radius be , This is the preset adjustment coefficient;

[0084] If a node is found satisfy: (8) Then add a new edge .

[0085] If the acceleration of a certain node Exceeding the threshold, along the direction of acceleration will The grid is split, new feature points are extracted, and new nodes are inserted. And by constructing a Delaunay triangulation network, a search radius is established. Connections to other nodes within the range;

[0086] For a three-dimensional surface, a particle The curvature is shown in the following formula: (9)

[0087] In the discrete model of the point mass spring model, it can be approximated by two-dimensional projection calculation, which is expressed by the following formula: (10) The numerator is the approximate second derivative, and the denominator is the product of the lengths of adjacent sides, used as a normalization factor. If , representing the curvature of the region Significant changes indicate intense strain, possibly indicating the emergence of nonlinear strain regions such as wrinkles and bends. It is necessary to refine the mesh, add new nodes and edges, using the same method as when the acceleration exceeds the threshold.

[0088] S230, based on Newton's second law and the nodal motion state during the deformation process of paper-based materials, adjusts the core mechanical parameters of the mass-spring system in real time.

[0089] In this first embodiment, step S230 may further include: establishing the dynamic equation of the node according to Newton's second law, which is expressed by the following formula: (11)

[0090] in, It is an inertial force. For damping force, The elastic force calculated according to Hooke's Law. This is an external force. According to the damping force term, as the speed increases, it needs to be increased... To suppress high-frequency oscillations (12)

[0091] in, These are preset material dependence parameters. This is the characteristic velocity used to normalize the dynamic response of materials. Simultaneously, based on the adjusted edge... stiffness

[0092] (13)

[0093] S240. Capture the spatiotemporal features of the simulated mass-spring system, construct a ConvLSTM decoder, and thus transform the physical features of the mass-spring system into displacement field prediction.

[0094] In this embodiment, step S240 may further include: viewing the nodal displacements, velocities, and other physical characteristics simulated by the mass-spring model as a spatiotemporal sequence. ,in For time steps, For the number of nodes, To capture both spatial locality and temporal dependence in the physical quantity dimension, a ConvLSTM is designed based on a convolutional gating mechanism. Multiple layers are stacked to reconstruct local strain details in the shallow layer and model global motion patterns in the deep layer. Hooke's law is embedded as a constraint into the LSTM's state update equation, and its output state... The recursive formula is: (14)

[0095] in, This indicates element-wise multiplication. This represents the convolution operation. and For convolution kernel, Elastic force calculated for Hooke's Law The strain coefficient is... This represents the forget gate weights that are dynamically adjusted based on the node speed. (15)

[0096] in, and It is the convolution kernel.

[0097] S250. Adjust the predicted displacement field based on a preset multi-constraint loss function.

[0098] In this first embodiment, step S250 may further include: the loss function is shown in the following formula: (16)

[0099] in, To balance the weights, parameters are dynamically adjusted using a multi-task learning strategy. (17)

[0100] in, As a task loss, the displacement field regression task is designed to account for the prediction error of displacement.

[0101] (18) in, Represents a node The predicted displacement, This is the true value measured for DIC.

[0102] The physical constraint loss is used to output a dynamic model that satisfies the mass-spring dynamics model through the loss constraint network: (19)

[0103] in, and Through the Differential calculations;

[0104] This represents the energy conservation loss, used to constrain damping dissipation to a reasonable value.

[0105] (20)

[0106] The kinetic energy is: (twenty one)

[0107] The elastic potential energy is: (twenty two)

[0108] Input function is: (twenty three)

[0109] To minimize this value, division by zero should be avoided. The total energy loss is close to the damped dissipation. .

[0110] To mitigate the regularization loss, the constraint parameters are set to change gradually to avoid overfitting. (twenty four)

[0111] As a sparsity constraint, it is used to encourage the removal of low-energy edges as much as possible, making the dynamic graph as sparse as possible: (25)

[0112] in, Here, the indicator function is represented. The Sigmoid function is used during training, while the step function is used to simplify computation during inference.

[0113] S300. The first model is trained using a meta-learning training framework.

[0114] In this first embodiment, step S300 may further include: using a meta-learning training framework to learn the temporal changes of the dynamic graph, treating each time step as an independent task, learning shared parameters across tasks through historical information, optimizing global shared parameters in the outer loop, and fine-tuning local parameters in the inner loop for a single snapshot;

[0115] The training process is divided into pre-training and fine-tuning. The network output is the predicted displacement field. The goal of pre-training is to understand the elastic deformation displacement of different sizes of paper under external force under the current environmental parameters. Fine-tuning is to fine-tune the network using real data from the production line to improve the prediction accuracy for specific process steps.

[0116] S400. Deploy the trained first model as a paper-based material displacement field estimation and image correction model, and then estimate the displacement field of the paper-based material to achieve correction of the image features of the paper-based material.

[0117] In this first embodiment, step S400 may further include: obtaining the pixel position relationship between the paper under force and the paper when it is not under force based on the displacement field predicted by the network, as expressed by the following formula: (26) (27)

[0118] in, and The mapping relationship is achieved for the pixel positions of the naturally tiled paper using bilinear interpolation. By solving this problem, the stress image can be directly corrected to a naturally flat image, avoiding the influence of paper elastic deformation on the detection features and improving the accuracy of visual defect detection and positioning.

[0119] Therefore, in summary, the present invention and its embodiments have the following advantages over the prior art, including but not limited to:

[0120] This invention presents a method for estimating the paper displacement field using a dynamic graph neural network. This method can effectively estimate the elastic deformation displacement of paper of different specifications under the influence of different environmental parameters and external forces. Furthermore, it can be fine-tuned based on actual image data collected from the production line to obtain more accurate results. By utilizing this displacement field solution geometric correction mapping, the two-dimensional image of the paper-based material after stress deformation is mapped to a naturally flat, stress-free image, thereby improving the accuracy of visual detection and positioning of the paper-based material.

[0121] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0122] Furthermore, the terms "system" and "network" are often used interchangeably in this paper. The term "and / or" in this paper merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this paper generally indicates that the preceding and following related objects have an "or" relationship.

[0123] It should be understood that, in the embodiments of the present invention, "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean that B is determined solely based on A; B can also be determined based on A and / or other information.

[0124] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0125] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0126] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, apparatuses, or units, or they may be electrical, mechanical, or other forms of connection.

[0127] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0128] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0129] From the above description of the embodiments, those skilled in the art will clearly understand that the present invention can be implemented in hardware, firmware, or a combination thereof. When implemented in software, the above-described functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. Storage media can be any available medium accessible to a computer. For example, but not limited to, computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having the form of instructions or data structures and accessible to a computer. Furthermore, any connection can suitably be a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. As used in this invention, disk and disc include compressed optical discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically magnetically copy data, while discs optically copy data using lasers. The combinations described above should also be included within the scope of protection for computer-readable media.

[0130] In summary, the above description is merely a preferred embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for paper-based material displacement field estimation and image correction in industrial vision inspection, characterized in that, The method comprises the following steps: S100, a pure vision-based deformation collection platform is constructed, data related to deformation of paper-based materials are collected, and the data are labeled; S200, based on the collected data related to deformation of the paper-based materials, a first model for estimating displacement field of the paper-based materials and image correction is constructed; S300, a meta-learning training framework is used to train the constructed first model; S400, the trained first model is deployed as a paper-based material displacement field estimation and image correction model, and then the displacement field of the paper-based material is estimated to correct the image features of the paper-based material.

2. The method for paper-based material displacement field estimation and image correction in industrial vision inspection according to claim 1, characterized in that, A pure vision-based deformation displacement collection platform is constructed, data related to deformation of paper-based materials are collected, and the data are labeled, comprising: S110, a frequency-adjustable precision vibration table is used to simulate the vibration of the production line; S120, a clamp with a force gauge is used to obtain the external force on the paper-based material sample; S130, a ring-shaped light opening light source and a high-frequency industrial camera are used to collect image data, and a temperature and humidity sensor is used to obtain environmental variables; S140, a random speckle image is printed on the paper-based flexible material sample; S150, a three-dimensional digital image technology is used to label the displacement field true value and establish a training set.

3. The method for paper-based material displacement field estimation and image correction in industrial vision inspection according to claim 1, characterized in that, The first model for estimating displacement field of the paper-based materials and image correction is constructed based on the collected data related to deformation of the paper-based materials, comprising: S210, a dynamic GNN framework is used to discretize the material surface into a particle-spring system, and then a dynamic graph of paper-based material deformation is constructed; S220, based on the physical properties in the paper-based material deformation process, elements related to paper-based material deformation in the dynamic graph are adjusted in real time to maintain the consistency of the graph structure and the paper-based material deformation state; S230, based on Newton's second law and the node motion state in the paper-based material deformation process, the core mechanical parameters in the particle-spring system are adjusted in real time; S240, the spatiotemporal characteristics simulated by the particle-spring system are captured, a ConvLSTM decoder is constructed, and the physical characteristics of the particle-spring system are converted into displacement field prediction; S250, based on a pre-set multi-constraint loss function, the predicted displacement field is adjusted.

4. The method for paper-based material displacement field estimation and image correction in industrial vision inspection according to claim 3, characterized in that, The dynamic GNN framework is used to discretize the material surface into a particle-spring system, and then a dynamic graph of paper-based material deformation is constructed, comprising: S211, define the graph structure as wherein the nodes of the graph represent material particles, store position information and mass properties , edges represent spring connections, store constitutive parameters stiffness coefficient and initial length , is a dynamic adjacency matrix, is a time variable; S212, model the stiffness coefficients by physical quantities , mass , damping which are expressed by the following formula: ； ； ； wherein, is the cross-sectional area, is the density, is the thickness, is the damping ratio, is the modulus of elasticity; wherein the elastic modulus is obtained by determining the stress-strain curve of the standard material learning the influence of the environmental parameters on the material properties by a material parameter encoder, which is formulated by ； wherein, represents a multilayer perceptron, parameters include temperature , humidity ; S213、material particle is the feature point of image, for unknown image of pattern, through setting adaptive grid spacing Divide the image, For image brightness, after obtaining the grid image, extract a point with the strongest feature in each grid as the node of GNN Encode the displacement and speed of the node into a high-dimensional feature vector S214, after obtaining the key point information of the image, a Bowyer-Watson algorithm is used to construct a Delaunay triangulation network, and the triangular network satisfies the empty circle property, which is expressed by the following formula: ； wherein the edges of the triangle are edges wherein is the circumscribed circle, is the vertex of the triangle, i.e. the mass point, from which the edge set and the adjacency matrix A are constructed.

5. The method for paper-based material displacement field estimation and image correction in industrial vision inspection according to claim 3, characterized in that, Based on the physical properties in the paper-based material deformation process, elements related to paper-based material deformation in the dynamic graph are adjusted in real time to maintain the consistency of the graph structure and the paper-based material deformation state, comprising: S221. Optimize real-time maintenance of edge sets through dynamic connections. Dynamic adjacency matrix When updating the graph, the edge is determined based on the strain energy. Whether to retain it or not is expressed by the following formula: ； wherein, is a minimum strain energy threshold, is the current distance between nodes and nodes . S222, when the strain energy of the spring the edge is considered to have negligible contribution to the system strain, the low-energy edge can be removed and the corresponding adjacency matrix entry is set to 0, for disconnected nodes, the corresponding node is removed; S223、if the speed of a certain node exceeds a threshold value, it indicates that the motion state of the node changes dramatically, and the connection needs to be re-established around the node; ​ If then the search radius is dynamically adjusted based on the current speed, which is expressed by the following equation: ； wherein, is an initial radius, is a preset adjustment coefficient; If the node is found to satisfy ； then add new edges ; If the acceleration of a certain node Exceeding the threshold, along the direction of acceleration will The grid is split, new feature points are extracted, and new nodes are inserted. And by constructing a Delaunay triangulation network, a search radius is established. Connections to other nodes within the range; For a three-dimensional curved surface, the curvature of a point is given by the following equation: ； In the discrete model of the particle-spring model, it can be approximated as a two-dimensional projection calculation, which is expressed by the following formula: ； Wherein, the numerator is the approximate second derivative, and the denominator is the product of the adjacent side lengths as a normalization factor.

6. The method for paper-based material displacement field estimation and image correction in industrial vision inspection according to claim 3, characterized in that, Based on Newton's second law and the node motion state in the paper-based material deformation process, the core mechanical parameters in the particle-spring system are adjusted in real time, comprising: The dynamic equation of the node is established according to Newton's second law, which is expressed by the following formula: ； As the speed increases, the need to lift to suppress high frequency oscillations, which is expressed by the following formula: ； At the same time, the stiffness of the adjustment edge is expressed by the following formula: ​ ； wherein, is an inertial force, is a damping force, is an elastic force calculated according to Hooke's law, is an external force, wherein, is a pre-set material-dependent parameter, is a characteristic velocity for normalizing the material dynamic response.

7. The method for paper-based material displacement field estimation and image correction in industrial vision inspection according to claim 3, characterized in that, The space-time characteristics of the particle-spring system simulation are captured, and a ConvLSTM decoder is constructed, so as to convert the physical characteristics of the particle-spring system into displacement field prediction, including: The node displacement, velocity and other physical characteristics simulated by the visual point-spring model are space-time sequences wherein is a time step, is a node number, is a physical quantity dimension, ConvLSTM is designed based on a convolution gating mechanism to simultaneously capture spatial locality and time dependence, multi-layer stacking is performed, local strain details are reconstructed in a shallow layer, global motion patterns are modeled in a deep layer, and Hook's law is embedded in the state update equation of LSTM as a constraint, and the recursive formula of the output state is: ； wherein, represents element-wise multiplication, represents a convolution operation, and is a convolution kernel, is an elastic force calculated by Hooke's law, is a strain coefficient, represents a forget gate weight dynamically adjusted according to a node velocity: ； wherein and is a convolution kernel.

8. The method for paper-based material displacement field estimation and image correction in industrial vision inspection according to claim 3, characterized in that, The predicted displacement field is adjusted based on a preset multi-constraint loss function, including: The loss function is as follows: ； wherein, as the balancing weight, dynamically adjust parameters through multi-task learning strategy; ； wherein, is the task loss, the displacement field regression task is designed as the prediction error of the displacement amount; ； wherein, represents the predicted displacement of a node is the true value measured by DIC;​ For the physical constraint loss, the network output is constrained to satisfy the particle-spring dynamics model by this loss: ； wherein and By taking the derivative of the calculation; differential calculation; represents the energy conservation loss, used to constrain the damping dissipation to reasonable values; ； Where the kinetic energy is: ； The elastic potential energy is: ； The input work is: ； The total energy loss is close to the amount of damping dissipation for a minimum ; is the regularization loss: ； For sparsity constraint: ； wherein wherein denotes an indicator function.

9. The method for paper-based material displacement field estimation and image correction in industrial vision inspection according to claim 1, characterized in that, The first model is trained by using a meta-learning training framework, including: The meta-learning training framework is used to realize the learning of the time sequence change of the dynamic graph, each time step is regarded as an independent task, the shared parameters across tasks are learned through historical information, the global shared parameters are optimized in the outer loop, and the local parameters are fine-tuned for a single snapshot in the inner loop; The training process includes pre-training and fine-tuning, and the output of the network is the predicted displacement field, the pre-training target is to understand the elastic deformation displacement rule of different specifications of paper under external force in the current environment parameters, and the fine-tuning is to fine-tune the network using real data of the production line to improve the prediction accuracy for specific process links.

10. The method for paper-based material displacement field estimation and image correction in industrial vision inspection according to claim 1, characterized in that, The first model trained is deployed as a paper-based material displacement field estimation and image correction model, and then the displacement field of the paper-based material is estimated to correct the image features of the paper-based material, including: According to the displacement field predicted by the network, the pixel position relationship of the paper under stress displacement and without stress is obtained, which is expressed by the following formula: ； ； wherein, and are the natural tiled paper pixel positions, the mapping relationship is achieved by bilinear interpolation solving, that is, the stress image can be directly corrected to a natural tiled image, avoiding the influence of paper elastic deformation on the detection features, and improving the accuracy of visual defect detection and positioning.

Images