Non-contact micro-newton force measurement method and system

Abstract

The application provides a non-contact micro-newton level force measurement method and system, relates to the field of precision measurement and intelligent sensing technology, and the method comprises the following steps: a high-resolution industrial camera with a resolution of more than 10 million pixels is used to collect reference images and deformation images of the surface of a measured object under a preset optical illumination condition; sub-pixel deformation analysis is performed on the reference images and the deformation images; the sub-pixel displacement field and the full-field strain distribution obtained through analysis are input into a pre-trained physical information neural network inversion model to output the three-dimensional force field distribution of the surface of the measured object. The system comprises a high-resolution optical imaging module, an AI inference calculation module, an active temperature control module, an anti-vibration module and a self-adaptive calibration software module. The method and the system provided by the application can help to improve the micro-newton level force measurement precision, solve the technical difficulties of "difficult force perception and expensive sensors" in the micro-nano field, and fill the domestic technical blank of micro-newton level visual force measurement.

Description

Technical Field

[0001] This application relates to the fields of precision measurement and intelligent sensing technology, and in particular to a non-contact micro-Newton force measurement method and system. Background Technology

[0002] With the rapid development of precision manufacturing technology, the demand for precise measurement of minute forces (such as microNewton forces) is becoming increasingly urgent in fields such as semiconductor wafer fabrication, microelectromechanical systems (MEMS) manufacturing, and biological cell manipulation. MicroNewton forces refer to minute forces with the dimension of microNewtons (μN). MicroNewton force measurement technology is a core sensing foundation for achieving closed-loop control in precision manufacturing processes, ensuring product yield, and guaranteeing operational safety.

[0003] Currently, methods for measuring microNewton forces mainly rely on microforce sensors, atomic force microscopes (AFM), and digital image correlation (DIC). However, these methods suffer from problems such as insufficient accuracy in measuring microNewton forces, high cost of detection equipment, and lack of dynamic real-time microNewton force measurement capabilities. Summary of the Invention

[0004] This application provides a non-contact micro-Newton level force measurement method and system, which helps to solve the technical pain points of "difficulty in force sensing and high cost of sensors" in the micro-nano field and fills the gap in domestic micro-Newton level visual force measurement technology.

[0005] In a first aspect, embodiments of this application provide a non-contact micro-Newton force measurement method, comprising the following steps: S1: A high-resolution industrial camera with a resolution of ≥10 million pixels is used to acquire reference images and deformed images of the surface of the object under test under preset optical illumination conditions. The reference image is the image of the surface of the object under test when it is not under force, and the deformed image is the image of the surface of the object under test after it is deformed under force. S2: Perform subpixel deformation analysis on the reference image and the deformed image to obtain the subpixel displacement field and the full-field strain distribution, specifically including: S2a: Preprocess the reference image and the deformed image. The preprocessing includes grayscale normalization, noise suppression and illumination unevenness correction. S2b: A preset integer pixel matching algorithm is used to establish an initial correspondence between sub-regions between the preprocessed reference image and the deformed image to obtain the integer pixel displacement field; S2c: The sub-pixel displacement field is obtained by refining the calculation of the whole pixel displacement field using a preset sub-pixel positioning algorithm. S2d: Based on the subpixel displacement field, the full-field strain distribution on the surface of the object under test is calculated using a preset numerical differentiation method. S3: Input the subpixel displacement field and the full-field strain distribution into the pre-trained physical information neural network inversion model, and output the three-dimensional force field distribution on the surface of the object being measured.

[0006] In one possible implementation, the expression for the loss function L of the physical information neural network inversion model is: ; ; ; ; in, For data-driven loss terms; This refers to the residual loss term in the elasticity equilibrium equation; This is the constitutive relation residual loss term; The loss term is defined by boundary condition constraints. 、 、 、 The adaptive weighting coefficients are dynamically adjusted using an uncertainty weighting strategy or a gradient normalization strategy. Predicting displacement using a neural network; This is the measured displacement; For stress tensor; For strain tensor; It is the elastic stiffness tensor; This is the force vector; , M , K These represent the number of data points, collocation points, and boundary points, respectively.

[0007] One possible implementation involves training the physical information neural network inversion model using the following methods: A simulation dataset containing various typical mechanical loading scenarios was constructed. The simulation dataset includes displacement field data and stress field data under known force boundary conditions. The physical information neural network inversion model is pre-trained based on the simulation dataset to obtain the pre-trained model; Collect calibration experimental data in the target application scenario, and use transfer learning strategy to fine-tune the pre-trained model to obtain an inversion model adapted to the target application scenario; An adaptive learning rate scheduling strategy and an early stopping mechanism are adopted to optimize the training process of the inversion model to suit the target application scenario.

[0008] One possible implementation of the method also includes: Continuously acquire image sequences of the surface of the object under test at a frame rate of not less than 100 frames per second; Steps S2 and S3 are performed on each frame of the image sequence to obtain the real-time reconstructed three-dimensional force field distribution; The real-time reconstructed three-dimensional force field distribution is output to the external control system, providing closed-loop control feedback for the precision manufacturing process.

[0009] One possible implementation integrates a temperature sensor, a thermoelectric cooler, and a PID controller within the housing of a high-resolution industrial camera; the method also includes: Temperature sensors are used to collect measured temperatures from high-resolution industrial cameras. A PID controller is used to output a control signal based on the deviation between the measured temperature from the high-resolution industrial camera and the preset temperature. A thermoelectric cooler is used to perform temperature control according to the control signal, so that the operating temperature of the high-resolution industrial camera is within the preset temperature range.

[0010] One possible implementation of the method also includes: Obtain an initial reference image of the surface of the object under test when it is not under stress; Static reference images of the surface of the object under test when it is not under stress are collected at preset time intervals; Calculate the virtual displacement field between the static reference image and the initial reference image; If the amplitude of the virtual displacement field is greater than the preset threshold, the static reference image is updated to the reference image in step S1, and system drift compensation is performed on the subsequent displacement solution based on the virtual displacement field.

[0011] Secondly, embodiments of this application provide a non-contact micro-Newton force measurement system for implementing the non-contact micro-Newton force measurement method as described in the first aspect, including: A high-resolution optical imaging module is used to acquire high-resolution images of the surface of the object being measured. The high-resolution optical imaging module includes a high-resolution industrial camera with a resolution of ≥10 million pixels, a telecentric lens, and a uniform illumination source; or the high-resolution optical imaging module includes a high-resolution industrial camera with a resolution of ≥10 million pixels, a macro lens, and a uniform illumination source. The AI ​​inference computing module includes an embedded computing unit equipped with an AI acceleration chip, which is used to run sub-pixel deformation analysis algorithms and physical information neural network inversion models. An active temperature control module is used to control the temperature fluctuation of a high-resolution industrial camera within ±0.1℃. The active temperature control module includes a temperature sensor, a thermoelectric cooler, and a PID controller. The vibration isolation module is used to isolate the interference of environmental vibration on optical imaging. The vibration isolation module includes a first vibration isolation module and / or a second vibration isolation module. The first vibration isolation module includes an air-floating vibration isolation platform, and the second vibration isolation module includes a vibration sensor and a real-time compensation unit. The adaptive calibration software module is used to perform initial system calibration, periodic self-calibration, and over-the-air download and remote upgrade of physical information neural network inversion model parameters.

[0012] In one possible implementation, the high-resolution optical imaging module also includes a multi-camera stereo vision unit, which includes at least two high-resolution industrial cameras. The multi-camera stereo vision unit is used to acquire the three-dimensional shape information of the surface of the object being measured through a stereo matching algorithm.

[0013] In one possible implementation, the AI ​​acceleration chip adopts a heterogeneous computing architecture of field-programmable gate array (FPGA) and neural network processor. The FPGA is used for hardware acceleration of image preprocessing and sub-pixel deformation analysis algorithms; the neural network processor is used for inference operations of physical information neural network inversion model; the heterogeneous computing architecture supports pipelined parallel processing, so that the overall data processing throughput of the system is not less than 100 frames / second.

[0014] In one possible implementation, the adaptive calibration software module includes: Initial calibration unit: used to perform geometric calibration of the system using a standard micrometer-level grating ruler or a standard sample with known displacement, and to establish the mapping relationship between pixels and physical dimensions; Periodic self-calibration unit: used to periodically acquire static reference images of the surface of the object under test when it is not under force during system operation, and to detect and compensate for system drift caused by environmental changes; Scene adaptation unit: It is used to receive calibration experimental data of different application scenarios through the transfer learning interface, and use the calibration experimental data to update the parameters of the physical information neural network inversion model of the corresponding application scenario. The application scenarios include semiconductor wafer inspection, biological cell manipulation and precision mold processing.

[0015] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory, wherein the memory is used to store a computer program; and the processor is used to run the computer program to implement the non-contact micro-Newton force measurement method as described in the first aspect.

[0016] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the non-contact micro-Newton force measurement method as described in the first aspect.

[0017] The beneficial effects of this application are as follows: This application provides a non-contact micro-Newton (μN) level force measurement method and system. The method employs a high-resolution industrial camera with a resolution ≥10 megapixels to acquire reference and deformation images of the surface of the object under test under preset optical illumination conditions. Sub-pixel deformation analysis is performed on the reference and deformation images, achieving deformation detection accuracy down to 0.01 pixels. The obtained sub-pixel displacement field and full-field strain distribution are input into a pre-trained Physics-Informed Neural Network (PINN) inversion model, outputting the three-dimensional force field distribution of the object's surface. The PINN inversion model embeds the elasticity control equations, constitutive relations, and boundary conditions as physical constraints into a deep learning loss function, achieving accurate inversion from the deformation field to the force field, with force measurement accuracy down to the micro-Newton (μN) level. The system includes a high-resolution optical imaging module, an AI inference calculation module, an active temperature control module, a vibration resistance module, and an adaptive calibration software module, supporting millisecond-level data acquisition and real-time reconstruction of the three-dimensional force field. This application achieves non-contact micro-Newton-level force measurement by integrating sub-pixel deformation analysis technology and physical information neural network, thereby improving the accuracy of micro-Newton-level force measurement, solving the technical pain points of "difficulty in force perception and high cost of sensors" in the micro-nano field, and filling the gap in domestic micro-Newton-level visual force measurement technology. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating the non-contact micro-Newton force measurement method provided in this application embodiment; Figure 2 A schematic flowchart of subpixel deformation analysis provided in an embodiment of this application; Figure 3 A schematic diagram illustrating the network structure and loss function of the PINN inversion model provided in this application embodiment; Figure 4 This is a schematic diagram of the overall structure of the non-contact micro-Newton force measurement system provided in the embodiments of this application; Figure 5 This is a schematic diagram illustrating the application of this application in a semiconductor wafer bonding force detection scenario; Figure 6 This is a schematic diagram illustrating the application of this application in a biological cell manipulation force feedback scenario; Figure 7This is a schematic diagram illustrating the process of adapting transfer learning to multiple scenarios as provided in the embodiments of this application. Detailed Implementation

[0020] The embodiments of this application will be further described in detail below with reference to the accompanying drawings and examples. The detailed description of the following embodiments and the accompanying drawings are used to illustrate the principles of this application by way of example, but should not be used to limit the scope of this application. This application can be implemented in many different forms and is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

[0021] These embodiments are provided to make the application thorough and complete, and to fully express the scope of the application to those skilled in the art. It should be noted that, unless otherwise specifically stated, the relative arrangement of components and steps, material composition, numerical expressions, and values ​​illustrated in these embodiments should be interpreted as merely exemplary and not as limiting.

[0022] All terms used in this application have the same meaning as understood by one of ordinary skill in the art to which this application pertains, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant art, and not as idealized or highly formalized, unless expressly defined herein.

[0023] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0024] Currently, microNewton force measurement mainly relies on the following traditional technical solutions: (1) Micro-force sensor: This type of sensor converts force signals into electrical signals through physical contact. Although it has high accuracy, it has the following inherent defects: First, physical contact measurement is prone to damage and stress concentration to the micro-nano structure being measured, and is especially unsuitable for extremely sensitive objects such as semiconductor wafers and biological cells; Second, the manufacturing process of high-precision micro-force sensors is complex and expensive, and it is heavily dependent on imports, which restricts the independent control of the domestic high-end manufacturing industry chain; Third, traditional micro-force sensors are usually single-point measurements, and it is difficult to obtain information on the force distribution across the entire field.

[0025] (2) Atomic force microscope: Atomic force microscope can measure forces at the nanonewton level or even the piconewton level, but its measurement range is extremely small, usually at the micrometer level. The scanning speed is slow and cannot meet the real-time online detection needs of industrial production lines. Moreover, the equipment is expensive and the operation is complicated.

[0026] (3) Digital image correlation method: Traditional digital image correlation method can only measure geometric deformation and cannot directly invert mechanical quantities. It is also limited by pixel resolution, and its measurement accuracy is insufficient in scenarios with extremely small deformation caused by small forces. In addition, the mechanical inversion of traditional digital image correlation method usually relies on simple linear models, which are difficult to handle complex boundary conditions and nonlinear mechanical behavior.

[0027] In summary, the relevant technologies share the following common pain points: (a) "Difficulty in force sensing" in the micro-nano field, as contact sensors can easily damage the object being measured, and non-contact measurement methods lack sufficient accuracy; (b) "Expensive sensors," with high-precision micro-force sensors heavily reliant on imports, resulting in high costs; (c) Lack of dynamic real-time micro-Newton-level force measurement capabilities. Current solutions cannot simultaneously meet the comprehensive requirements of micro-Newton-level force measurement accuracy, non-contact measurement, and millisecond-level real-time response.

[0028] Based on the above problems, this application proposes a non-contact micro-Newton level force measurement method and system, which integrates sub-pixel deformation analysis with physical information neural networks to achieve non-contact micro-Newton level measurement, solves the technical pain points of "difficult force perception and expensive sensors" in the micro-nano field, and fills the gap in micro-Newton level visual force measurement technology in the industry.

[0029] like Figure 1 As shown, Figure 1 A flowchart illustrating the non-contact micro-Newton force measurement method provided in this application embodiment, specifically including the following steps: Step S1: Use a high-resolution industrial camera with a resolution of ≥10 million pixels to acquire reference images and deformed images of the surface of the object under test under preset optical illumination conditions.

[0030] The reference image is the image of the surface of the object under test when it is not under stress, and the deformation image is the image of the surface of the object under test after it is deformed under stress.

[0031] In this application, a high-resolution industrial camera with a resolution of ≥10 million pixels (which can be a CMOS camera or a CCD industrial camera, such as a CMOS camera with 4096×3072 pixels and a pixel size of 3.45μm) is used, along with a telecentric lens (the magnification can be adjusted within the range of 1x to 5x) or a high-magnification macro lens, and a uniform illumination source (such as an LED coaxial light source or a ring light source). A reference image of the surface of the object under test is acquired under preset optical illumination conditions when it is not under stress. When the object under test is subjected to external force causing micro-deformation of its surface, a deformed image of the object under stress is acquired. The preset optical illumination conditions refer to a standardized illumination acquisition environment where the position of the light source, the intensity of the light, the camera exposure parameters, and the shading environment remain constant during image acquisition, retaining only the deformation of the object under stress as the sole variable in the image grayscale change.

[0032] Optionally, a random speckle pattern can be pre-fabricated on the surface of the object under test, or the natural texture of the object's surface can be used as an image matching feature. The random speckle pattern or the object's natural texture are dedicated features for digital image correlation matching, used to establish pixel correspondence between an unstressed reference image and a stressed deformation image, enabling the solution of the global sub-pixel displacement field, and providing basic deformation data for subsequent strain calculation and force field inversion.

[0033] This application constructs image matching features by pre-fabricating random speckle or utilizing the natural texture of objects, ensuring that there is a significant gray-level gradient in each region within the imaging field of view, improving the stability of image correlation matching and the accuracy of sub-pixel solution, avoiding the problem of matching failure on smooth surfaces, and ensuring the measurement accuracy of minute deformations and weak contact forces.

[0034] Step S2: Perform subpixel deformation analysis on the reference image and the deformed image to obtain the subpixel displacement field and the full-field strain distribution.

[0035] In this step, subpixel-level deformation analysis is performed on the reference image and deformed image acquired in step S1 to achieve deformation detection with an accuracy of 0.01 pixels.

[0036] Specifically, Figure 2 This is a flowchart illustrating the subpixel deformation analysis provided in an embodiment of this application, as shown below. Figure 2 As shown, step S2 includes the following steps: S2a: Preprocess the reference image and the deformed image.

[0037] The preprocessing includes grayscale normalization, noise suppression, and illumination unevenness correction. Grayscale normalization is performed on the acquired reference and deformed images, normalizing the image grayscale values ​​to the [0, 1] range to eliminate the influence of illumination intensity fluctuations. Noise suppression is performed using Gaussian filtering or median filtering, such as using a 5×5 window Gaussian filter (standard deviation of 1.0) for smoothing and denoising. Illumination unevenness correction is performed using Top-hat transform or background fitting methods, such as using Top-hat transform with the structuring element radius set to 1 / 3 of the subset size to eliminate large-scale illumination gradients.

[0038] S2b: A preset integer pixel matching algorithm is used to establish an initial correspondence between sub-regions between the preprocessed reference image and the deformed image, thereby obtaining the integer pixel displacement field.

[0039] Among them, the integer pixel matching algorithm traverses the image search window with the whole pixel as the smallest search unit, and uses the gray-level correlation criterion to solve the initial integer displacement between the sub-regions of the image before and after deformation, providing the initial offset for subsequent sub-pixel deformation calculation.

[0040] Optionally, an integer pixel matching algorithm using Normalized Cross-Correlation (NCC), Zero-Mean Normalized Cross-Correlation (ZNCC), or Zero-mean Normalized Sum of Squared Differences (ZNSSD) can be used to establish an initial correspondence between sub-regions between the reference image and the deformed image, thereby obtaining an integer pixel-level displacement field.

[0041] For example, the reference image is divided into subsets (i.e., sub-regions) ranging in size from 21×21 pixels to 41×41 pixels, and the size of the sub-regions can be adaptively selected according to the speckle size and density. Using the ZNSSD correlation criterion as the matching metric, the best matching position of each subset is searched in the deformed image to obtain the whole pixel level displacement field.

[0042] The ZNSSD-related criteria are defined as follows: ; Where f and g are the gray values ​​of the reference image subset and the deformed image subset, respectively. and Let be the gray mean of the corresponding image subset, Δf, and To correspond to the grayscale standard deviation, Let N be the coordinates of the i-th pixel within the reference window, N be the total number of pixels within the matching window, and p be the displacement parameter vector.

[0043] S2c: The sub-pixel displacement field is obtained by refining the calculation of the integer pixel displacement field using a preset sub-pixel positioning algorithm.

[0044] Based on the integer pixel matching results, a sub-pixel localization algorithm based on Gaussian surface fitting or bicubic spline interpolation is adopted to fit the cross-correlation coefficient distribution in the neighborhood of the integer pixel matching point. The sub-pixel level displacement is determined by solving the extreme points of the fitted surface, and the displacement resolution is better than 0.01 pixels.

[0045] Among them, the sub-pixel localization algorithm based on Gaussian surface fitting or bicubic spline interpolation is a refined matching algorithm that, based on the initial integer displacement obtained by integer pixel matching, solves the precise position of the relevant extreme values ​​between discrete pixel grids by fitting the relevant peaks of two-dimensional Gaussian surface or reconstructing the continuous grayscale field by bicubic interpolation, and obtains the sub-pixel level displacement correction amount, thereby improving the accuracy of displacement solution.

[0046] For example, within the neighborhood of integer pixel matching points (typically within ±2 pixels), the ZNSSD correlation coefficient distribution is fitted using a Gaussian surface: ; Where (x,y) represents the pixel plane coordinates within the local image window, and C(x,y) is the fitted gray value at the corresponding coordinates; parameter B is the gray level of the image background, A is the peak amplitude of the Gaussian signal, and k is the Gaussian attenuation control coefficient. The extreme coordinates of the correlation coefficient surface are obtained using Gaussian surface least squares fitting. The sub-pixel offset of a single matched sub-region is obtained; this offset is then superimposed with the integer pixel matching result, and a global sub-pixel displacement field is generated after traversing all calculated sub-regions. This application achieves a displacement resolution consistently better than 0.01 pixels by optimizing the fitting window size and initial parameter estimation strategy.

[0047] This application employs a pre-defined sub-pixel positioning algorithm to perform refined calculations on the whole-pixel displacement field. By optimizing the interpolation kernel function and fitting strategy, the displacement resolution is better than 0.01 pixels, thereby enabling the detection of extremely small deformations caused by micro-Newton forces.

[0048] S2d: Based on the subpixel displacement field, the full-field strain distribution on the surface of the object under test is calculated using a preset numerical differentiation method.

[0049] In this step, based on the subpixel displacement field data, the full-field strain distribution on the surface of the object under test is calculated using a preset numerical differentiation method (such as Savitzky-Golay (SG) filter or finite difference). The numerical differentiation method, for discretely distributed subpixel displacement field data, approximates the displacement space gradient using finite difference or neighborhood fitting differentiation, and then solves for the normal strain and shear strain numerically.

[0050] For example, the subpixel displacement field (u, v) is smoothed using a Savitzky-Golay filter (window size 15×15, polynomial order 2), and then the strain components are calculated using finite difference. ; in, For the normal strain in the x-direction, The strain is the normal strain in the y-direction. Let be the shear strain in the xy plane.

[0051] Step S3: Input the subpixel displacement field and the full-field strain distribution into the pre-trained physical information neural network inversion model, and output the three-dimensional force field distribution on the surface of the object under test.

[0052] The sub-pixel displacement field data and full-field strain distribution obtained in step S2 are input into the pre-trained PINN inversion model. The core innovation of this PINN inversion model lies in: incorporating the elasticity governing equations... Constitutive relations (σ=C:ε) and boundary conditions are embedded as physical constraints in the loss function of the neural network, subjecting the network's training and inference processes to both data-driven and physical constraints. The PINN inversion model outputs the three-dimensional force field distribution on the surface of the measured object, including normal and tangential force components, achieving micro-Newton-level precision in non-contact force measurement.

[0053] In some embodiments, before performing the force measurement method (i.e., performing steps S1 to S3), a system preparation phase is first performed, including: (1) Create a random speckle pattern on the surface of the object being tested. For objects with smooth surfaces, such as semiconductor wafers, use vacuum evaporation or inkjet printing to create a high-contrast random speckle pattern on the surface. The speckle size is controlled within the range of 2 to 5 pixels, and the speckle coverage is controlled between 40% and 60% to ensure the uniqueness and accuracy of image matching. For objects with natural texture surfaces (such as certain processed surfaces), the natural texture can be directly used as a matching feature.

[0054] (2) High-resolution optical imaging system calibration. The imaging system is geometrically calibrated using a standard micrometer-level grating ruler (accuracy ±0.5μm) to establish the mapping relationship between the pixel coordinate system and the world coordinate system. For binocular stereo vision configuration, the intrinsic and extrinsic parameters of the two cameras and epipolar correction also need to be completed.

[0055] (3) Loading the PINN inversion model. Based on the material properties (elastic modulus, Poisson's ratio, etc.) and geometric parameters of the object under test, load the corresponding pre-trained PINN inversion model, or use the transfer learning interface for rapid scene adaptation.

[0056] In some embodiments, the network structure of the PINN inversion model includes: Input layer: used to receive the spatial coordinates (x, y) (two-dimensional case) or (x, y, z) (three-dimensional case) of each measurement point on the surface of the object being measured.

[0057] Hidden layers: contain at least 6 fully connected layers, each with no fewer than 64 neurons, using Tanh or Swish activation functions; optionally, residual connections are introduced into the hidden layers to accelerate the convergence of deep network training.

[0058] Output layer: For the two-dimensional case, output the predicted displacement components ( , ) and stress components ( , , For the three-dimensional case, the output is ( , , , , , ).

[0059] For example, such as Figure 3 As shown, Figure 3 This is a schematic diagram of the network structure and loss function of the PINN inversion model provided in this application embodiment. The PINN inversion model uses a fully connected deep neural network, which includes one input layer and eight hidden layers. Figure 3 Only hidden layers 1, 2, 3, 6, and 8 are shown, along with one output layer. The input layer receives the spatial coordinates (x, y) of the measurement point; each hidden layer contains 128 neurons, using the Swish activation function f(x) = x·sigmoid(x); a residual connection is introduced between layers 3 and 6; the output layer outputs the displacement components (u, v) and stress components (u, v). , , ).

[0060] The PINN inversion model outputs the stress components at various points on the surface of the measured object. Concentrated and distributed forces are obtained through integration calculations. ; Where A is the area of ​​the surface subjected to force, n is the unit vector of the surface normal to the surface subjected to force, and σ is the stress tensor obtained at the integration point (output from the global domain of the PINN model). Using Cauchy's surface force formula, the stress tensor is multiplied by the normal vector to obtain the distributed surface force t of each infinitesimal element on the surface.

[0061] In some embodiments, the expression for the loss function L of the physical information neural network inversion model is: ; The general tensor form for all losses is: ; ; ; in, This is a data-driven loss term used to measure the mean square error between the neural network-predicted displacement field and the measured subpixel displacement field. The residual loss term in the equilibrium equation of elasticity is achieved by arranging a large number of collocation points in the computational domain and calculating the partial derivatives using automatic differentiation techniques. This is the constitutive relation residual loss term, ensuring that the predicted stress-strain relationship satisfies the material constitutive equation; The boundary condition constraint loss term ensures that the prediction results satisfy the known force and / or displacement boundary conditions. 、 、 、 To adapt the weighting coefficients, they are dynamically adjusted using either uncertainty weighting or gradient normalization to balance the competing relationships between various losses. Predict displacement for a neural network (i.e., PINN); The measured displacement is obtained from the subpixel displacement field data obtained from the above subpixel deformation analysis; For stress tensor; For strain tensor; It is the elastic stiffness tensor; This is the force vector; , M , K These represent the number of data points, collocation points, and boundary points, respectively.

[0062] Optionally, for the scenario of inverting a small force field in a component under plane stress, this application expands the above tensor expression into a two-dimensional scalar component form and supplements the displacement and surface force coupled boundary residual term. The specific definitions of each loss are as follows: ; ; ; ; in, The partial derivatives are calculated by automatic differentiation for the equilibrium equations under plane stress conditions. The linear constitutive relation residuals are given, where E is the elastic modulus and ν is Poisson's ratio. The residuals are the displacement boundary and force boundary conditions, and P is the number of boundary sampling points. For the neural network at the boundary coordinates The displacement components predicted at the location; The known displacement components given for this boundary location; The predicted surface force (normal or tangential force) is calculated by combining the stress output by the network with the Cauchy surface force formula. The known surface force loads are preset for the boundary.

[0063] Optionally, an uncertainty-weighted method can be used to parameterize the weights of each loss term into learnable parameters, i.e. Furthermore, a regularization term log(σᵢ) is added to the loss function to enable the network to automatically balance the contributions of various losses.

[0064] In some embodiments, the training method for the physical information neural network inversion model includes: A simulation dataset containing various typical mechanical loading scenarios was constructed. The simulation dataset includes displacement field data and stress field data under known force boundary conditions. The physical information neural network inversion model is pre-trained based on the simulation dataset to obtain the pre-trained model; Collect calibration experimental data in the target application scenario, and use transfer learning strategy to fine-tune the pre-trained model to obtain an inversion model adapted to the target application scenario; An adaptive learning rate scheduling strategy and an early stopping mechanism are adopted to optimize the training process of the inversion model to suit the target application scenario.

[0065] Specifically, a simulation dataset containing various typical mechanical loading scenarios, such as concentrated force, distributed force, and contact force, is constructed. This dataset can be generated through finite element analysis (FEA) and includes displacement and stress field data under known force boundary conditions. The PINN inversion model is pre-trained using this simulation dataset to obtain a pre-trained model with general mechanical inversion capabilities. Calibration experimental data from the target application scenario is collected, and a transfer learning strategy is used to fine-tune the pre-trained model, quickly obtaining an inversion model adapted to the specific application scenario (i.e., the target application scenario). During the adaptation process, the parameters of the first few network layers are frozen, and only the last few layers and the output layer are fine-tuned. During the training of the inversion model adapted to the target application scenario, an adaptive learning rate scheduling strategy (such as cosine annealing or adaptive moment estimation Adam optimizer combined with learning rate decay) and an early stopping mechanism are used to optimize the training process, ensuring that the inversion error of the trained model is less than 5%.

[0066] In this application, the Adam optimizer with adaptive moment estimation is used to update the gradient of network parameters. An adaptive learning rate scheduling strategy in the form of cosine annealing or learning rate decay is configured to dynamically adjust the iteration step size during the training process, which accelerates the convergence speed in the early stage and finely approximates the optimal solution in the later stage. An early stopping mechanism is introduced to monitor the change of validation set loss. When the validation loss has not been optimized for several consecutive rounds, the training is terminated in advance to suppress model overfitting and optimize the training stability and solution accuracy of the physical information neural network as a whole.

[0067] For example, the training process of the PINN inversion model can be divided into two stages: Phase 1: Simulation Data Pre-training (1) Use the finite element analysis software ABAQUS to establish a parametric finite element model of the object under test. The material parameters cover the range of common engineering materials, with elastic modulus of 1 GPa to 200 GPa and Poisson's ratio of 0.1 to 0.49. (2) Design a variety of typical loading scenarios: concentrated force loading (range of 0.1μN~1000μN), uniform force loading, gradient force loading, multi-point contact force loading, etc., and generate a total of 5000 sets of simulation data; (3) Add Gaussian noise to the simulation data, with a signal-to-noise ratio (SNR) of 30dB to 60dB, to enhance the noise robustness of the model; (4) Use the Adam optimizer to set the initial learning rate. Cosine annealing scheduling is used to train the PINN network for 10,000 training rounds with a batch size of 256. 10,000 collocation points are evenly distributed in the computation domain to calculate the physical residuals.

[0068] Phase Two: Fine-tuning of Transfer Learning (1) For specific application scenarios, such as semiconductor wafer bonding force detection, collect 20 to 50 sets of actual calibration experimental data; (2) Freeze the parameters of the first 6 layers of the PINN network. These layers learn the general mechanical feature representation. Only fine-tune the parameters of the last 2 layers and the output layer. (3) Use a smaller learning rate Perform fine-tuning training for 500-1000 rounds, combined with an early stopping strategy, and stop if the validation set loss does not decrease for 50 consecutive rounds; (4) The inversion error of the fine-tuned model in the target scene is less than 5%, and the adaptation process only takes a few minutes to tens of minutes.

[0069] Current micro-force visual measurement methods suffer from low processing speed, delayed data output, and significant delays in 3D force field reconstruction. They can only perform offline static detection and cannot provide real-time force signal feedback to external precision control systems such as machine tools and displacement stages. Therefore, they cannot adapt to the real-time closed-loop control issues in precision machining. Based on this, this application further proposes: Continuously acquire image sequences of the surface of the object under test at a frame rate of not less than 100 frames per second; Steps S2 and S3 are performed on each frame of the image sequence to obtain the real-time reconstructed three-dimensional force field distribution; The real-time reconstructed three-dimensional force field distribution is output to the external control system, providing closed-loop control feedback for the precision manufacturing process.

[0070] In this embodiment, image sequences of the surface of the object under test are continuously acquired at a frame rate of not less than 100 frames per second. Subpixel deformation analysis and PINN mechanical inversion are performed on each frame of the image sequence. The real-time three-dimensional force field distribution data reconstructed from each frame of the image is transmitted to external control systems such as machine tools and precision displacement stages through a high-speed communication interface. This allows the external control system to dynamically adjust the machining parameters (such as feed rate and pressure) based on the real-time feedback of normal and tangential micro-force data, forming a closed-loop control system of "image acquisition - deformation analysis - force field inversion - machining parameter adjustment".

[0071] To stabilize the temperature of the industrial camera and improve the accuracy of force detection, this application further proposes: A temperature sensor, thermoelectric cooler, and PID controller are integrated into the housing of a high-resolution industrial camera. Temperature sensors are used to collect measured temperatures from high-resolution industrial cameras. A PID controller is used to output a control signal based on the deviation between the measured temperature from the high-resolution industrial camera and the preset temperature. A thermoelectric cooler is used to perform temperature control according to the control signal, so that the operating temperature of the high-resolution industrial camera is within the preset temperature range.

[0072] In this embodiment, a temperature sensor, a thermoelectric cooler, and a PID controller are integrated within the housing of the high-resolution industrial camera. The temperature sensor monitors the operating temperature of the industrial camera in real time with a temperature measurement accuracy of ±0.01℃. The PID controller outputs a control signal based on the deviation between the measured temperature and the preset temperature, driving the thermoelectric cooler to perform bidirectional temperature control. This controls the operating temperature of the industrial camera within the range of ±10℃ of the ambient temperature, maintaining a temperature control accuracy of ±0.1℃. This eliminates focal length drift and image sensor pixel shift caused by thermal expansion and contraction of the camera, avoids sub-pixel displacement calculation errors caused by temperature disturbances, and improves the long-term stability of subsequent deformation inversion and three-dimensional force field measurement results.

[0073] To update the reference image and compensate for system drift, this application further proposes: Obtain an initial reference image of the surface of the object under test when it is not under stress; Static reference images of the surface of the object under test when it is not under stress are collected at preset time intervals; Calculate the virtual displacement field between the static reference image and the initial reference image; If the amplitude of the virtual displacement field is greater than the preset threshold, the static reference image is updated to the reference image in step S1, and system drift compensation is performed on the subsequent displacement solution based on the virtual displacement field.

[0074] In this embodiment, the system automatically performs periodic self-calibration at preset time intervals (e.g., 30 minutes). During each self-calibration, a static reference image in the current state is acquired, and the spurious displacement field between the static reference image and the initial reference image of the system is calculated. It is determined whether the amplitude of the spurious displacement field exceeds a preset threshold (e.g., 0.005 pixels). When the spurious displacement is determined to exceed the limit, the static reference image acquired this time is automatically updated to the new reference image of the system, and the system drift compensation is performed on the subsequent displacement solution results based on the calculated spurious displacement field. This reduces the system drift error caused by long-term operation and ensures the accuracy of sub-pixel displacement solution and the long-term measurement accuracy of three-dimensional force field.

[0075] In summary, the non-contact micro-Newton force measurement method provided in this application has the following technical advantages: (1) Ultra-high precision measurement: Using a high-resolution industrial camera with ≥10 million pixels and a sub-pixel deformation analysis algorithm, deformation detection at the 0.01 pixel level is achieved. Combined with PINN to construct a mechanical inversion model, the force measurement accuracy reaches the μN level. Compared with the traditional DIC technology, the force measurement accuracy is improved by more than 3 times.

[0076] (2) Non-contact measurement: Force is measured indirectly through optical imaging, completely avoiding physical contact. This solves problems such as easy damage to micro and nano structures and stress concentration, and is particularly suitable for extremely sensitive scenarios such as semiconductor wafer processing and biological cell manipulation.

[0077] (3) Dynamic real-time perception: The system supports millisecond-level data acquisition and processing with an end-to-end delay of less than 10ms. It can reconstruct the three-dimensional force field distribution in real time, providing closed-loop control feedback for precision manufacturing and filling the gap in dynamic micro-Newton force measurement technology.

[0078] (4) Physical-data dual-drive: The PINN model embeds physical laws into the deep learning framework, which has stronger generalization ability and physical consistency compared with the pure data-driven model. The inversion error is less than 5%, and the amount of training data required is significantly reduced.

[0079] (5) Intelligent adaptation to multiple scenarios: Through transfer learning, the model can be quickly adapted to different scenarios such as semiconductors, medical care, and precision molds, breaking through the limitations of "single technology - single scenario" and realizing universal intelligent perception.

[0080] (6) Significantly reduced costs: By replacing expensive precision force sensors with optical imaging and AI computing, the system cost is reduced by more than 50% compared to imported sensors, and the time for a single measurement is less than 1 second, which significantly improves the production line yield and R&D speed.

[0081] (7) Independent and controllable: fills the gap in domestic micro-Newton level visual force measurement technology, breaks the monopoly of imported high-precision micro-force sensors, and supports the independent and controllable industrial chain of high-end manufacturing such as semiconductor equipment and medical robots.

[0082] Based on the same idea, this application also provides a non-contact micro-Newton force measurement system, such as... Figure 4 As shown, Figure 4 This is a schematic diagram of the overall structure of the non-contact micro-Newton force measurement system provided in this application embodiment. The non-contact micro-Newton force measurement system includes: (1) A high-resolution optical imaging module for acquiring high-resolution images of the surface of the object being measured. The high-resolution optical imaging module includes a high-resolution industrial camera with a resolution of ≥10 million pixels, a telecentric lens and a uniform illumination source; or the high-resolution optical imaging module includes a high-resolution industrial camera with a resolution of ≥10 million pixels, a macro lens and a uniform illumination source.

[0083] In some embodiments, the high-resolution optical imaging module further includes a multi-camera stereo vision unit, which includes at least two high-resolution industrial cameras and is used to acquire three-dimensional topographic information of the surface of the object under test through a stereo matching algorithm.

[0084] For example, the main camera is a CMOS industrial camera with a resolution of 4096×3072 pixels, a pixel size of 3.45μm, a maximum frame rate of 200fps, a telecentric lens with a magnification of 2, a working distance of 110mm, a depth of field of 0.5mm, and LED coaxial illumination with a color temperature of 5600K and adjustable brightness. In a 3D measurement configuration, an auxiliary camera of the same specifications is added, with the angle between the two cameras being approximately 15°~25°, forming a binocular stereo vision system.

[0085] In this application, a multi-camera stereo vision configuration is adopted, that is, at least two high-resolution industrial cameras constitute a binocular or multi-view stereo vision system, and the three-dimensional topography information of the surface of the object under test is obtained through a stereo matching algorithm, so as to realize the synchronous measurement of the three-dimensional displacement field and the three-dimensional force field.

[0086] (2) AI inference computing module, which includes an embedded computing unit equipped with an AI acceleration chip. The AI ​​acceleration chip is used to run sub-pixel deformation analysis algorithm and physical information neural network inversion model.

[0087] In some embodiments, the AI ​​acceleration chip adopts a heterogeneous computing architecture of a field-programmable gate array (FPGA) and a neural processing unit (NPU). The FPGA is used for hardware acceleration of image preprocessing and sub-pixel deformation analysis algorithms; the NPU is used for inference operations of physical information neural network inversion models. The heterogeneous computing architecture supports pipelined parallel processing, so that the overall data processing throughput of the system is not less than 100 frames / second.

[0088] For example, the AI ​​acceleration chip adopts a heterogeneous computing architecture of "FPGA + NPU". The FPGA is a Xilinx Zynq UltraScale+ series, responsible for the hardware pipeline acceleration of image preprocessing and sub-pixel deformation analysis algorithms; the NPU is a neural network processor with a computing power of ≥20 TOPS, responsible for the inference operation of the PINN inversion model. The FPGA and NPU are interconnected through a high-speed AXI bus to achieve pipelined parallel processing: when the NPU processes the PINN inference of the nth frame, the FPGA simultaneously processes the image preprocessing and sub-pixel analysis of the (n+1)th frame, so that the overall system throughput reaches more than 100 frames / second and the end-to-end latency is <10 milliseconds.

[0089] (3) Active temperature control module, used to control the temperature fluctuation of high-resolution industrial camera within ±0.1℃, and eliminate the influence of thermal drift on sub-pixel measurement accuracy. The active temperature control module includes a temperature sensor, a thermoelectric cooler and a PID controller.

[0090] Optionally, a PT1000 temperature sensor (accuracy ±0.01℃), a thin-film thermoelectric cooler (TEC), and a PID controller can be integrated into the industrial camera housing. The PID controller has a temperature control accuracy of ±0.1℃ and a temperature control range of ambient temperature ±10℃. By stabilizing the camera temperature, lens focal length drift and sensor pixel shift caused by thermal expansion and contraction are eliminated, ensuring long-term stability of sub-pixel measurements.

[0091] (4) Vibration isolation module, used to isolate the interference of environmental vibration on optical imaging and ensure measurement stability in industrial environment. The vibration isolation module includes a first vibration isolation module and / or a second vibration isolation module. The first vibration isolation module includes an air-floating vibration isolation platform, and the second vibration isolation module includes a vibration sensor and a real-time compensation unit.

[0092] Optionally, the vibration-damping module includes a first vibration-damping module and a second vibration-damping module.

[0093] This application employs a two-stage vibration isolation and reduction scheme to eliminate the error interference of vibration on force measurement. The first stage implements passive vibration isolation by assembling an optical imaging module on an air-floating vibration isolation platform to isolate ground vibrations with a frequency greater than 2Hz. The second stage implements active vibration compensation by deploying a three-axis MEMS accelerometer (i.e., a vibration sensor) with a sensitivity of no less than 1mg on an industrial camera to collect residual vibration parameters in real time. The real-time compensation unit uses the residual vibration data to run an image motion compensation algorithm to correct image distortion and pixel shift caused by vibration, optimize the accuracy of sub-pixel displacement field solution, reduce the error of vibration noise transmission to the mechanical inversion link, and improve the accuracy of three-dimensional force field, contact surface normal and tangential force measurement.

[0094] (5) Adaptive calibration software module, used to perform initial system calibration, periodic self-calibration and over-the-air (OTA) remote upgrade of physical information neural network inversion model parameters.

[0095] In some embodiments, the adaptive calibration software module is developed based on Python and runs on a host computer or embedded computing unit, and includes the following functional units: Initial calibration unit: Used to perform geometric calibration of the system using a standard micrometer-level grating ruler or a standard sample with known displacement, and to establish the mapping relationship between pixels and physical dimensions. For example, by photographing a standard micrometer-level grating ruler with a line spacing of 10μm and an accuracy of ±0.5μm, the pixel-to-physical-size mapping coefficient (μm / pixel) is automatically calculated, and lens distortion parameters are corrected.

[0096] Periodic self-calibration unit: Used to periodically acquire static reference images of the surface of the object under test when it is not under stress during system operation, and to detect and compensate for system drift caused by environmental changes. For example, every preset time interval (e.g., 30 minutes), a set of static reference images is automatically acquired, the spurious displacement field between the static reference image and the initial reference image is calculated, and if the amplitude of the spurious displacement field exceeds a threshold (e.g., 0.005 pixels), the reference image is automatically updated and the system drift is compensated.

[0097] The scenario adaptation unit receives calibration experimental data from different application scenarios through a transfer learning interface. It then uses this data to update the parameters of the physical information neural network inversion model for the corresponding application scenario. These application scenarios include semiconductor wafer inspection, biological cell manipulation, and precision mold processing. The system in this application provides a standardized transfer learning interface. Users only need to provide a small amount (20-50 sets) of calibration data for the new scenario, and the system can automatically complete the fine-tuning and updating of the PINN model to adapt to the new scenario.

[0098] Optionally, the adaptive calibration software module also includes an OTA upgrade unit, which supports remote updates of algorithm firmware and PINN model parameters via network, continuously optimizing system performance and reducing the barriers to use and maintenance.

[0099] To better understand the method and system provided in this application, the following three specific application scenarios are used to illustrate the non-contact micro-Newton force measurement method of this application.

[0100] Example 1: Application of semiconductor wafer bonding force detection like Figure 5 As shown, Figure 5 This is a schematic diagram illustrating the application of the non-contact micro-Newton force measurement system in the semiconductor wafer bonding process provided in this embodiment.

[0101] Application Scenario Description: The wafer bonding equipment in this embodiment includes two wafers (Wafer A and Wafer B, with monocrystalline silicon (Si) as the substrate material) with a diameter of 200mm or 300mm. The two wafers are sandwiched between an upper pressure head and a lower base (or heating stage). Axial pressure is applied by the upper pressure head to perform a pressure and temperature bonding process on the wafer interface. The wafer bonding equipment also includes a bonding equipment pressure control system for real-time adjustment of pressure or temperature.

[0102] In the wafer bonding process, two wafers are atomically bonded under preset temperature and pressure. The local bonding force distribution at the bonding interface directly affects the bonding quality and device yield. Traditional methods use imported thin-film pressure sensor arrays for contact measurement, which has the following problems: (a) the sensor may damage the bonding interface when in contact with the wafer; (b) the sensor has low spatial resolution and can usually only measure macroscopic average force; (c) imported sensors are expensive, with a single system costing more than 500,000 yuan.

[0103] The implementation scheme of this embodiment: (1) A fine speckle pattern is fabricated on the surface of the upper pressure head of the wafer bonding equipment, with a speckle size of about 5 μm; (2) A high-resolution optical imaging module consisting of a 4096×3072 pixel camera and a 5x telecentric lens was installed at the observation window on the side of the bonding device, and the speckle deformation on the surface of the upper pressure head was observed through the window. (3) Calculate the displacement field based on the DIC speckle deformation, load the pre-trained PINN inversion model, with elastic modulus E=130GPa, Poisson's ratio ν=0.28, corresponding to the constitutive parameters of silicon material, and use 20 sets of bond force calibration data for transfer learning fine-tuning. (4) During the bonding process, the system acquires images in real time and calculates the three-dimensional force field distribution of the bonding interface, with a measurement accuracy of ±5μN and a spatial resolution better than 50μm; (5) Feed back the real-time force field data to the pressure control system of the bonding equipment to realize closed-loop feedback control of the bonding force.

[0104] Application results: Pilot verification shows that the measurement error of this embodiment is reduced by 20% compared with the traditional imported sensor solution, the system cost is reduced by more than 50%, and the time for a single measurement is less than 1 second, which effectively improves the yield of wafer bonding process.

[0105] Example 2: Application of bio-cellular force feedback like Figure 6 As shown, Figure 6 This is a schematic diagram illustrating the application of the non-contact micro-Newton force measurement system in a biological cell manipulation force feedback scenario provided by an embodiment of this application. This embodiment applies the above-mentioned non-contact micro-Newton force measurement system to a cell micro-operating system in the biomedical field.

[0106] Application Scenario Description: In assisted reproductive technology (ICSI) and cell mechanics research, microneedles (1-10 μm tip diameter) are required for cell puncture and injection. The force applied during these procedures typically ranges from 10 to 500 μN; excessive force will damage the cells, while insufficient force will prevent successful puncture. Traditional methods utilize microforce sensors integrated onto the microneedles, but the size of these sensors limits the fineness of the microneedles and they are expensive.

[0107] The experimental setup in this embodiment includes a micromanipulation robot, borosilicate glass microneedles (hereinafter referred to as microneedles), and cell culture dishes and cells placed on a sample stage. During cell manipulation, the micromanipulation robot can perform force feedback control.

[0108] The implementation scheme of this embodiment: (1) Utilize the natural texture or artificial markings on the surface of microneedles as image matching features; (2) Install a high-resolution optical imaging module consisting of a 4096×3072 high-speed camera and a 10x long working distance objective lens on the imaging optical path of the inverted microscope and observe the deformation of the microneedle tip from below. (3) Load the pre-trained PINN inversion model with elastic modulus E=69GPa and Poisson's ratio ν=0.22, corresponding to the borosilicate glass material parameters, and use 30 sets of microneedle force calibration data for transfer learning fine-tuning; (4) During cell manipulation, the system acquires original images in real time at a frame rate of 200fps, performs subpixel image matching on the original images, calculates the subpixel deformation of the microneedle tip with an accuracy of 0.01 pixels, corresponding to a displacement resolution of approximately 3.5nm. Then, the contact force between the microneedle and the cell is inverted through the PINN inversion model, with a force measurement accuracy of ±2μN. Specifically, this includes input parameters, pre-trained PINN model, and transfer learning fine-tuning, and finally outputs the contact force F. (5) The visual force feedback (such as the calculated contact force F) of the cell contact interface is transmitted to the micromanipulation robot controller. The micromanipulation robot controller uses the PID control algorithm to output drive commands based on the real-time force feedback signal to control the feeding action of the microneedle actuator, adjust the interaction force between the microneedle actuator and the cell contact interface, complete the cell puncture or injection operation, realize the closed-loop precise control of the operation force, and prevent cell damage.

[0109] Application results: Verification shows that the cell manipulation force feedback system in this embodiment has an operation success rate of over 95%, which is about 30% higher than that without feedback, and the cell survival rate is increased to over 98%.

[0110] Example 3: Intelligent Adaptation for Multiple Scenarios like Figure 7 As shown, Figure 7 This is a flowchart illustrating the multi-scenario adaptation process of transfer learning provided in this application embodiment. This embodiment describes a method for achieving rapid multi-scenario adaptation through transfer learning: (1) Build a scenario library: Pre-build a pre-trained model library covering multiple fields such as semiconductors (silicon, germanium, gallium arsenide, etc.), biomedicine (cells, tissues, etc.), and precision molds (hard alloys, ceramics, etc.). Each pre-trained model corresponds to a type of material or scenario. (2) Scene recognition and selection: When the user uses it in a new scene, he / she inputs the material type and geometric parameters of the object being tested, and the system automatically selects the most matching pre-trained model from the model library; (3) Rapid fine-tuning: After the pre-trained model is matched, the system will complete the model fine-tuning within 5 to 30 minutes based on 20 to 50 sets of calibration data provided by the user, and obtain a dedicated model adapted to the new scenario; (4) Continuous optimization: After fine-tuning, the system uploads the model parameters after each fine-tuning to the cloud and uses the federated learning framework to continuously optimize the global pre-trained model while protecting user data privacy.

[0111] This multi-scenario adaptation solution breaks through the limitations of the traditional "single technology - single scenario" approach, achieving universal intelligent perception and significantly reducing the barriers to technology promotion and application.

[0112] This application also provides an electronic device, including: a processor and a memory, wherein the memory is used to store a computer program; and the processor is used to run the computer program to implement the non-contact micro-Newton force measurement method provided in the embodiments of this application.

[0113] This application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to execute the non-contact micro-Newton force measurement method provided in the embodiments shown in this application.

[0114] The embodiments of this application have now been described in detail. To avoid obscuring the concept of this application, some details known in the art have not been described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein based on the above description.

[0115] While specific embodiments of this application have been described in detail by way of examples, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of this application. Those skilled in the art should understand that modifications can be made to the above embodiments or equivalent substitutions can be made to some technical features without departing from the scope and spirit of this application. In particular, as long as there is no structural conflict, the various technical features mentioned in the embodiments can be combined in any manner.

Claims

1. A non-contact micro-Newton force measurement method, characterized in that, Includes the following steps: S1: Using a high-resolution industrial camera with a resolution of ≥10 million pixels, a reference image and a deformed image of the surface of the object under test are acquired under preset optical illumination conditions. The reference image is the image of the surface of the object under test when it is not under stress, and the deformed image is the image of the surface of the object under test after it is deformed under stress. S2: Perform subpixel deformation analysis on the reference image and the deformed image to obtain the subpixel displacement field and the full-field strain distribution, specifically including: S2a: Preprocess the reference image and the deformed image, the preprocessing including grayscale normalization, noise suppression and uneven illumination correction; S2b: A preset integer pixel matching algorithm is used to establish an initial correspondence between sub-regions between the preprocessed reference image and the deformed image to obtain the integer pixel displacement field; S2c: The sub-pixel displacement field is obtained by refining the calculation of the integer pixel displacement field using a preset sub-pixel positioning algorithm. S2d: Based on the subpixel displacement field, the full-field strain distribution on the surface of the object under test is calculated using a preset numerical differentiation method; S3: Input the subpixel displacement field and the full-field strain distribution into the pre-trained physical information neural network inversion model, and output the three-dimensional force field distribution on the surface of the object under test.

2. The method according to claim 1, characterized in that, The expression for the loss function L of the physical information neural network inversion model is: in, For data-driven loss terms; This refers to the residual loss term in the elasticity equilibrium equation; This is the constitutive relation residual loss term; The loss term is defined by boundary condition constraints. The adaptive weighting coefficients are dynamically adjusted using an uncertainty weighting strategy or a gradient normalization strategy. Predicting displacement using a neural network; ε is the measured displacement; σ is the measured displacement; ε is the strain tensor; C is the elastic stiffness tensor; f is the body force vector; N, M, and K are the number of data points, collocation points, and boundary points, respectively.

3. The method according to claim 1, characterized in that, The training method for the physical information neural network inversion model includes: A simulation dataset containing various typical mechanical loading scenarios is constructed. The simulation dataset includes displacement field data and stress field data under known force boundary conditions. The physical information neural network inversion model is pre-trained based on the simulation dataset to obtain a pre-trained model. Collect calibration experimental data in the target application scenario, and fine-tune the pre-trained model using a transfer learning strategy to obtain an inversion model adapted to the target application scenario; An adaptive learning rate scheduling strategy and an early stopping mechanism are used to optimize the training process of the inversion model adapted to the target application scenario.

4. The method according to claim 1, characterized in that, The method further includes: The image sequence of the surface of the object under test is continuously acquired at a frame rate of not less than 100 frames per second; Steps S2 and S3 are performed on each frame of the image sequence to obtain the real-time reconstructed three-dimensional position distribution; The real-time reconstructed three-dimensional position distribution is output to an external control system to provide closed-loop control feedback for the precision manufacturing process.

5. The method according to claim 1, characterized in that, A temperature sensor, a thermoelectric cooler, and a PID controller are integrated within the housing of the high-resolution industrial camera; the method further includes: The temperature sensor is used to collect the measured temperature from the high-resolution industrial camera. The PID controller outputs a control signal based on the deviation between the measured temperature of the high-resolution industrial camera and the preset temperature. The thermoelectric cooler is used to perform temperature control according to the control signal, so that the operating temperature of the high-resolution industrial camera is controlled within a preset temperature range.

6. The method according to claim 1, characterized in that, The method further includes: Obtain an initial reference image of the surface of the object under test when it is not under stress; Static reference images of the surface of the object under test when it is not under stress are acquired at preset time intervals; Calculate the virtual displacement field between the static reference image and the initial reference image; If the amplitude of the virtual displacement field is greater than a preset threshold, the static reference image is updated to the reference image in step S1, and system drift compensation is performed on the subsequent displacement solution based on the virtual displacement field.

7. A non-contact micro-Newton force measurement system for implementing the method according to any one of claims 1 to 6, characterized in that, include: A high-resolution optical imaging module is used to acquire high-resolution images of the surface of an object under test. The high-resolution optical imaging module includes a high-resolution industrial camera with a resolution of ≥10 million pixels, a telecentric lens, and a uniform illumination source; or the high-resolution optical imaging module includes a high-resolution industrial camera with a resolution of ≥10 million pixels, a macro lens, and a uniform illumination source. The AI ​​inference computing module includes an embedded computing unit equipped with an AI acceleration chip, which is used to run sub-pixel deformation analysis algorithms and physical information neural network inversion models. An active temperature control module is used to control the temperature fluctuation of the high-resolution industrial camera within ±0.1℃. The active temperature control module includes a temperature sensor, a thermoelectric cooler, and a PID controller. An anti-vibration module is used to isolate the interference of environmental vibration on optical imaging. The anti-vibration module includes a first anti-vibration module and / or a second anti-vibration module. The first anti-vibration module includes an air-floating vibration isolation platform, and the second anti-vibration module includes a vibration sensor and a real-time compensation unit. The adaptive calibration software module is used to perform initial system calibration, periodic self-calibration, and over-the-air download and remote upgrade of physical information neural network inversion model parameters.

8. The system according to claim 7, characterized in that, The high-resolution optical imaging module also includes a multi-camera stereo vision unit, which includes at least two of the high-resolution industrial cameras. The multi-camera stereo vision unit is used to acquire the three-dimensional shape information of the surface of the object under test through a stereo matching algorithm.

9. The system according to claim 7, characterized in that, The AI ​​acceleration chip adopts a heterogeneous computing architecture of field-programmable gate array (FPGA) and neural network processor. The FPGA is used for hardware acceleration of image preprocessing and sub-pixel deformation analysis algorithms. The neural network processor is used for inference operations of physical information neural network inversion model. The heterogeneous computing architecture supports pipelined parallel processing, so that the overall data processing throughput of the system is not less than 100 frames / second.

10. The system according to claim 7, characterized in that, The adaptive calibration software module includes: Initial calibration unit: used to perform geometric calibration of the system using a standard micrometer-level grating ruler or a standard sample with known displacement, and to establish the mapping relationship between pixels and physical dimensions; Periodic self-calibration unit: used to periodically acquire static reference images of the surface of the object under test when it is not under force during system operation, and to detect and compensate for system drift caused by environmental changes; Scene adaptation unit: used to receive calibration experimental data of different application scenarios through transfer learning interface, and use the calibration experimental data to update the physical information neural network inversion model parameters of the corresponding application scenario. The application scenarios include semiconductor wafer inspection, biological cell manipulation and precision mold processing.

Images