A micro-precision visual motion tracking method against off-plane disturbance

Abstract

The application discloses a kind of self anti-off-plane disturbance micron-level precision visual motion tracking methods, method includes: S1: constructing micron-level precision visual motion tracking measurement system;S2: the motion image of observation object is acquired, and real-time detection off-plane disturbance;S3: for the visual precision degradation caused by off-plane disturbance, self anti-off-plane disturbance device and self anti-off-plane disturbance program are optimized to self anti-interference scheme;S4: according to the fuzzy rule of preestablished, for low frequency high amplitude type off-plane disturbance, precision repair is realized to self anti-off-plane disturbance device in hardware level optimization;For high frequency low amplitude type off-plane disturbance, precision repair is realized to self anti-off-plane disturbance program in software level optimization;For wide frequency or cross amplitude hybrid type off-plane disturbance, precision repair is realized to self anti-off-plane disturbance device and self anti-off-plane disturbance program synchronous linkage;The application realizes the precision optimization of micro visual motion tracking under different off-plane displacement disturbances, improves the anti-off-plane disturbance ability of system.

Description

Technical Field

[0001] This invention relates to the field of visual motion tracking, and in particular to a micrometer-level precision visual motion tracking method that is resistant to out-of-plane perturbations. Background Technology

[0002] Visual measurement technology involves focusing a camera onto a specific plane to create an image. Visual motion tracking technology uses pixel data from several vision-acquired images to calculate displacement. This principle restricts the measurement object to remaining within the "focal plane." Off-plane displacement perpendicular to the "focal plane" reduces the accuracy of visual measurement, and consequently, the accuracy of visual motion tracking. Combined with the precision of microscopy, visual measurement can reach the (sub)micron level. However, as measurement accuracy increases, the impact of off-plane displacement becomes more pronounced. This means that the measurement object needs to be kept even more strictly within the "focal plane." Yet, in industrial settings, objects undergoing measurement often exhibit off-plane displacement away from the "focal plane," especially those in motion.

[0003] The out-of-plane displacement caused by the object's movement, deviating from the focal plane, affects the imaging data and, consequently, the measurement accuracy through image processing algorithms. Current technologies fail to offer effective methods for optimizing the accuracy of micro-visual motion tracking under out-of-plane displacement perturbations. They can only treat this measurement error as part of the noise in the visual motion tracking system, unable to isolate, compensate for, or eliminate it, thus limiting further improvements in measurement accuracy. Existing micro / nano-level visual motion tracking technologies cannot avoid the accuracy degradation caused by out-of-plane displacement. Due to the uncertainty of the excitation signal generating out-of-plane displacement, the same optimization method yields different results in micro-visual motion tracking accuracy under different conditions. Current technologies cannot optimize the accuracy of micro-visual motion tracking under different out-of-plane displacement perturbations, and there is no good solution for the errors caused by out-of-plane displacement. With the widespread application of micro / nano-visual motion tracking technology and the gradual improvement of measurement accuracy, the number of objects requiring adaptation is increasing, and the tolerance for out-of-plane displacement decreases with higher-precision measurement methods. Current technologies cannot achieve accuracy optimization for micro-visual motion tracking under different out-of-plane displacement perturbations, thus limiting the range of measurement objects. Summary of the Invention

[0004] To address the problem that existing technologies cannot achieve accuracy optimization in micro-visual motion tracking under different off-plane displacement disturbances, this invention proposes a micrometer-level precision visual motion tracking method with self-resistant off-plane disturbances. This method optimizes the accuracy of micro-visual motion tracking under different off-plane displacement disturbances and improves the resistance to off-plane disturbances.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] The present invention discloses a micron-level precision visual motion tracking method for self-anti off-plane disturbance, including the following steps:

[0007] S1: Construct a micron-level precision visual motion tracking measurement system; the micron-level precision visual motion tracking measurement system includes a self-anti off-plane disturbance device and a self-anti off-plane disturbance program;

[0008] S2: Obtain the motion image of the observation object through the micron-level precision visual motion tracking measurement system, and detect the off-plane disturbance in real time;

[0009] S4: According to the preset fuzzy rules, for the low-frequency high-amplitude off-plane disturbance, optimize and repair the accuracy at the hardware level for the self-anti off-plane disturbance device; for the high-frequency low-amplitude off-plane disturbance, optimize and repair the accuracy at the software level for the self-anti off-plane disturbance program; for the broadband or cross-amplitude mixed off-plane disturbance, synchronously link the self-anti off-plane disturbance device and the self-anti off-plane disturbance program to achieve accuracy repair.

[0010] The working principle of the present invention is as follows:

[0011] Generate controllable axial motion compensation for off-plane displacement and synchronously link with the self-anti off-plane disturbance algorithm to repair in real time the visual tracking accuracy degradation caused by different off-plane displacement disturbances, and improve the off-plane disturbance resistance ability of the system.

[0012] Preferably, according to the degree of visual accuracy degradation caused by the off-plane disturbance, different off-plane disturbances are divided into the following five levels:

[0013] For the visual tracking accuracy degradation ratio L;

[0014] If L ≤ 5%, it is excellent and no optimization is required;

[0015] If 5% < L ≤ 40%, it is good and optimization is required;

[0016] If 40% < L ≤ 80%, it is medium and optimization is required;

[0017] If 80% < L ≤ 150%, it is poor and optimization is required;

[0018] If 150% < L, the tracking accuracy is lost and optimization cannot be carried out.

[0019] Further, for different types and levels of off-plane disturbances, corresponding self-anti disturbance schemes are adopted;

[0020] The self-anti disturbance scheme is to adopt different optimization methods to optimize the self-anti off-plane disturbance device and the self-anti off-plane disturbance program for different off-plane disturbances; select any optimization method of the self-anti off-plane disturbance device and several optimization methods of the self-anti off-plane disturbance program to synchronously link the self-anti off-plane disturbance device and the self-anti off-plane disturbance program.

[0021] Furthermore, after selecting an active disturbance rejection scheme, standard test experiments were conducted to calculate the accuracy improvement effect of different selected schemes on micron-level visual motion tracking under different off-surface disturbances. Based on the evaluation results, an off-surface excitation-visual active disturbance rejection mapping was formed and quantified.

[0022] The standard test experiment is as follows:

[0023] S301: Design 20 motion tracking experiments;

[0024] S302: In the first to fifth experiments, visual motion tracking was performed without adding any out-of-plane perturbations, and the standard accuracy f of visual motion tracking was obtained. 00 ;

[0025] S303: During visual motion tracking in the i-th experiment, add the acquired out-of-plane perturbation to obtain the degraded accuracy f of visual motion tracking. 10 Where 6≤i≤15;

[0026] S304: During visual motion tracking in experiments i+1 to i+4, the acquired out-of-plane perturbation is added, and the corresponding selection scheme is used to improve accuracy, resulting in the improved visual motion tracking accuracy f. 11 ;

[0027] S305: When performing visual motion tracking in experiments i+5 to 20, without adding any out-of-plane perturbations, the standard accuracy f of visual motion tracking is obtained. 01 ;

[0028] S306; Calculate the accuracy results obtained from S302 to S306 above, and obtain the improvement effect of this selection scheme according to the evaluation formula.

[0029] Furthermore, the optimization methods for the self-resisting off-plane disturbance device include: single piezoelectric drive low-frequency compensation method, dual piezoelectric drive high-frequency compensation method, single piezoelectric drive objective lens focal plane method, and tracking object off-plane motion suppression method;

[0030] Optimization methods for self-resistant out-of-plane perturbation programs include: template type / quantity / area variation method, real-time region of interest transformation method, out-of-plane-degraded fuzzy set method, real-time evaluation function solution algorithm, multi-focal surface real-time matching method, algorithm category / computation optimization method, frequency-precision dynamic balancing method, and deep learning tool parameter tuning method.

[0031] Furthermore, the self-resisting off-surface disturbance device adopts the tracking object off-surface motion suppression method; the self-resisting off-surface disturbance program adopts the evaluation function real-time solution algorithm and the algorithm category / computation optimization method.

[0032] The evaluation function real-time solution algorithm is used to solve off-plane disturbances into off-plane displacements. The tracking object off-plane motion suppression method is used to control the self-resisting off-plane disturbance device to generate controllable axial motion to compensate for off-plane displacements. The algorithm category / computation optimization method is used to control the mapping relationship between the self-resisting off-plane disturbance program and the corresponding off-plane displacement to the visual algorithm and the off-plane displacement to the controllable parameters.

[0033] Furthermore, the steps to convert out-of-plane disturbances into out-of-plane displacements are as follows:

[0034] S101: Acquire the motion image at time t0;

[0035] S102: Obtain the current sharpness evaluation value through the sharpness evaluation function preset in the self-resistant out-of-plane perturbation procedure, and set it as the maximum sharpness value F. max ;

[0036] S103: The industrial camera (1) moves downward a distance s to control the self-resisting surface disturbance device;

[0037] S104: Obtain the sharpness evaluation value after the movement through the preset sharpness evaluation function, and record it as F1;

[0038] S105: Determine the current sharpness rating value and F max Size, set threshold λ, if |Δ|=|F1-F max |>λ and Δ=F1-F max >0, proceed to step S106;

[0039] If |Δ|=|F1-F max If |≤λ, proceed to step S107;

[0040] If |Δ|=|F1-F max |>λ and Δ=F1-F max If the value is less than 0, proceed to step S108.

[0041] S106: Real-time update of maximum resolution, allowing F max =F1, and control the industrial camera (1) to continue moving downwards a distance s;

[0042] S107: End focusing and record the current position of the industrial camera (1);

[0043] S108; Keeping the maximum sharpness constant, control the industrial camera (1) to move upward by a distance s;

[0044] S109: Obtain the motion image at time t0+Δt, and repeat S102 to S108;

[0045] S110: Set the direction of movement of the industrial camera (1) away from the plane as the positive direction and the direction of movement towards the plane as the negative direction. Based on the distances x0 and x1 moved by the industrial camera (1) during the two focusing processes, calculate the magnitude, direction and speed of the displacement from the plane according to the calculation formulas d = x0 - x1 and v = d / Δt.

[0046] Furthermore, the method for generating controllable axial motion to compensate for out-of-plane displacement is as follows:

[0047] The calculated magnitude, direction, and velocity of the out-of-plane displacement are converted into displacement compensation signals. Axial displacement is then generated based on these signals to achieve hardware-level anti-out-of-plane disturbance. Specifically:

[0048] The conversion of axial runout frequency is as follows:

[0049] If v∈(A) n A n+1 ),but

[0050] The conversion of axial runout amplitude is as follows:

[0051] If d∈(B) n B n+1 ),but

[0052] Where d and v are the calculated out-of-plane displacement amplitude and velocity, respectively; δ1 and δ2 are the axial runout frequency and axial runout amplitude, respectively; and n is the number of fuzzy mapping rules.

[0053] Based on the converted displacement compensation signal, an axial motion compensation for the out-of-plane displacement is generated with an axial runout frequency of δ1, an axial runout amplitude of δ2, and the opposite direction to the out-of-plane displacement.

[0054] Furthermore, based on the calculated out-of-plane displacement, the self-resisting out-of-plane disturbance program calls the visual algorithm and controllable parameters according to the mapping relationship:

[0055]

[0056] Where i = 1, 2, ..., n, d and v are the calculated displacement amplitude and velocity, respectively, x is the mapping relationship obtained from the previous experiment, and D i V i and X i R is a set consisting of d, v, and x respectively. i (x,y,z) is derived from D i ×V i To X i The fuzzy relationship is as follows:

[0057]

[0058] The mapping relationship between out-of-plane displacement, visual algorithm, and controllable parameters was obtained through preliminary experiments. The specific steps of the preliminary experiments are as follows:

[0059] S201: Obtain the visual motion tracking accuracy without out-of-plane perturbation, denoted as f0;

[0060] S202: Axial motion is generated, denoted as out-of-plane disturbance a;

[0061] S203: Preset template matching method based on grayscale and template matching method based on feature points;

[0062] S204: Extract controllable parameters from the gray-scale-based template matching method, and denote the number of templates n and the template area s as controllable parameters α1 and α2, respectively; extract controllable parameters from the feature point-based template matching method, and denote the Gaussian filter coefficient σ and the number of pyramid layers m as controllable parameters β1 and β2, respectively.

[0063] S205: Using a grayscale-based template matching method, the controllable parameters α1 and α2 are changed respectively to obtain the visual tracking accuracy under the first out-of-plane perturbation, denoted as f. 11 f 12 Using a feature-point-based template matching method, the controllable parameters β1 and β2 are changed respectively to obtain the visual tracking accuracy under the first out-of-plane perturbation, denoted as f. 21 f 22 ;

[0064] S206: Based on the different visual tracking accuracies f obtained above 11 f 12 f 21 f 22 The results of the accuracy repair ratio calculation are recorded as p1, p2, p3, and p4, respectively.

[0065] S207: Filter out the minimum accuracy repair ratio p min Establish the mapping relationship between the first out-of-plane displacement and the corresponding visual algorithm and parameters, denoted as the first mapping;

[0066] S208: Generate two other different axial movements, denoted as the second off-plane disturbance and the third off-plane disturbance; repeat S203-S207 above to establish the mapping between the off-plane displacement and the corresponding visual algorithm and parameters, denoted as the second mapping and the third mapping.

[0067] Furthermore, by comparing the obtained visual motion tracking results with the detection results of a third-party nanometer-precision sensor, the visual motion tracking accuracy under out-of-plane displacement perturbation is obtained, and the accuracy optimization ratio is calculated, as shown in the following expression:

[0068] P = 100% × |f ij -f0|,i,j=1,2,3···,n

[0069] Among them, f ij f0 represents the visual motion tracking accuracy under off-surface perturbation after changing the visual algorithm and controllable parameters; f0 represents the visual motion tracking accuracy without off-surface perturbation, which is regarded as the baseline accuracy.

[0070] Compared with the prior art, the beneficial effects of the present invention are:

[0071] 1. This invention achieves synchronous linkage between the device and the program by selecting and matching self-disturbance rejection schemes, thereby improving the degradation of visual tracking accuracy caused by different out-of-plane displacement disturbances.

[0072] 2. This invention provides accurate repair for different out-of-plane displacements, is suitable for dealing with out-of-plane disturbances that are widespread in industrial settings, and has strong robustness and good real-time performance. Attached Figure Description

[0073] Figure 1 A flowchart of a micrometer-precision visual motion tracking method with self-resistant off-plane perturbation provided for an embodiment.

[0074] Figure 2 This is a schematic diagram of an out-of-plane excitation-visual degradation fuzzy set provided for an embodiment.

[0075] Figure 3 This is a schematic diagram of an out-of-plane excitation-visual optimization fuzzy set provided for an embodiment.

[0076] Figure 4 This is a schematic diagram of the optimization method for the self-resistant off-plane disturbance device provided in the embodiment.

[0077] Figure 5 A flowchart of the selected active interference rejection scheme provided for the embodiment.

[0078] Figure 6 This is a schematic diagram of the self-resistant off-plane disturbance device provided in the embodiment.

[0079] Figure 7 This is a schematic diagram of off-plane displacement compensation provided for an embodiment.

[0080] The components include: 1. Industrial camera; 2. Microscope; 3. Piezoelectric driven compliant mechanism; 4. Control card. Detailed Implementation

[0081] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0082] Example 1

[0083] In this embodiment, as Figure 1 As shown, a micron-level precision visual motion tracking method for self-anti off-plane disturbance includes the following steps:

[0084] S1: Construct a micron-level precision visual motion tracking measurement system; the micron-level precision visual motion tracking measurement system includes a self-anti off-plane disturbance device and a self-anti off-plane disturbance program;

[0085] S2: Obtain the motion image of the observed object through the micron-level precision visual motion tracking measurement system and detect the off-plane disturbance in real time;

[0086] S3: In response to the visual accuracy degradation caused by the off-plane disturbance, select and match a self-anti disturbance scheme to optimize the self-anti off-plane disturbance device and the self-anti off-plane disturbance program;

[0087] S4: According to the preset fuzzy rules, for low-frequency high-amplitude off-plane disturbances, optimize and repair the accuracy at the hardware level for the self-anti off-plane disturbance device; for high-frequency low-amplitude off-plane disturbances, optimize and repair the accuracy at the software level for the self-anti off-plane disturbance program; for broadband or cross-amplitude mixed off-plane disturbances, use the synchronous linkage of the self-anti off-plane disturbance device and the self-anti off-plane disturbance program to achieve accuracy repair.

[0088] In step S2, for the off-plane disturbance, quantify the off-plane excitation-visual degradation mapping and construct an off-plane excitation-visual degradation fuzzy set; based on the off-plane excitation-visual degradation fuzzy set, embed the self-anti off-plane disturbance program into the self-anti off-plane disturbance device.

[0089] In step S3, quantify the off-plane excitation-visual optimization mapping, construct an off-plane excitation-visual optimization fuzzy set and quantify it, and embed it into the self-anti off-plane disturbance program.

[0090] In this embodiment, according to the degree of visual accuracy degradation caused by the off-plane disturbance, different off-plane disturbances are divided into the following five levels:

[0091] For the visual tracking accuracy degradation ratio L;

[0092] If L ≤ 5%, it is excellent and no optimization is required;

[0093] If 5% < L ≤ 40%, it is good and optimization is required;

[0094] If 40% < L ≤ 80%, it is medium and optimization is required;

[0095] If 80% < L ≤ 150%, it is poor and optimization is required;

[0096] If 150% < L, the tracking accuracy is lost and no optimization can be performed.

[0097] In this embodiment, as Figure 2As shown, the different out-of-plane perturbations have different types, amplitudes, and frequencies, including but not limited to high, medium, and low frequency, high, medium, and low amplitude simple harmonic, impulse, step, noise, and random out-of-plane perturbations.

[0098] More specifically, such as Figure 3 As shown, corresponding active disturbance rejection schemes are adopted for different types and levels of out-of-plane disturbances;

[0099] The aforementioned active disturbance rejection scheme employs different optimization methods to optimize the active disturbance rejection device and the active disturbance rejection program for different off-plane disturbances; it selects the optimization method of any active disturbance rejection device and several optimization methods of the active disturbance rejection program to enable the active disturbance rejection device and the active disturbance rejection program to work in sync.

[0100] In S3, information such as displacement compensation range and frequency compensation range are obtained based on the out-of-plane excitation-visual degradation fuzzy set.

[0101] More specifically, after selecting the active disturbance rejection scheme, standard test experiments are conducted. The off-surface excitation-visual optimization fuzzy set is used to calculate the accuracy improvement effect of different selected schemes on micron-level visual motion tracking under different off-surface disturbances. Based on the evaluation results, an off-surface excitation-visual active disturbance rejection mapping is formed and quantified.

[0102] The standard test experiment is as follows:

[0103] S301: Design 20 motion tracking experiments;

[0104] S302: In the first to fifth experiments, visual motion tracking was performed without adding any out-of-plane perturbations, and the standard accuracy f of visual motion tracking was obtained. 00 ;

[0105] S303: During visual motion tracking in the i-th experiment, add the acquired out-of-plane perturbation to obtain the degraded accuracy f of visual motion tracking. 10 Where 6≤i≤15;

[0106] S304: During visual motion tracking in experiments i+1 to i+4, the acquired out-of-plane perturbation is added, and the corresponding selection scheme is used to improve accuracy, resulting in the improved visual motion tracking accuracy f. 11 ;

[0107] S305: When performing visual motion tracking in experiments i+5 to 20, without adding any out-of-plane perturbations, the standard accuracy f of visual motion tracking is obtained. 01 ;

[0108] S306; Calculate the accuracy results obtained from S302 to S306 above, and obtain the improvement effect of this selection scheme according to the evaluation formula; the evaluation formula is as follows:

[0109]

[0110] More specifically, such as Figure 4 As shown, the optimization methods for the self-resisting off-plane disturbance device include: H1 single piezoelectric drive low-frequency compensation method, H2 dual piezoelectric drive high-frequency compensation method, H3 single piezoelectric drive objective lens focal plane method, and H4 tracking object off-plane motion suppression method.

[0111] The optimization methods for self-resistant off-surface perturbation programs include: T1 Template type / quantity / area method, T2 Real-time transformation of region of interest method, T3 "off-surface-deterioration" fuzzy set method, T4 Real-time solution algorithm for evaluation function, T5 Real-time matching of multiple focal planes method, T6 Algorithm category / computation optimization method, T7 "frequency-precision" dynamic equalization method, and T8 Parameter tuning method for deep learning tools.

[0112] In this embodiment, a single piezoelectric drive is used to achieve low-frequency compensation for the self-resistant off-plane disturbance device, a dual piezoelectric drive is used to achieve high-frequency compensation for the self-resistant off-plane disturbance device, a piezoelectric objective lens is used to achieve focal plane method for the self-resistant off-plane disturbance device, and off-plane suppression is used to achieve off-plane motion suppression method for the tracked object.

[0113] In the specific implementation process, for low-frequency, high-vibration-mode out-of-plane disturbances, such as out-of-plane disturbances with a frequency less than 50 Hz and an amplitude greater than 300 μm, a hardware solution is used for accuracy repair. For high-frequency, low-vibration-mode out-of-plane disturbances, such as out-of-plane disturbances with a frequency greater than 50 Hz and an amplitude less than 300 μm, a software solution is used for accuracy repair. For mixed-type out-of-plane displacements, a combination of hardware and software is used for accuracy repair.

[0114] Example 2

[0115] Based on Example 1, such as Figure 5 As shown, in this embodiment, based on the out-of-plane excitation-visual optimization fuzzy set, simple harmonic out-of-plane displacements with medium frequency and large amplitude, medium frequency and medium amplitude, and medium frequency and low amplitude are selected, namely out-of-plane displacements D111, D112, and D113, as out-of-plane disturbances. The out-of-plane motion suppression method for the self-resistance out-of-plane disturbance device is adopted; the real-time solution algorithm of the evaluation function and the algorithm category / computation optimization method are adopted for the self-resistance out-of-plane disturbance program; that is, the selected scheme H4T4T6 is the self-resistance scheme.

[0116] In this embodiment, a micron-level precision visual motion tracking system includes an anti-off-plane disturbance device and an anti-off-plane disturbance program, and adopts the aforementioned anti-off-plane disturbance micron-level precision visual motion tracking method.

[0117] like Figure 6As shown, the self-resistant anti-disruption device includes an industrial camera 1, a microscope 2, a control card 4, and a piezoelectric driven compliant mechanism 3.

[0118] The control card 4 is electrically connected to the industrial camera 1, microscope 2, and piezoelectric drive compliant mechanism 3, respectively.

[0119] The industrial camera 1 and microscope 2 are used to acquire in-plane images with micron-level precision.

[0120] The control card 4 has an embedded self-resistance program for off-plane disturbances.

[0121] The control card 4 is used to control the industrial camera 1, microscope 2, and piezoelectric driven compliant mechanism 3.

[0122] The piezoelectric driven compliant mechanism 3 is used to generate out-of-plane displacement compensation.

[0123] The evaluation function real-time solution algorithm is used to convert off-plane disturbances into off-plane displacements. The tracking object off-plane motion suppression method is used to control the self-resisting off-plane disturbance device to generate controllable axial motion to compensate for off-plane displacements. The algorithm category / computation optimization method is used to control the mapping relationship between the self-resisting off-plane disturbance program and the corresponding off-plane displacement to the visual algorithm, and the off-plane displacement to the controllable parameters of the visual algorithm. The hardware and software scheme based on fuzzy rules synchronous linkage is used to realize self-resisting off-plane disturbances. The standard test experiment evaluates the accuracy improvement effect of the selected scheme H4T4T6.

[0124] Scheme T4 focuses on two images acquired consecutively within a short period of time. Based on the distance and direction of movement of industrial camera 1 during the focusing process, the direction, magnitude, and speed of the displacement from the surface can be calculated.

[0125] More specifically, the steps for scheme T4 to convert out-of-plane disturbances into out-of-plane displacements are as follows:

[0126] S101: Acquire the motion image at time t0;

[0127] S102: Obtain the current sharpness evaluation value through the sharpness evaluation function preset in the self-resistant out-of-plane perturbation procedure, and set it as the maximum sharpness value F. max ;

[0128] S103: The industrial camera 1, controlled by the self-resisting surface disturbance device, moves downward a distance s;

[0129] S104: Obtain the sharpness evaluation value after the movement through the preset sharpness evaluation function, and record it as F1;

[0130] S105: Determine the current sharpness rating value and F max Size, set threshold λ, if |Δ|=|F1-F max |>λ and Δ=F1-F max>0, proceed to step S106;

[0131] If |Δ|=|F1-F max If |≤λ, proceed to step S107;

[0132] If |Δ|=|F1-F max |>λ and Δ=F1-F max If the value is less than 0, proceed to step S108.

[0133] S106: Real-time update of maximum resolution, allowing F max =F1, and control the industrial camera 1 to continue moving downwards a distance s;

[0134] S107: End focusing, record the current position of industrial camera 1;

[0135] S108; Keeping the maximum sharpness constant, control the industrial camera 1 to move upward a distance s;

[0136] S109: Obtain the motion image at time t0+Δt, and repeat S102 to S108;

[0137] S110: Set the direction of movement of industrial camera 1 away from the plane as the positive direction and the direction of movement towards the plane as the negative direction. Based on the distances x0 and x1 moved by industrial camera 1 during the two focusing processes, calculate the magnitude, direction and velocity of the displacement from the plane according to the calculation formulas d=x0-x1 and v=d / Δt.

[0138] More specifically, the sharpness evaluation function used in the calculation of out-of-plane displacement is the variance function. However, in practical applications, it is not limited to the sharpness evaluation function mentioned above. Other sharpness evaluation functions, such as the Brenner function, the Laplacian gradient function, and the Tenengrad evaluation function, can be used to meet different requirements.

[0139] More specifically, the method for generating controllable axial motion to compensate for off-plane displacement is as follows:

[0140] The calculated magnitude, direction, and velocity of the out-of-plane displacement are converted into displacement compensation signals. Control card 4 inputs these signals into the piezoelectrically driven compliant mechanism 3, which generates axial displacement based on the compensation signals to compensate for the out-of-plane displacement, thus achieving hardware-level anti-out-of-plane disturbance. Specifically:

[0141] The conversion of axial runout frequency is as follows:

[0142] If v∈(A) n A n+1 ),but

[0143] The conversion of axial runout amplitude is as follows:

[0144] If d∈(B) n B n+1 ),but

[0145] Where d and v are the calculated out-of-plane displacement amplitude and velocity, respectively; δ1 and δ2 are the axial runout frequency and axial runout amplitude, respectively; n is the number of fuzzy mapping rules, i.e. Figure 2 , Figure 3 The number of dashed lines.

[0146] like Figure 7 As shown, based on the calculated displacement compensation signal, the piezoelectrically driven compliant mechanism 3 generates an axial runout frequency of δ1 and an axial runout amplitude of δ2, and compensates for the off-plane displacement with an axial motion opposite to the off-plane displacement.

[0147] More specifically, based on the off-plane displacement calculated by T4, the self-resistance off-plane disturbance program calls the visual algorithm and controllable parameters according to the mapping relationship to reduce the impact of off-plane disturbances at the software level:

[0148]

[0149] Where i = 1, 2, ..., n, d and v are the calculated displacement amplitude and velocity, respectively, x is the mapping relationship obtained from the previous experiment, and D i V i and X i R is a set consisting of d, v, and x respectively. i (x,y,z) is derived from D i ×V i To X i The fuzzy relationship is as follows:

[0150]

[0151] The mapping relationship between out-of-plane displacement, visual algorithm, and controllable parameters was obtained through preliminary experiments. The specific steps of the preliminary experiments are as follows:

[0152] S201: Obtain the visual motion tracking accuracy without out-of-plane perturbation, denoted as f0;

[0153] S202: Axial motion is generated, denoted as out-of-plane disturbance a;

[0154] S203: Preset template matching method based on grayscale and template matching method based on feature points;

[0155] S204: Extract controllable parameters from the gray-scale-based template matching method, and denote the number of templates n and the template area s as controllable parameters α1 and α2, respectively; extract controllable parameters from the feature point-based template matching method, and denote the Gaussian filter coefficient σ and the number of pyramid layers m as controllable parameters β1 and β2, respectively.

[0156] S205: Using a grayscale-based template matching method, the controllable parameters α1 and α2 are changed respectively to obtain the visual tracking accuracy under the first out-of-plane perturbation, denoted as f. 11 f 12 Using a feature-point-based template matching method, the controllable parameters β1 and β2 are changed respectively to obtain the visual tracking accuracy under the first out-of-plane perturbation, denoted as f. 21 f 22 ;

[0157] S206: Based on the different visual tracking accuracies f obtained above 11 f 12 f 21 f 22 The results of the accuracy repair ratio calculation are recorded as p1, p2, p3, and p4, respectively.

[0158] S207: Filter out the minimum accuracy repair ratio p min Establish the mapping relationship between the first out-of-plane displacement and the corresponding visual algorithm and parameters, denoted as the first mapping;

[0159] S208: Generate two other different axial movements, denoted as the second off-plane disturbance and the third off-plane disturbance; repeat S203-S207 above to establish the mapping between the off-plane displacement and the corresponding visual algorithm and parameters, denoted as the second mapping and the third mapping.

[0160] In this embodiment, the out-of-plane displacements a, b, and c input to the piezoelectric driven compliant mechanism 3 in the pre-experiment are three out-of-plane displacements with the same frequency but different amplitudes. The piezoelectric driven compliant mechanism 3 is used to generate controllable out-of-plane displacements based on the three simple harmonic excitation source signals D111, D112, and D113 with the same frequency but different amplitudes input from the control card 4.

[0161] More specifically, the mapping relationship x from off-plane displacement to visual algorithm and from off-plane displacement to controllable parameter is expressed as follows: the mapping relationship from off-plane displacement a to visual algorithm and from off-plane displacement a to controllable parameter is mapping one, denoted as x = 1; and so on, the mapping relationships corresponding to off-plane displacement b and off-plane displacement c are denoted as x = 2 and x = 3, respectively.

[0162] More specifically, by comparing the obtained visual motion tracking results with the detection results of a third-party nanometer-precision sensor, the visual motion tracking accuracy under out-of-plane displacement perturbation is obtained, and the accuracy optimization ratio is calculated, as shown in the following expression:

[0163] P = 100% × |f ij -f0|,i,j=1,2,3···,n

[0164] Among them, f ij f0 represents the visual motion tracking accuracy under off-surface perturbation after changing the visual algorithm and controllable parameters; f0 represents the visual motion tracking accuracy without off-surface perturbation, which is regarded as the baseline accuracy.

[0165] Example 3

[0166] A micrometer-level precision visual motion tracking method that is self-resistant to off-plane perturbations includes the following steps:

[0167] S1: Construct a micrometer-level precision visual motion tracking and measurement system; the micrometer-level precision visual motion tracking and measurement system includes an anti-disruption device and an anti-disruption program;

[0168] S2: Acquire motion images of the observed object through a micron-level precision visual motion tracking measurement system and detect out-of-surface disturbances in real time;

[0169] S3: To address the degradation of visual accuracy caused by off-plane disturbances, select an active disturbance rejection scheme to optimize the active off-plane disturbance rejection device and the active off-plane disturbance rejection program;

[0170] S4: Based on the preset fuzzy rules, for low-frequency, high-amplitude off-plane disturbances, the accuracy is restored by optimizing the self-resistance off-plane disturbance device at the hardware level; for high-frequency, low-amplitude off-plane disturbances, the accuracy is restored by optimizing the self-resistance off-plane disturbance program at the software level; for wide-frequency or wide-amplitude mixed off-plane disturbances, the accuracy is restored by synchronously linking the self-resistance off-plane disturbance device and the self-resistance off-plane disturbance program.

[0171] In this embodiment, a micron-level precision visual motion tracking system includes an anti-off-plane disturbance device and an anti-off-plane disturbance program, and adopts the aforementioned anti-off-plane disturbance micron-level precision visual motion tracking method.

[0172] The self-resistant anti-disruption device includes an industrial camera 1, a microscope 2, a control card 4, and a piezoelectric driven compliant mechanism 3.

[0173] The control card 4 is electrically connected to the industrial camera 1, microscope 2, and piezoelectric drive compliant mechanism 3, respectively.

[0174] The industrial camera 1 and microscope 2 are used to acquire in-plane images with micron-level precision.

[0175] The control card 4 has an embedded self-resistance program for off-plane disturbances.

[0176] The control card 4 is used to control the industrial camera 1, microscope 2, and piezoelectric driven compliant mechanism 3.

[0177] The piezoelectric driven compliant mechanism 3 is used to generate out-of-plane displacement compensation.

[0178] This embodiment uses a combination of hardware and software to correct the degradation in tracking accuracy caused by different off-plane displacement disturbances. First, before actual operation, the custom off-plane displacement generated by the piezoelectrically driven compliant mechanism 3 is tested and trained using different vision algorithms and parameters to form a precise mapping from the off-plane displacement to the vision algorithm and parameters, which is then embedded in the control card 4. Next, in practical application, the value and direction of the off-plane displacement are calculated based on the sharpness evaluation function. The piezoelectrically driven compliant mechanism 3 generates axial motion compensation for the off-plane displacement based on the off-plane displacement compensation signal, achieving anti-off-plane disturbance at the hardware level. The control card 4 uses a self-anti-off-plane disturbance algorithm to call a preset mapping relationship, achieving anti-off-plane disturbance at the software level. Finally, based on preset fuzzy rules, hardware and software are synchronized to accurately correct the visual motion tracking accuracy.

[0179] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A method for visual motion tracking with micron-level precision that is self- resistant to off-plane disturbances, characterized in that, It comprises the following steps: S1: Construct a micron-level precision visual motion tracking measurement system; the micron-level precision visual motion tracking measurement system comprises a self anti-off-plane disturbance device and a self anti-off-plane disturbance program; S2: Obtain the motion image of the observation object by the micron-level precision visual motion tracking measurement system, and detect the off-plane disturbance in real time; S3: For the visual precision degradation caused by off-plane disturbance, select a self-disturbance scheme to optimize the self anti-off-plane disturbance device and the self anti-off-plane disturbance program; the self anti-off-plane disturbance is specifically that for different types and levels of off-plane disturbance, corresponding self-disturbance schemes are adopted; The self-disturbance scheme is to optimize the self anti-off-plane disturbance device and the self anti-off-plane disturbance program by using different optimization methods for different off-plane disturbances; any optimization method of the self anti-off-plane disturbance device and the optimization method of a plurality of self anti-off-plane disturbance programs are selected to make the self anti-off-plane disturbance device and the self anti-off-plane disturbance program work synchronously; The optimization method of the self anti-off-plane disturbance device comprises a single piezoelectric drive low-frequency compensation method, a double piezoelectric drive high-frequency compensation method, a single piezoelectric drive objective lens focal plane method, and a tracking object off-plane motion suppression method; The optimization method of the self anti-off-plane disturbance program comprises a template variable type / quantity / area method, a region of interest real-time transformation method, an off-plane-degradation fuzzy set method, an evaluation function real-time solving method, a multi-focal plane real-time matching method, an algorithm category / computational optimization method, a frequency-precision dynamic balance method, and a deep learning tool parameter adjustment method; S4: According to the preset fuzzy rule, for low-frequency high-amplitude off-plane disturbance, the self anti-off-plane disturbance device is optimized on the hardware level to realize precision repair; for high-frequency low-amplitude off-plane disturbance, the self anti-off-plane disturbance program is optimized on the software level to realize precision repair; for wide-frequency or cross-amplitude mixed off-plane disturbance, the self anti-off-plane disturbance device and the self anti-off-plane disturbance program are used to realize precision repair.

2. The method of claim 1, wherein, According to the degree of visual precision degradation caused by off-plane disturbance, different off-plane disturbances are divided into the following five levels: For visual tracking precision degradation ratio L; If L 5%, it is optimal and no optimization is needed. If Good, optimization needed. If then is medium, optimization is needed; If then it is different and needs to be optimized; If , the tracking accuracy is lost and optimization cannot be performed.

3. The method of claim 2, wherein, After selecting the self-disturbance scheme, standard test experiments are carried out, the precision improvement effect of the micron-level precision visual motion tracking under different off-plane disturbances is calculated for different selected schemes, the off-plane excitation-visual self-disturbance mapping is formed according to the evaluation results, and the mapping is quantified; The standard test experiment is as follows: S301: Design 20 motion tracking experiments; S302: In the first to fifth experiments, no off-plane disturbance is added when visual motion tracking is performed, and the standard accuracy of visual motion tracking is obtained ; S303: In the first i Visual motion tracking is performed in the second experiment, and the obtained off-plane disturbance is added to obtain the visual motion tracking precision ; wherein 6≤ i ≤15; S304: In the first i +1 i +4 experiments, the off-plane disturbance is added to the acquired visual motion tracking, the corresponding matching scheme is used for precision improvement, and the improved precision of visual motion tracking is obtained ; S305: In the first i The standard accuracy of visual motion tracking is obtained without adding any off-plane disturbance in 5~20 experiments ; S306: Calculate the precision results obtained in S302-S305, and obtain the improvement effect of this selected scheme according to the evaluation formula.

4. The method of claim 3, wherein, The tracking object off-plane motion suppression method is used for the self anti-off-plane disturbance device; the evaluation function real-time solving method, the algorithm category / computational optimization method are used for the self anti-off-plane disturbance program; The evaluation function real-time solving method is used to solve the off-plane disturbance into off-plane displacement, the tracking object off-plane motion suppression method is used to control the self anti-off-plane disturbance device to generate controllable axial motion compensation off-plane displacement, and the algorithm category / computational optimization method is used to control the self anti-off-plane disturbance program to call the mapping relationship of the corresponding off-plane displacement to the visual algorithm and the controllable parameter.

5. The method of claim 4, wherein, The steps of solving the off-plane disturbance into off-plane displacement are as follows: S101: acquire time-lapse moving image; S102: Obtain the current definition evaluation value by the definition evaluation function preset in the self anti-off surface disturbance program, and set it as the maximum definition value ; S103: Control the industrial camera (1) in the anti-off-plane disturbance device to move downward by a distance s; S104: Obtain the sharpness evaluation value after the movement by a preset sharpness evaluation function, denoted as ; S105: judging whether the current definition evaluation value is greater than the threshold value size, setting a threshold value , if and , executing step S106; If , step S107 is executed. If and step S108 is executed. S106: update the maximum value of the definition in real time, let = , and control the industrial camera (1) to continue to move down by a distance s; S107: End the focusing and record the current position of the industrial camera (1); S108: Keep the maximum sharpness unchanged and control the industrial camera (1) to move upward by a distance s; S109: Acquire The time-lapse moving image is acquired, and S102 to S108 are repeated. S110: Set the moving direction of the industrial camera (1) away from the plane as the positive direction, and the moving direction of the industrial camera (1) close to the plane as the negative direction, and calculate the distance moved by the industrial camera (1) during the two focusing processes according to the distance With , according to the calculation formula And , the size, direction and speed of the displacement from the plane are calculated.

6. The method of claim 5, wherein, The method for generating controllable axial motion compensation off-plane displacement is as follows: The size, direction and speed of the calculated off-plane displacement are converted into a displacement compensation signal, and an axial displacement is generated according to the displacement compensation signal to achieve anti-off-plane disturbance at the hardware level, specifically as follows: The conversion of the axial runout frequency is specifically as follows: If v (A n , A n+1 ), then ; The conversion of the axial runout amplitude is specifically as follows: If d (B n , B n+1 ), then ; wherein, d with respectively the calculated out-of-plane displacement amplitude and velocity, and respectively the axial run-out frequency and axial run-out amplitude; n is the number of fuzzy mapping rules; According to the displacement compensation signal after conversion, the axial run-out frequency is , the axial run-out amplitude is , and the axial movement compensates for the out-of-plane displacement in the opposite direction.

7. The method of claim 6, wherein, According to the calculated off-plane displacement, the anti-off-plane disturbance program calls the visual algorithm and controllable parameters according to the mapping relationship: wherein, , d and v are the calculated amplitude and velocity of the out-of-plane displacement respectively, is the mapping relationship obtained from the pre-experiment, , and are the set consisting of d , v and x , is the fuzzy relationship from to , specifically: The mapping relationship of the off-plane displacement to the visual algorithm and controllable parameters is obtained through pre-experiment, and the specific steps of the pre-experiment are as follows: S201: Obtain visual motion tracking accuracy when there is no off-plane disturbance, denoted as ; S202: Generate axial motion, denoted as off-plane disturbance a; S203: Pre-set gray-based template matching method and feature point-based template matching method; S204: Extract controllable parameters from the gray-based template matching method, and record the template number n and the template area s as controllable parameters , controllable parameters ; Extract controllable parameters from the feature point-based template matching method, and record the Gaussian filter coefficient and the pyramid layer number m as controllable parameters , controllable parameters ; S205: Change the controllable parameters respectively by using the gray-based template matching method and the controllable parameters , to obtain the visual tracking accuracy under the first out-of-plane disturbance, denoted as ; Change the controllable parameters respectively by using the feature point-based template matching method and the controllable parameters , to obtain the visual tracking accuracy under the first out-of-plane disturbance, denoted as ; S206: Based on the different visual tracking accuracy obtained above , the accuracy repair ratio calculation result is recorded as ; S207: screen out the minimum value of precision repair proportion , a mapping relationship between the first off-face displacement and the corresponding vision algorithm and parameters is established, denoted as a first mapping; S208: Generate two other different axial motions, denoted as second off-plane disturbance and third off-plane disturbance; repeat the above S203-S207 to establish the mapping of the off-plane displacement to the corresponding visual algorithm and parameters, denoted as second mapping and third mapping.

8. The method of claim 7, wherein, By comparing the obtained visual motion tracking result with the detection result of the third-party nanometer-level precision sensor, the visual motion tracking accuracy under the off-plane displacement disturbance is obtained, and the accuracy optimization ratio is calculated, and the expression is as follows: wherein, represents the visual motion tracking accuracy under off-plane disturbance after changing the visual algorithm and controllable parameters; represents the visual motion tracking accuracy without off-plane disturbance, which is regarded as the reference accuracy.

Images