All-directional vibration measurement method and system for thin-walled workpiece based on binocular vision intelligent continuous tracking

Abstract

The application belongs to the technical field of vibration measurement, and discloses a kind of omnidirectional vibration measurement method and system of thin-walled piece based on binocular vision intelligent continuous tracking, and the core technology is improved from multiple dimensions.The improvements are not isolated, but need to be designed in coordination to form a unified, end-to-end enhanced framework.A lightweight adaptive dual-mode tracking architecture is constructed to improve efficiency.Through this design, 99% of inter-frame tracking is completed by an efficient front-end, and only in 1% of critical challenge moments, a computationally expensive back-end is started.This not only significantly reduces the average computational delay, enabling real-time processing, but more importantly, it concentrates valuable computing resources on solving the most difficult tracking problems, thereby achieving an optimal balance between efficiency and accuracy at the system level.

Description

Technical Field

[0001] This invention belongs to the field of vibration measurement technology, and in particular relates to a method and system for measuring the omnidirectional vibration of thin-walled components based on binocular vision intelligent continuous tracking. Background Technology

[0002] In the process of omnidirectional vibration measurement of thin-walled parts based on binocular vision, it is necessary to stably and accurately track multiple feature points on the surface of the thin-walled parts for a long time to obtain high-fidelity vibration data of the thin-walled parts.

[0003] Traditional optical flow tracking methods, such as Lucas-Kanade and its variants, are based on the assumption of gray-level conservation. They estimate the motion vector of a pixel by calculating the spatiotemporal gradient of pixel intensity between adjacent image frames. However, this assumption reveals many inherent defects and shortcomings when applied to the omnidirectional vibration measurement of thin-walled parts. (1) The primary shortcoming is its high sensitivity to changes in illumination. Thin-walled parts are generally metal parts, and the metal surface is prone to high reflectivity, specular reflection, and shadows, resulting in a large nonlinear change in surface luster. This directly violates the basic premise of gray-level conservation on which optical flow tracking methods rely, causing the signal-to-noise ratio of the tracking point to drop sharply or even be lost. (2) Secondly, the optical flow method is essentially a short-time, local matching technique. It lacks long-term memory and global semantic understanding capabilities. Once the tracking point is briefly occluded, blurred due to rapid surface movement, or moves out of the field of view, it cannot be recaptured. This is fatal for omnidirectional vibration measurement that requires a long duration. (3) Furthermore, the tracking performance of optical flow tracing methods drops sharply in areas with poor texture. The metal surface of thin-walled parts may have been polished or oxidized, lacking rich corner points or texture features, making optical flow calculations extremely unstable and unreliable due to insufficient gradient information. (4) Finally, optical flow tracing methods based on handcrafted features have difficulty effectively distinguishing different feature points with similar appearances. During vibration measurement, when multiple points have similar motion patterns or intersect, the inability to effectively distinguish feature points can easily lead to confusion of feature point identities, resulting in erroneous measurement results.

[0004] Deep learning-based continuous tracking methods, such as PIPs and their variants, offer a novel approach to addressing the shortcomings and deficiencies of traditional optical flow tracking methods, demonstrating numerous advantages. (1) The core advantage of the PIPs method lies in its abandonment of the traditional gray-scale conservation assumption, instead learning more robust feature representations from the data. It extracts features with advanced semantic information through deep neural networks. These features are highly invariant to changes in illumination, minor deformations, and appearance changes, effectively overcoming interference from reflections on metal surfaces. (2) PIPs employs a cyclic iterative update mechanism and spatiotemporal memory. It not only predicts and updates the trajectory based on the current frame but also integrates information from multiple historical frames, giving it powerful long-term tracking capabilities and trajectory persistence. Even if the target point is briefly occluded or disappears for several frames, PIPs can still associate and re-identify it when it reappears, based on its learned motion priors and appearance memory, greatly ensuring the integrity of the vibration data sequence. (3) PIPs predict the position of all points in all frames simultaneously by processing a short video segment (rather than just two adjacent frames). This implicit global optimization can effectively solve the ID Preservation problem when targets cross, providing a solid foundation for analyzing the independent vibration modes of each feature point. (4) The end-to-end training method of PIPs also enables the model to learn how to deal with complex scenes directly from a large amount of data. Ultimately, it surpasses traditional optical flow tracking methods in terms of tracking accuracy, long-term stability, and adaptability to challenging scenes.

[0005] However, despite the significant advantages of the PIPs method, it was not developed specifically for the vibration measurement of thin-walled components. Directly applying it to the specific working condition of omnidirectional vibration measurement of thin-walled components still has some significant shortcomings and deficiencies. (1) The primary problem is the challenge of computational efficiency and real-time performance. Vibration measurement usually requires high frame rate acquisition. PIPs and its subsequent improved versions are complex and computationally burdensome, making it difficult to meet the needs of online or real-time vibration monitoring. (2) Secondly, the ability to estimate extremely high-precision sub-pixel displacement needs to be strengthened. Vibration measurement often requires sub-pixel level displacement resolution. The PIPs network design was initially optimized for general target tracking, and its output accuracy may not be able to directly meet the extremely high signal-to-noise ratio requirements for displacement derivatives in vibration analysis. (3) Dependence on the distribution of training data domain. Currently, PIPs are mostly trained on natural scene datasets. The features they learn may differ from the image domain of industrial scenes, and they may not perform well in terms of the unique texture and reflection characteristics of metals, resulting in domain adaptation problems. (4) Lack of embedding and constraint of physical laws. The motion of vibration points is not entirely independent and random, but is constrained by the laws of physics and mechanics. Purely data-driven PIPs do not incorporate such prior knowledge and may produce physically unreliable trajectories in cases of data scarcity or extreme situations.

[0006] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0007] (1) The primary problem is the challenge of computational efficiency and real-time performance. Vibration measurement usually requires high frame rate acquisition. PIPs and their subsequent improved versions have complex models and high computational load, making it difficult to meet the needs of online or real-time vibration monitoring.

[0008] (2) The ability to estimate subpixel displacement with extremely high precision needs to be improved. Vibration measurement often requires subpixel-level displacement resolution. The network design of PIPs was originally optimized for general target tracking, and its output accuracy may not be able to directly meet the extremely high signal-to-noise ratio requirements for displacement derivatives in vibration analysis.

[0009] (3) Dependence on the distribution of training data domain. Currently, most PIPs are trained on natural scene datasets. The features they learn may differ from the image domain of industrial scenes. They may not perform well in terms of the unique texture and reflection characteristics of metals, resulting in a domain adaptation problem.

[0010] (4) Lack of embedding and constraint of physical laws. The motion of vibration points is not completely independent and random, but is constrained by the laws of physics and mechanics. Purely data-driven PIPs do not incorporate such prior knowledge, and may produce physically unreliable trajectories in cases of data scarcity or extreme situations. Summary of the Invention

[0011] To address the problems existing in the prior art, this invention provides a method and system for measuring the omnidirectional vibration of thin-walled components based on binocular vision intelligent continuous tracking.

[0012] This invention is implemented as follows: A method for measuring the omnidirectional vibration of thin-walled components based on binocular vision intelligent continuous tracking includes:

[0013] Step 1: Construct a lightweight adaptive dual-mode tracking architecture;

[0014] Step 2: Introduce sub-pixel-level regression and frequency domain loss function;

[0015] Step 3: Implement unsupervised domain adaptation and stylized data augmentation;

[0016] Step 4: Embed the differentiable physics engine and vibration model.

[0017] Furthermore, the lightweight adaptive dual-mode tracking architecture is constructed as follows:

[0018] The architecture consists of a lightweight high-speed tracker and an enhanced PIPs module. The lightweight tracker uses an extremely simplified optical flow network, whose responsibility is to perform displacement prediction between frames. It can maintain tracking with extremely low computational cost between most frames with clear textures and no drastic changes.

[0019] The system runs a confidence assessment module; this module analyzes the output of the lightweight tracker in real time, and the evaluation indicators include: image patch correlation score of the tracked point, local contrast, and consistency of motion of adjacent points; once the confidence of a point or a group of points is lower than a preset threshold, the system will automatically trigger the enhanced PIPs module; the PIPs module here does not need to process the entire long sequence, but instead takes the current low-confidence frame as the center, extracts a short temporal window, and performs fine-grained trajectory correction and reassociation.

[0020] Furthermore, the introduction of sub-pixel-level regression and frequency domain loss function:

[0021] First, the output head design at the end of the decoder is improved. Instead of directly regressing absolute coordinates, a sub-pixel regression head based on a learnable upsampling module is introduced at the end of the decoder. This module takes the high-dimensional features extracted by the PIPs backbone network as input, first predicts a low-resolution displacement field, and then gradually upsamples it through sub-pixel convolutional layers to generate a high-resolution, high-precision displacement vector map. This allows the network to explicitly learn how to infer the precise location of the center point from the semantic information of the surrounding pixels, breaking through the limitations of the physical resolution of the image.

[0022] Secondly, at the loss function level, a frequency domain consistency constraint is introduced. Vibration signals are sparsity in the frequency domain, and this prior knowledge is used to guide network training. Specifically, for the temporal displacement sequence of each point predicted by the network, a fast Fourier transform is performed to calculate its power spectrum. Then, a frequency domain loss term is constructed, which encourages the power spectrum of the predicted signal to be as consistent as possible with an ideal smooth spectrum, while suppressing the energy of out-of-band noise. This loss term is combined with the original trajectory regression loss of PIPs for backpropagation optimization. In this way, the network not only learns how to minimize coordinate errors, but also learns how to generate a "physically more credible" and purer vibration signal, thereby fundamentally improving the accuracy of its sub-pixel estimation.

[0023] Furthermore, the implementation of unsupervised domain adaptation and stylized data enhancement is as follows:

[0024] PIPs are trained on natural video datasets, which have significant domain differences from industrial scenarios; the core method is adversarial domain adaptation.

[0025] In the improved PIPs framework, in addition to the backbone tracking network, an additional domain discriminator is introduced. The goal of this discriminator is to distinguish whether the features come from the natural image source domain or the metal target domain. The goal of the backbone feature extractor is to "deceive" the domain discriminator as much as possible while ensuring tracking performance, so that it cannot distinguish the source of the features. Through this adversarial training, the backbone network is forced to extract domain-invariant features. These features are not sensitive to changes in texture and lighting style, and only focus on geometric structure and motion patterns, thereby significantly improving the model's generalization ability on metal parts.

[0026] Simultaneously, we implemented stylized data augmentation based on generative adversarial networks; trained a style transfer model to convert images from source datasets such as COCO into images with typical metallic appearances; and trained the model together with this massive amount of stylized "pseudo-metal data" and a small amount of real metal labeled data. This is equivalent to exposing the network to various possible metal visual characteristics in advance during the training process, greatly enhancing the model's adaptability to real testing environments and reducing its dependence on scarce labeled data.

[0027] Furthermore, the embedded differentiable physics engine and vibration model:

[0028] Combining data-driven and model-driven approaches; the motion of vibration points is constrained by physical laws, and their trajectories should satisfy smoothness and a certain dynamic model; the improvement scheme is to embed a differentiable physical consistency constraint module into the network;

[0029] This module, located after the PIPs network, receives the original trajectory sequence predicted by the network. Assuming that the motion of each point can be described by a damped vibration model, this module implements a differentiable ODE solver. It attempts to fit the trajectory predicted by the network to this physical model and solve for the optimal model parameters for each point.

[0030] Then, two quantities are calculated: the difference between the network-predicted trajectory and the optimal trajectory reconstructed by the physical model, and the rationality of the physical model parameters; these two quantities together constitute a physical loss term.

[0031] During training, this physical loss term acts as a regularizer, working together with the original loss of PIPs to optimize the network. During backpropagation, the network receives not only the "error between predicted coordinates and ground truth coordinates" signal, but also the "whether the prediction is physically plausible" signal. This drives the network to learn an internal representation that conforms to physical laws, and its prediction results will naturally tend to be smooth, continuous, and conform to vibration laws, even without ground truth.

[0032] During the inference phase, the physics module can act as a post-processing optimizer to smooth and correct the network's raw output, especially in segments that are occluded or disturbed by noise. It uses learned physical priors to perform interpolation and extrapolation to generate a more reliable and reasonable trajectory output.

[0033] Another objective of this invention is to provide a binocular vision-based intelligent continuous tracking omnidirectional vibration measurement system for thin-walled components, comprising:

[0034] Build modules for constructing lightweight adaptive dual-mode tracking architectures;

[0035] An introduction module is used to introduce subpixel-level regression and frequency domain loss functions;

[0036] The implementation module is used to implement unsupervised domain adaptation and stylized data augmentation;

[0037] Embedded modules are used to embed differentiable physics engines and vibration models.

[0038] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the method for measuring the omnidirectional vibration of thin-walled components based on binocular vision intelligent continuous tracking.

[0039] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method for measuring the omnidirectional vibration of thin-walled components based on binocular vision intelligent continuous tracking.

[0040] Another objective of this invention is to provide an information data processing terminal for implementing the thin-walled component omnidirectional vibration measurement system based on binocular vision intelligent continuous tracking.

[0041] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0042] This invention improves the core technology from multiple dimensions. These improvements are not isolated, but require collaborative design to form a unified, end-to-end enhanced framework.

[0043] (1) Construct a lightweight adaptive dual-mode tracking architecture to improve efficiency;

[0044] With this design, 99% of inter-frame tracking is performed by the efficient front-end, with the computationally expensive back-end only activated during the 1% of critical challenging moments. This not only significantly reduces the average computational latency and achieves near real-time processing, but more importantly, it concentrates valuable computing resources on solving the most difficult tracking problems, thus achieving an optimal balance between efficiency and accuracy at the system level.

[0045] (2) Introduce subpixel-level regression and frequency domain loss function to improve accuracy;

[0046] (3) Implement unsupervised domain adaptation and stylized data augmentation to improve generalization ability;

[0047] We implemented stylized data augmentation based on generative adversarial networks. We trained a style transfer model that can transform images from source datasets such as COCO into images with typical metallic appearances. This massive amount of stylized "pseudo-metallic data" was trained alongside a small amount of real metal-annotated data. This effectively broadened the network's exposure to various possible metallic visual characteristics during training, significantly enhancing the model's adaptability to real-world testing environments and reducing its reliance on scarce labeled data.

[0048] (4) Embed a differentiable physics engine and vibration model to improve trajectory rationality;

[0049] This is the most innovative improvement of the invention, aiming to combine data-driven and model-driven approaches. The motion of the vibration point is constrained by physical laws, and its trajectory should satisfy smoothness and a certain dynamic model. The improved solution is to embed a differentiable physical consistency constraint module into the network.

[0050] In summary, traditional optical flow tracking methods face fundamental challenges in vibration measurement of thin-walled components due to limitations in model assumptions. PIPs-like intelligent continuous tracking methods offer a superior solution due to their data-driven approach and long-term modeling capabilities. However, to maximize their effectiveness in industrial measurement practice, systematic and in-depth improvements must be made from multiple theoretical dimensions, including lightweight design, accuracy enhancement, domain adaptation and transfer, and physical model embedding, to address shortcomings such as computational efficiency, sub-pixel accuracy, domain variability, and physical consistency. Ultimately, this will lead to the construction of an efficient, accurate, robust, and physically reliable omnidirectional vibration measurement system for thin-walled components.

[0051] (1) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows:

[0052] Thin-walled components, with their advantages of light weight, compact structure, and strong load-bearing capacity, have been widely used in engineering fields. The technical solution of this invention is mainly applied to typical thin-walled components such as aero-engine blades, car body hoods, and satellite battery panels. Thin-walled components also have characteristics such as low relative stiffness, weak strength, and large size, which can easily cause noise, instability, and other problems. During operation, the equipment may even cause serious mechanical failures, leading to major safety accidents. Therefore, it is necessary to perform efficient and accurate vibration measurement of thin-walled components to achieve real-time monitoring. Due to the very complex structure of thin-walled components, the following three special requirements are put forward for vibration measurement methods: (1) Non-contact vibration measurement to avoid the influence of added mass on dynamic characteristics; (2) Single-point omnidirectional vibration measurement; (3) High spatial resolution and high precision. Traditional contact vibration testing methods, single-point laser single-point vibration measurement methods, and single-point laser continuous scanning vibration measurement methods are all difficult to meet the special requirements of omnidirectional vibration measurement of thin-walled components. This invention provides a binocular vision-based omnidirectional vibration measurement method for thin-walled components, meeting the requirements for non-contact, omnidirectional, high spatial resolution, and high-precision vibration measurement. The key to this method is the binocular vision-based measurement point tracking approach. This invention is highly attractive to its target audience, with urgent market demand and promising prospects. The specific benefits of this invention will rely on product sales, technical support, and subsequent product and technology updates and after-sales service.

[0053] (2) The technical solution of this invention fills a technical gap in the industry both domestically and internationally:

[0054] In the process of omnidirectional vibration measurement of thin-walled parts based on binocular vision, it is necessary to stably and accurately track multiple feature points on the surface of the thin-walled part for a long time to obtain high-fidelity vibration data. Traditional optical flow tracking methods face fundamental challenges in the vibration measurement of thin-walled parts due to the limitations of model assumptions. PIPs-like intelligent continuous tracking methods offer a better solution due to their data-driven and long-term modeling advantages. However, to maximize their effectiveness in industrial measurement practice, their shortcomings in computational efficiency, sub-pixel accuracy, domain differences, and physical consistency must be addressed. This invention systematically and deeply improves upon these shortcomings from multiple theoretical dimensions, including lightweight design, accuracy enhancement, domain adaptation transfer, and physical model embedding, constructing an efficient, accurate, robust, and physically reliable omnidirectional vibration measurement system for thin-walled parts, thereby fully realizing the enormous potential of machine vision-based non-contact vibration measurement.

[0055] (3) Whether the technical solution of the present invention solves the technical problem that people have long wanted to solve but have never been able to solve successfully:

[0056] The demand for omnidirectional vibration measurement of typical thin-walled components such as aero-engine blades, automotive body-in-white hoods, and satellite solar panels is urgent. Binocular vision-based omnidirectional vibration measurement of thin-walled components can meet the requirements of non-contact, omnidirectional, high spatial resolution, and high precision measurement. The key is to stably and accurately track multiple feature points on the surface of the thin-walled component over a long period to obtain high-fidelity vibration data. Traditional optical flow tracking methods face fundamental challenges in thin-walled component vibration measurement due to limitations in model assumptions. PIPs-like intelligent continuous tracking methods offer a better solution due to their data-driven and long-term modeling advantages. However, to maximize their effectiveness in industrial measurement practice, their shortcomings in computational efficiency, sub-pixel accuracy, domain variability, and physical consistency must be addressed. These are technical challenges that the industry has long sought to solve but has yet to achieve success in. This invention systematically and deeply improves upon these shortcomings from multiple theoretical dimensions, including lightweight design, precision enhancement, domain adaptation transfer, and physical model embedding, constructing an efficient, accurate, robust, and physically reliable omnidirectional vibration measurement system for thin-walled components, thereby fully realizing the enormous potential of machine vision-based non-contact vibration measurement.

[0057] (4) Does the technical solution of the present invention overcome technical bias?

[0058] The demand for omnidirectional vibration measurement of typical thin-walled components such as aero-engine blades, automotive body-in-white hoods, and satellite solar panels is urgent. Binocular vision-based omnidirectional vibration measurement of thin-walled components can meet the requirements of non-contact, omnidirectional, high spatial resolution, and high precision measurement. The key is the stable and accurate long-term tracking of multiple feature points on the surface of the thin-walled component to obtain high-fidelity vibration data. However, current binocular vision-based intelligent tracking methods suffer from core shortcomings such as computational efficiency, sub-pixel accuracy, domain variability, and physical consistency. Therefore, the industry generally has significant doubts about the application capabilities and level of binocular vision-based omnidirectional vibration measurement methods for thin-walled components. This invention addresses these core shortcomings by systematically and deeply improving the technology from multiple theoretical dimensions, including lightweight design, precision enhancement, domain adaptation transfer, and physical model embedding. Its application capabilities and level have been fully verified, completely overcoming industry prejudices and eliminating doubts about its application. Attached Figure Description

[0059] Figure 1 This is a flowchart of the method for measuring omnidirectional vibration of thin-walled components based on binocular vision intelligent continuous tracking, provided in an embodiment of the present invention.

[0060] Figure 2 This is a structural block diagram of a thin-walled component omnidirectional vibration measurement system based on binocular vision intelligent continuous tracking, provided in an embodiment of the present invention.

[0061] Figure 3This is a diagram of the vibration measurement experimental platform for thin-walled components provided in an embodiment of the present invention.

[0062] Figure 4 This is a vibration displacement time history diagram of feature point #1 provided in an embodiment of the present invention. (a) X-axis displacement; (b) Y-axis displacement; (c) Z-axis displacement diagram.

[0063] Figure 5 This is the Y-axis displacement spectrum diagram of feature point 1# provided in the embodiment of the present invention.

[0064] Figure 6 This is a comparison between the method of the present invention provided in the embodiments of the present invention and the theoretical vibration displacement. (a) X-axis displacement; (b) Y-axis displacement; (c) Z-axis displacement diagram. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0066] like Figure 1 As shown, the omnidirectional vibration measurement method for thin-walled components based on binocular vision intelligent continuous tracking provided by this embodiment of the invention includes the following steps:

[0067] S101, building a lightweight adaptive dual-mode tracking architecture;

[0068] S102 introduces sub-pixel-level regression and frequency domain loss function;

[0069] S103, Implement unsupervised domain adaptation and stylized data augmentation;

[0070] S104, embedding a differentiable physics engine and vibration model.

[0071] The present invention provides a lightweight adaptive dual-mode tracking architecture:

[0072] The original PIPs model is computationally intensive because it requires multiple iterations of updates to the entire video segment; for vibration measurement, this computation mode is difficult to perform in real time; the core idea of ​​the improvement is to change from "high energy throughout" to "on-demand start", and build a dual-mode tracking system consisting of a lightweight front-end and a semantically strong back-end;

[0073] Specifically, the architecture consists of a lightweight high-speed tracker and an enhanced PIPs module; the lightweight tracker uses an extremely simplified optical flow network, whose responsibility is to perform displacement prediction between frames, and can maintain tracking with extremely low computational cost between most frames with clear textures and no drastic changes.

[0074] Meanwhile, the system runs a confidence evaluation module; this module analyzes the output of the lightweight tracker in real time, and the evaluation indicators include: image patch correlation score of the tracked point, local contrast, and consistency of motion of adjacent points; once the confidence of a point or a group of points is lower than a preset threshold, the system will automatically trigger the enhanced PIPs module; the PIPs module here does not need to process the entire long sequence, but instead takes the current low-confidence frame as the center, extracts a short temporal window, and performs fine-grained trajectory correction and re-association;

[0075] With this design, 99% of inter-frame tracking is completed by the efficient front end, and the computationally expensive back end is only activated during the 1% of critical challenge moments. This not only significantly reduces the average computation latency and achieves near real-time processing, but more importantly, it concentrates valuable computing resources on solving the most difficult tracking problems, thereby achieving the optimal balance between efficiency and accuracy at the system level.

[0076] The present invention provides an introduction of sub-pixel-level regression and frequency domain loss functions:

[0077] Vibration measurement requires displacement resolution at the sub-pixel level or even higher; the output accuracy of original PIPs is often limited by the resolution of the feature map and the simple coordinate regression method; the improvement takes a two-pronged approach from both the network output head and the loss function.

[0078] First, the output head design at the end of the decoder is improved. Instead of directly regressing absolute coordinates, a sub-pixel regression head based on a learnable upsampling module is introduced at the end of the decoder. This module takes the high-dimensional features extracted by the PIPs backbone network as input, first predicts a low-resolution displacement field, and then gradually upsamples it through sub-pixel convolutional layers to generate a high-resolution, high-precision displacement vector map. This allows the network to explicitly learn how to infer the precise location of the center point from the semantic information of the surrounding pixels, breaking through the limitations of the physical resolution of the image.

[0079] Secondly, at the loss function level, a frequency domain consistency constraint is introduced. Vibration signals are sparsity in the frequency domain, and this prior knowledge is used to guide network training. Specifically, for the temporal displacement sequence of each point predicted by the network, a fast Fourier transform is performed to calculate its power spectrum. Then, a frequency domain loss term is constructed, which encourages the power spectrum of the predicted signal to be as consistent as possible with an ideal smooth spectrum, while suppressing the energy of out-of-band noise. This loss term is combined with the original trajectory regression loss of PIPs for backpropagation optimization. In this way, the network not only learns how to minimize coordinate errors, but also learns how to generate a "physically more credible" and purer vibration signal, thereby fundamentally improving the accuracy of its sub-pixel estimation.

[0080] The embodiments of this invention provide implementations of unsupervised domain adaptation and stylized data enhancement:

[0081] PIPs are trained on natural video datasets, which have significant domain differences from industrial scenarios; the core method is adversarial domain adaptation.

[0082] In the improved PIPs framework, in addition to the backbone tracking network, an additional domain discriminator is introduced. The goal of this discriminator is to distinguish whether the features come from the natural image source domain or the metal target domain. The goal of the backbone feature extractor is to "deceive" the domain discriminator as much as possible while ensuring tracking performance, so that it cannot distinguish the source of the features. Through this adversarial training, the backbone network is forced to extract domain-invariant features. These features are not sensitive to changes in texture and lighting style, and only focus on geometric structure and motion patterns, thereby significantly improving the model's generalization ability on metal parts.

[0083] Simultaneously, we implemented stylized data augmentation based on generative adversarial networks; trained a style transfer model that can transform images from source datasets such as COCO into images with typical metallic appearances; and trained the model together with this massive amount of stylized "pseudo-metal data" and a small amount of real metal labeled data. This is equivalent to exposing the network to various possible metal visual characteristics in advance during the training process, greatly enhancing the model's adaptability to real testing environments and reducing its dependence on scarce labeled data.

[0084] The embedded differentiable physics engine and vibration model provided in this embodiment of the invention:

[0085] This is the most innovative improvement of the present invention, which aims to combine data-driven and model-driven approaches; the motion of the vibration point is constrained by physical laws, and its trajectory should satisfy smoothness and a certain dynamic model; the improvement scheme is to embed a differentiable physical consistency constraint module into the network.

[0086] This module, located after the PIPs network, receives the original trajectory sequence predicted by the network. Assuming that the motion of each point can be described by a damped vibration model, this module implements a differentiable ODE solver. It attempts to fit the trajectory predicted by the network to this physical model and solve for the optimal model parameters for each point.

[0087] Then, two quantities are calculated: the difference between the network-predicted trajectory and the optimal trajectory reconstructed by the physical model, and the rationality of the physical model parameters; these two quantities together constitute a physical loss term.

[0088] During training, this physical loss term acts as a regularizer, working together with the original loss of PIPs to optimize the network. During backpropagation, the network receives not only the "error between predicted coordinates and ground truth coordinates" signal, but also the "whether the prediction is physically plausible" signal. This drives the network to learn an internal representation that conforms to physical laws, and its prediction results will naturally tend to be smooth, continuous, and conform to vibration laws, even without ground truth.

[0089] During the inference phase, the physics module can act as a post-processing optimizer to smooth and correct the network's raw output, especially in segments that are occluded or interfered with by noise. It uses the learned physical priors to perform interpolation and extrapolation to generate a more reliable and reasonable trajectory output.

[0090] In summary, traditional optical flow tracking methods face fundamental challenges in vibration measurement of thin-walled components due to limitations in model assumptions. PIPs-like intelligent continuous tracking methods offer a superior solution due to their data-driven and long-term modeling advantages. However, to maximize their effectiveness in industrial measurement practice, it is necessary to address their shortcomings in computational efficiency, sub-pixel accuracy, domain differences, and physical consistency through systematic and in-depth improvements from multiple theoretical dimensions, including lightweight design, accuracy enhancement, domain adaptation and transfer, and physical model embedding. Ultimately, this will lead to the construction of an efficient, accurate, robust, and physically reliable omnidirectional vibration measurement system for thin-walled components.

[0091] like Figure 2 As shown, an embodiment of the present invention provides a thin-walled component omnidirectional vibration measurement system based on binocular vision intelligent continuous tracking, comprising:

[0092] Build modules for constructing lightweight adaptive dual-mode tracking architectures;

[0093] An introduction module is used to introduce subpixel-level regression and frequency domain loss functions;

[0094] The implementation module is used to implement unsupervised domain adaptation and stylized data augmentation;

[0095] Embedded modules are used to embed differentiable physics engines and vibration models.

[0096] In this embodiment of the invention, the omnidirectional vibration measurement system for thin-walled components based on binocular vision intelligent continuous tracking is not simply a patchwork of isolated technical features. Instead, it addresses the comprehensive technical challenges of thin-walled components under complex working conditions, such as micro-amplitude vibration, rapid deformation, frequent changes in viewing angle, and inconsistent imaging domains, by constructing a multi-mechanism deeply coupled collaborative working system. The system first forms a lightweight adaptive dual-mode tracking architecture through building modules, coordinating a temporal tracking mechanism based on appearance continuity with a binocular constraint mechanism based on geometric consistency within a unified network structure. This ensures that the target maintains stable correlation even when it experiences local occlusion, shape changes, or illumination disturbances, avoiding the drift and loss problems common in traditional single-mode tracking during vibration measurement at the system level. Furthermore, the introduced modules are not merely for improving pixel positioning accuracy; rather, through the joint design of sub-pixel-level regression and frequency domain loss functions, spatial displacement estimation and temporal vibration characteristics converge collaboratively under the same optimization objective, thereby achieving stable analysis of the true dynamic response of thin-walled components in scenarios with small amplitudes and high-frequency vibrations. Furthermore, the implementation module, through unsupervised domain adaptation and stylized data augmentation mechanisms, enables the tracking network to develop a unified representation capability for different imaging devices, shooting distances, and environmental backgrounds without relying on manual annotation. This mechanism does not act independently but provides continuous and effective data distribution support for the aforementioned dual-mode tracking architecture and frequency domain constraints, preventing the model from failing due to domain shifts in actual measurement scenarios. Simultaneously, the embedding module directly embeds the differentiable physics engine and the thin-walled component vibration model into the learning process, subjecting the visual observation results to physical consistency constraints during backpropagation. This suppresses spurious vibration solutions that do not conform to structural dynamics at the network level, achieving a mechanism leap from "image tracking" to "physically interpretable vibration measurement." These modules form a closed-loop synergy within the same computational framework: visual tracking results drive physical model correction, physical constraints guide feature learning in reverse, and the domain adaptation mechanism ensures long-term stable operation. Ultimately, this enables the system to achieve high-precision, continuous measurement of the omnidirectional vibration state of thin-walled components without the need for external sensors. Compared to existing technologies that handle visual tracking, vibration analysis, or physical modeling in a decentralized manner, this invention achieves simultaneous improvements in vibration measurement accuracy, robustness, and engineering adaptability through a synergistic mechanism at the overall architecture level, demonstrating significant technological advancement.

[0097] Specific implementation of the invention: Using a self-made T-shaped thin-walled component as the experimental object, a vibration measurement experimental platform for the thin-walled component was built, such as... Figure 3As shown, seven feature points were selected from the T-shaped thin-walled component for tracking vibration measurement. Long-term tracking vibration measurements were performed using the traditional optical flow tracking method, the PIPs method, and the improved PIPs method of this invention. Experimental results show that the feature point tracking success rates of the traditional optical flow tracking method, the PIPs method, and the improved PIPs method of this invention are 57.14%, 71.42%, and 100%, respectively. Comparative analysis with the displacement measurement results of a laser displacement sensor is performed, and the maximum displacement measurement error values ​​of the traditional optical flow tracking method, the PIPs method, and the improved PIPs method of this invention are 56... ,twenty two and 8 When adjusting the illumination of the thin-walled component from weak to strong, the traditional optical flow tracking method experienced feature point loss and confusion due to the reflective properties of the weak metal texture. Of the seven feature points, two were lost and two were confused. The PIPs method and the improved PIPs method of this invention did not exhibit this phenomenon. Furthermore, when adjusting from strong to weak, the traditional optical flow tracking method could not re-track the feature points, while the PIPs method could re-track, but two feature points were confused. The improved PIPs method of this invention could re-track correctly. Verification using calibrated vibration video data of the thin-walled component showed that the improved PIPs method of this invention had no accumulated error.

[0098] This invention relates to specific application areas or related products. Thin-walled components, with their numerous advantages such as light weight, compact structure, and high load-bearing capacity, have been widely used in engineering fields. Specific application areas of this invention mainly include typical thin-walled components such as aero-engine blades, automotive body-in-white hoods, and satellite solar panels.

[0099] like Figure 3 As shown in the figure, this embodiment of the invention constructs a vibration measurement experimental platform for thin-walled components based on binocular vision. The platform includes an exciter, a cantilever thin-walled beam, a stable light source, and a binocular camera system. The exciter applies controllable periodic excitation to the thin-walled component, causing it to generate multi-directional coupled vibration. The binocular camera is fixed on a stable support and synchronously acquires dynamic image sequences of feature points on the surface of the thin-walled component from different viewpoints, providing raw data for subsequent three-dimensional displacement reconstruction and vibration analysis. By establishing a unified mapping relationship between the world coordinate system and the camera coordinate system during the system initialization phase, a unified description of the displacement of feature points in the X, Y, and Z directions in space is achieved, thus laying the foundation for omnidirectional vibration measurement.

[0100] Under these experimental conditions, the measurement method based on binocular vision intelligent continuous tracking proposed in this invention was used to continuously track and calculate the displacement of feature point #1 selected on the surface of the thin-walled part. The vibration displacement time history results are as follows: Figure 4 As shown. Among them, Figure 4 (a) Figure 4(b) and Figure 4 (c) The displacement of the feature point in the X, Y, and Z axes over time is shown respectively. It can be seen that the displacement curves in the three directions can continuously and stably reflect the vibration response of the thin-walled component under excitation, without obvious frame dropping, drifting, or abrupt changes. This indicates that the dual-mode tracking architecture and sub-pixel regression mechanism adopted in this invention have good temporal consistency and spatial resolution capabilities in complex vibration scenarios.

[0101] Furthermore, on Figure 4 Frequency domain analysis was performed on the displacement time history signal obtained from the data, and the following results were obtained: Figure 5 The displacement spectrum of feature point 1# along the Y-axis is shown. The peak value of the main vibration frequency can be clearly identified from the spectrum results, and the spectral energy is concentrated with few stray components, indicating that the displacement signal obtained by the method of this invention has a high signal-to-noise ratio and frequency resolution. This result is due to the synergistic effect of the frequency domain loss function and differentiable physical constraints introduced into the system, enabling the network to effectively suppress high-frequency noise that does not conform to the actual structural vibration characteristics while learning the displacement change law.

[0102] To further verify the accuracy of the measurement results, the vibration displacement obtained by the method of this invention was compared with the theoretical calculation results, and the results are as follows: Figure 6 As shown. Figure 6 (a) Figure 6 (b) and Figure 6 (c) Displacement comparison curves corresponding to the X, Y, and Z directions, respectively. The comparison results show that the displacement curves obtained by the method of this invention are highly consistent with the theoretical vibration displacement in terms of amplitude, phase, and periodicity, accurately reflecting the true dynamic behavior of the thin-walled component. Therefore, this invention does not rely on a single visual measurement or a single algorithm optimization, but rather achieves high-precision and stable measurement of the omnidirectional vibration state of thin-walled components through the synergistic effect of multiple mechanisms such as binocular geometric constraints, time-series tracking, frequency domain analysis, and physical model embedding, demonstrating significant technological advancements compared to existing technologies.

[0103] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for measuring omnidirectional vibration of thin-walled components based on binocular vision intelligent continuous tracking, characterized in that, Includes the following steps: Acquire a continuous image sequence of thin-walled components under a binocular vision system; Within a unified temporal framework, multiple feature points on the surface of thin-walled parts are continuously tracked, and a dual-mode tracking mechanism that includes both fast prediction and fine correction is constructed. During the continuous tracking process, the spatial displacement of the feature points is estimated at the sub-pixel level, and frequency domain consistency constraints are applied to the corresponding time-series displacements. Domain-adaptive training of the tracking model is performed without manual annotation, enabling the model to maintain consistent geometric perception capabilities across different imaging environments. Physical consistency constraints are embedded in the process of solving the displacement of feature points so that the obtained three-dimensional displacement trajectory satisfies the dynamic characteristics of the vibration of thin-walled parts. Based on the displacement of the feature points that satisfy the above constraints, the vibration displacement information of the thin-walled component in three spatial directions is output.

2. The method as described in claim 1, characterized in that, The dual-mode tracking mechanism includes a low-computation continuous displacement prediction mechanism and a highly robust short-time trajectory correction mechanism. The two mechanisms are adaptively switched based on the tracking confidence level, so that the tracking process can maintain long-term stability while ensuring real-time performance.

3. The method as described in claim 1, characterized in that, The frequency domain consistency constraint and the physical consistency constraint work together on the feature point displacement time series during the same training process, so that the displacement estimation simultaneously satisfies the spectral concentration and dynamic rationality.

4. A method for solving physically constrained visual displacement for vibration measurement of thin-walled components, characterized in that, Includes the following steps: Obtain the initial displacement sequence of feature points obtained based on visual tracking; Based on the damped vibration characteristics of thin-walled components, a differentiable dynamic constraint model is constructed. The initial displacement sequence is subjected to dynamic consistency fitting to obtain a corrected displacement sequence that satisfies physical constraints; Based on the corrected displacement sequence, the vibration displacement results of the thin-walled component in each direction in space are output.

5. The method as described in claim 4, characterized in that, The dynamic constraint model participates in optimization as a regularization constraint during the model training phase, and performs smoothing and interpolation processing on the displacement sequence during the inference phase.

6. The method as described in claim 4, characterized in that, When feature points are occluded or interfered with by noise in the image sequence, the dynamic constraint model extrapolates and compensates for the missing displacement based on the estimated vibration parameters.

7. A binocular vision-based intelligent continuous tracking omnidirectional vibration measurement system for thin-walled components, characterized in that, include: The image acquisition module is used to synchronously acquire binocular image sequences of thin-walled components; The continuous tracking module is used for dual-mode collaborative tracking of surface feature points of thin-walled parts in the time dimension; The sub-pixel displacement estimation module is used to generate high-precision displacement information of feature points in various directions in space; The domain adaptation module is used to maintain consistent representation of tracking features under different imaging environments; The physical consistency constraint module is used to apply vibration dynamic constraints to the displacement of feature points; The vibration output module is used to output the omnidirectional vibration displacement results of thin-walled components.

8. The system as described in claim 7, characterized in that, The continuous tracking module, subpixel displacement estimation module, and physical consistency constraint module work together in a unified calculation process to form a closed-loop optimization structure.

9. The system as described in claim 7, characterized in that, The domain adaptive module uses adversarial learning to make the tracking features insensitive to changes in illumination and surface texture.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 6.

Images