Rotating body vibration error compensation method and system under high-speed camera shaking condition
By constructing a camera jitter error compensation network model based on deep learning, combining multi-objective automatic labeling and feature extraction methods, the vibration error compensation problem under slight jitter of high-speed cameras is solved, and high-precision measurement of rotating body vibration is achieved.
Patent Information
- Application Number
- CN202510591160.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2025-08-12
AI Technical Summary
The prior art cannot effectively compensate vibration errors when facing slight jitter of high-speed cameras, especially in complex and variable motions, and traditional methods cannot meet the needs of small-range vibration measurements.
A camera jitter error compensation network model based on deep learning is constructed. Through the multi-object automatic labeling and feature extraction method, combined with the backbone network and the auxiliary reversible branch network, the RepCSP-CIB module and the AKConv module are used to realize the precise detection and displacement calibration of the rotating body target and the eddy current sensor probe target.
It effectively solves the measurement error problem caused by the slight jitter of the camera, improves the measurement accuracy and stability of the vibration of the rotating body, and significantly enhances the extraction ability and detection accuracy of small target features.
Smart Images
Figure CN120471950A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method and system for compensating for vibration errors of a rotating body under the condition of high-speed camera shaking, and belongs to the field of visual vibration displacement measurement. Background Art
[0002] Vibration measurement plays a crucial role in structural health monitoring. For example, wear, looseness, and corrosion of rotating components can cause abnormal vibrations during operation. Continuous vibration monitoring can detect these abnormal vibration signals before a fault occurs, identifying potential problems in advance and giving maintenance personnel ample time to take action before the fault develops further, preventing sudden equipment damage and even serious accidents.
[0003] Improving camera stability is one of the keys to reducing camera shake. Before shooting, you can use stabilization equipment and set shooting parameters. Common stabilization devices include tripods and gimbals to maintain camera stability. However, these methods cannot completely prevent external interference and can only reduce camera shake. Alternatively, other anti-shake methods can be introduced during shooting, such as optical image stabilization (OIS). This method uses a gyroscope within the lens to detect tiny camera movement, transmit a signal, and then calculate the displacement to be compensated. Based on this calculation, a floating lens in the camera lens is moved to offset the tiny movement, effectively combating image blur caused by camera shake. These methods all study camera shake during shooting. They can also be used to post-process videos containing shake to eliminate it. However, these methods have certain limitations. In real-world camera shake, motion is often complex and variable, potentially involving rotations in multiple directions, translations, and irregular shaking. For relative camera pose estimation in multi-view environments using a deep learning network architecture, the proposed network, a variant of AlexNet, acts as a regressor to predict relative translation and rotation, achieving good relative camera pose estimation results. This method is effective for large-scale camera displacements, but it cannot meet the requirements for measuring small-scale camera vibrations. Summary of the Invention
[0004] The present invention provides a method for compensating rotating body vibration errors under high-speed camera shake conditions, and constructs a camera shake error compensation network model based on deep learning to realize rotating body vibration error calibration.
[0005] The technical solution of the present invention is:
[0006] According to a first aspect of the present invention, a method for compensating for vibration errors of a rotating body under high-speed camera shake is provided. Step 1: capturing an image of a rotating body by a high-speed camera; the image of the rotating body includes at least a rotating body target and a first eddy current sensor probe target;
[0007] Step 2: construct an unlabeled training dataset and an unlabeled verification dataset based on the rotating body image; automatically label the rotating body target and the first eddy current sensor probe target in the rotating body image in the unlabeled training dataset using a multi-target automatic labeling method to obtain a labeled training dataset;
[0008] Step 3: Build a camera shake error compensation network model based on deep learning;
[0009] Step 4. Train the camera shake compensation network model based on deep learning according to the labeled training data set to obtain the optimal weight; based on the optimal weight, detect each frame of the rotating body image in the unlabeled verification data set to obtain the Y-axis coordinate of the upper left corner pixel of the detection frame of the rotating body target and the first eddy current sensor probe target in each frame of the rotating body image; convert the Y-axis coordinate of the upper left corner pixel of the detection frame of the two targets in each frame of the rotating body image into a Y-direction displacement value in the world coordinate system according to the camera calibration result; draw a first vibration curve with the Y-direction displacement value of the upper left corner pixel of the rotating body target in each frame of the rotating body image in the world coordinate system, and draw a second vibration curve with the Y-direction displacement value of the upper left corner pixel of the first eddy current sensor probe target in each frame of the rotating body image in the world coordinate system;
[0010] Step 5: Subtract the first vibration curve from the second vibration curve to obtain the true value of the vibration displacement of the rotating body in the Y direction after the error is eliminated.
[0011] Furthermore, the high-speed camera is installed on a lifting platform, and a first eddy current sensor is installed directly above the rotating body, and the installation height of the first eddy current sensor is within the effective range of the first eddy current sensor; a second eddy current sensor is installed directly above the high-speed camera, and within the effective range of the second eddy current sensor, the high-speed camera is driven by the lifting platform to perform irregular up and down movement to realize the up and down jitter simulation of the high-speed camera; the rotating body image and the rotating body reference vibration signal are synchronously collected by the high-speed camera and the second eddy current sensor is used to collect the camera shake signal.
[0012] Furthermore, by comparing the second vibration curve with the camera shake signal, it is determined that the second vibration curve obtained in step 4 can represent the camera shake signal collected by the second eddy current sensor.
[0013] Furthermore, the multi-target automatic labeling method is specifically as follows: a binary edge contour map is obtained from the original rotating body image to be labeled based on an edge detection algorithm; based on the binary edge contour map, the straight lines of the two targets are detected in the region of interest using the Hough line detection method; circumscribed rectangles are constructed for the four straight lines of the rotating body target and the four straight lines of the first eddy current sensor probe target respectively; after recording the coordinates of the upper left and lower right pixel points of the circumscribed rectangle, the circumscribed rectangle is mapped back to the original rotating body image coordinate system to obtain the labeled rotating body image.
[0014] Furthermore, the camera shake error compensation network model based on deep learning is based on YOLOv9 as the framework, which is composed of a backbone network and an auxiliary reversible branch network in parallel. The AKConv module is further introduced at the input end of the third decoupling head of the auxiliary reversible branch network of YOLOv9, and the RepCSP-CIB module is designed to replace all RepNCSPELAN4 modules in the YOLOv9 framework; the backbone network receives the original image input, and performs multi-scale feature extraction through the embedded RepCSP-CIB module, and the extracted features output preliminary detection results through the prediction head; the auxiliary reversible branch network and the backbone network process the input data in parallel: first, the global stable features of the first eddy current sensor probe target are extracted through the RepCSP-CIB module, and before the features are passed to the decoupling head, the multi-dimensional features of the rotating body and the first eddy current sensor probe target are further integrated through the introduced AKConv module to generate a feature map containing multiple feature information.
[0015] Furthermore, the RepCSP-CIB module uses the RepNCSPELAN4 module as its framework and replaces the RepNCSP module in the RepNCSPELAN4 module with the DCIB module. The DCIB module uses the RepNCSP module as its framework and replaces the n repeated bottleneck modules in the first branch of the original RepNCSP module with the CIB module. At the same time, n CIB modules are introduced after the convolution module of the second branch of the RepNCSP module.
[0016] According to a second aspect of the present invention, a system for compensating for rotational body vibration errors in the case of high-speed camera shake is provided, comprising a module of any one of the above-mentioned methods for compensating for rotational body vibration errors in the case of high-speed camera shake.
[0017] According to a third aspect of the present invention, a processor is provided, configured to execute operations including executing any one of the above-mentioned methods for compensating for rotating body vibration errors in the case of high-speed camera shake.
[0018] The beneficial effects of the present invention are as follows: the present invention uses a high-speed industrial camera as an image acquisition medium, and uses rotating bodies such as rotors in high-speed videos as the objects of vibration displacement measurement, introduces a feature extraction method based on deep learning into the field of visual vibration measurement of rotating bodies, and obtains the camera shake posture of adjacent frames by identifying the position of static objects in the target background and regressing the real movement of the target, effectively solving the measurement error problem caused by slight camera shake. Specifically, the camera shake compensation network framework of the present invention consists of a backbone network and an auxiliary reversible branch network. The backbone network and the auxiliary reversible branch network both use the RepCSP-CIB module, which replaces the original RepNCSPELAN4 module and significantly enhances the target feature extraction capability. Through the reliable gradient generated by the auxiliary reversible branch network, the deep feature layer can effectively retain the key information of small targets in the image; the DCIB module is constructed in the RepCSP-CIB module, and the CIB module uses an inverted residual structure. This structure performs independent convolution operations on each channel through deep convolution, spatially fuses the rotating body rotor and background features, and uses point convolution to realize information exchange between channels, thereby improving the detection effect of static background reference objects; in order to capture the features of rotating body targets and background reference objects at different levels, an AKConv module is added before the network decoupling head, which integrates the multi-dimensional features of the target and generates a feature map containing rich information, thereby completing the multi-target detection task. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 It is a structural block diagram of the present invention;
[0020] Figure 2 Schematic diagram of the installation of the first eddy current sensor and the second eddy current sensor;
[0021] Figure 3 Flowchart of the multi-object automatic labeling method;
[0022] Figure 4 A camera shake error compensation network model framework based on deep learning;
[0023] Figure 5 This is the RepCSP-CIB module structure diagram;
[0024] Figure 6 This is the structure diagram of the AKConv module;
[0025] Figure 7 The predicted second vibration curve results for different algorithms for camera vertical shake regression;
[0026] Figure 8 The time domain diagrams are respectively comparing the prediction results of the true value of the Y-direction vibration displacement of the rotating body using different algorithms with the vibration signal collected by the first eddy current;
[0027] Figure 9 The time domain diagrams are respectively comparing the rotor vibration results regressed by different algorithms with the first eddy current vibration signal;
[0028] Figure 10 The frequency domain diagrams are compared with the rotor vibration results regressed by different algorithms and the first eddy current vibration signal. DETAILED DESCRIPTION
[0029] To make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other in any way.
[0030] Example 1: Figures 1-10 According to a first aspect of an embodiment of the present invention, a method for compensating for vibration errors of a rotating body in the case of high-speed camera shake is provided, comprising the following steps:
[0031] Step 1: synchronously collecting the image of the rotating body and the reference vibration signal of the rotating body by using a high-speed camera and a first eddy current sensor, and simultaneously collecting the camera shake signal by using a second eddy current sensor;
[0032] A high-speed camera is installed on a lifting platform, and a first eddy current sensor is installed directly above the rotating body, and the installation height of the first eddy current sensor is within the effective range of the first eddy current sensor; a second eddy current sensor is installed directly above the high-speed camera, and within the effective range of the second eddy current sensor, the high-speed camera is driven by the lifting platform to perform irregular up and down movements to simulate the up and down shaking of the high-speed camera; the image of the rotating body and the reference vibration signal of the rotating body are synchronously collected by the high-speed camera and the first eddy current sensor, and the camera shaking signal is collected by the second eddy current sensor. In order to demonstrate the installation effect of the present invention, the installation schematic diagrams of the first eddy current sensor and the second eddy current sensor were photographed respectively by a mobile phone, as shown in the following figure. Figure 2 shown.
[0033] Step 2: construct an unlabeled training dataset and an unlabeled verification dataset based on the rotating body image; automatically label the rotating body target and the first eddy current sensor probe target (reference target) in the rotating body image in the unlabeled training dataset using a multi-target automatic labeling method to obtain a labeled training dataset;
[0034] Furthermore, the multi-target automatic labeling method is specifically as follows: a binary edge contour map is obtained from the original rotating body image to be labeled based on an edge detection algorithm; based on the binary edge contour map, the straight lines of the two targets are detected by the Hough line detection method for the region of interest; a circumscribed rectangle is constructed for the four straight lines of the rotating body target and the four straight lines of the first eddy current sensor probe target (that is, a rectangle can be obtained by extending the four straight lines and taking their intersection); the coordinates of the upper left and lower right pixel points of the circumscribed rectangle are recorded, and then the circumscribed rectangle is mapped back to the original rotating body image coordinate system to obtain the labeled rotating body image.
[0035] Exemplarily, the edge detection algorithm adopts the Sobel edge detection algorithm.
[0036] refer to Figure 3 Specifically: Figure 3 The image of the rotating body to be annotated shown in (a) is converted into a grayscale image containing only brightness information (3(b)). In the grayscale image, each pixel has only one value to represent its grayscale level. Each pixel represents the brightness feature with a single grayscale value, where a high grayscale value corresponds to a high brightness area. Secondly, the generated grayscale image is Gaussian filtered (specifically, a weighted average calculation is performed using a 9×9 Gaussian filter kernel) to suppress noise interference and improve image quality. Subsequently, the Sobel operator is used to calculate the amplitude and direction of the gradient of each pixel. Then, a clear binary edge contour image is generated by combining non-maximum suppression and double threshold segmentation (a high threshold T h and a low threshold T l ),like Figure 3 (c) Based on the binary edge contour map, the Hough line detection method (by mapping the straight line in the Cartesian coordinate system to the Hough parameter space for identification through coordinate space transformation) is used to detect the rotating target and the first eddy current sensor probe target (for clarity, Figure 3 (d) only shows the four straight lines of the rotating object), and the circumscribed rectangles of the four straight lines of the two targets are constructed (such as Figure 3 (e) shows the bounding rectangle of the rotating object); finally, after recording the coordinates of the upper left and lower right pixel points P1(x1, y1) and P2(x2, y2) of the bounding rectangle, the bounding rectangle is mapped back to the original rotating object image coordinate system 3(a), thereby obtaining the annotated rotating object image as shown in Figure 3 (f) By extending the above method to the first eddy current sensor probe, automatic labeling can be completed in batches.
[0037] It should be noted that during the neural network training process, the target position needs to be finely labeled in advance, and the quality of the labeling directly affects the final detection effect. However, in the process of manual labeling, it is very easy to produce errors by relying on visual judgment and manual operation. Especially when performing visual measurement tasks, the displacement of the target in the image is often only a few pixels. Even a small manual labeling error will cause a huge deviation in the subsequent measurement results. In addition, for the labeling of large amounts of data, the manual labeling operation requires long periods of concentration, which inevitably leads to the occurrence of labeling errors. The present invention can effectively automatically label the two types of specific targets of the present invention by cleverly utilizing the edge detection algorithm and combining it with the Hough line detection method.
[0038] Step 3: Build a camera shake error compensation network model based on deep learning;
[0039] like Figure 4 As shown, the camera shake error compensation network model based on deep learning takes YOLOv9 as the framework, which is composed of a backbone network and a reversible branch network in parallel. The AKConv module is further introduced at the input end of the third decoupling head of the auxiliary reversible branch network of YOLOv9, and the RepCSP-CIB module is designed to replace all RepNCSPELAN4 modules in the YOLOv9 framework; the backbone network receives the original image input, and performs multi-scale feature extraction through the embedded RepCSP-CIB module, and the extracted features output preliminary detection results through the prediction head module; the reversible branch network and the backbone network process the input data in parallel: first, the global stable features of the background static reference object are extracted through the RepCSP-CIB module, and before the features are passed to the decoupling head, the multi-dimensional features of the rotating body and the first eddy current sensor probe target are further integrated through the introduced AKConv module to generate a feature map containing multiple feature information, thereby helping the network to better complete the multi-target detection task.
[0040] Specifically, the RepCSP-CIB module is designed to replace all RepNCSPELAN4 modules in the YOLOv9 framework. Figure 5As shown in the figure, the RepCSP-CIB module uses the RepNCSPELAN4 module as its framework, replacing the RepNCSP module in the RepNCSPELAN4 module with the DCIB module. The DCIB module uses the RepNCSP module as its framework, replacing the n repeated bottleneck modules in the first branch of the original RepNCSP module with the CIB module. Furthermore, n CIB modules are introduced after the convolutional module in the second branch of the RepNCSP module. The CIB module is derived from the Compact Inversion Block (CIB) of YOLOv10. This module uses an inverted residual structure, achieves feature space mixing through deep convolution, and promotes information exchange between channels through point convolution, effectively improving the detection capability of small objects.
[0041] like Figure 6 As shown, the AKConv module is an existing well-known module. Traditional convolution kernels only operate within a window of fixed size and cannot capture information from other windows. To this end, the present invention adopts an AKConv module that is different from the traditional convolution method. The AKConv module adopts an AK convolution kernel with an initial size of N=5, and obtains the offset of the convolution kernel through a convolution operation, whose dimension is (B, 2N, H, W). The offset Pn is then added to the original coordinate P0 to obtain the modified coordinate (P0+Pn); finally, the interpolation and resampling methods are used to extract the key features of the corresponding position. The introduction of this module not only effectively reduces irrelevant calculations, but also enables the network to focus more on the rotating body targets and reference objects that need to be identified, thereby better capturing the contextual information of the rotating body and the reference object, and improving the accuracy of the network.
[0042] By applying the above technical solution, it can be seen that in the present invention, through the collaborative training of the backbone network and the auxiliary reversible branch network, the auxiliary reversible branch is used to generate a stable gradient, thereby alleviating the problem of feature loss of small targets in the deep network; further, the RepCSP-CIB module designed in the model can enhance the feature representation of the stationary reference object (the first eddy current sensor probe), improve the detection robustness under complex backgrounds, reduce the loss of key features of small targets in deep features, and add an AKConv module before the decoupling head. The feature offset is dynamically learned through the deformable convolution kernel of N=5, the multi-level features of the rotating target and the background are integrated, and a composite feature map is generated based on coordinate offset adjustment and resampling, thereby achieving support for multi-target detection tasks with small targets, so that the model of the present invention can not only effectively detect rotating targets, but also has a high accuracy for small targets such as the first eddy current sensor probe.
[0043] Step 4. Train the camera shake compensation network model based on deep learning according to the labeled training data set to obtain the optimal weight; based on the optimal weight, detect each frame of the rotating body image in the unlabeled verification data set to obtain the Y-axis coordinate of the upper left corner pixel of the detection frame of the rotating body target and the first eddy current sensor probe target in each frame of the rotating body image; convert the Y-axis coordinate of the upper left corner pixel of the detection frame of the two targets in each frame of the rotating body image into a Y-direction displacement value in the world coordinate system according to the camera calibration result; draw a first vibration curve with the Y-direction displacement value of the upper left corner pixel of the rotating body target in each frame of the rotating body image in the world coordinate system, and draw a second vibration curve with the Y-direction displacement value of the upper left corner pixel of the first eddy current sensor probe target in each frame of the rotating body image in the world coordinate system;
[0044] Furthermore, the step 4 includes:
[0045] Step 4.1. Before formal training, configure the training hyperparameters in the train.py file of the camera shake error compensation network model. The hyperparameters in the configuration file mainly include the training batch and the number of iterations. The rest of the hyperparameters are the default values.
[0046] Step 4.2. Train the camera shake compensation network model based on deep learning according to the labeled training data set to obtain the optimal weight; load the optimal weight into detect.py, detect each frame of the rotating body image in the unlabeled verification data set, and obtain the Y-axis coordinate of the upper left corner pixel of the detection frame of the rotating body target and the first eddy current sensor probe target in each frame of the rotating body image; convert the Y-axis coordinate of the upper left corner pixel of the detection frame of the two targets in each frame of the rotating body image into the Y-direction displacement value in the world coordinate system based on the camera calibration result; draw a first vibration curve with the Y-direction displacement value of the upper left corner pixel of the rotating body target in each frame of the rotating body image in the world coordinate system, and draw a second vibration curve with the Y-direction displacement value of the upper left corner pixel of the first eddy current sensor probe target in each frame of the rotating body image in the world coordinate system; illustratively, write the Y-axis coordinates of the two targets in each frame of the rotating body image into two text documents respectively, and use matlab to draw vibration curves with the two document data.
[0047] The camera calibration is specifically as follows:
[0048] 1) Using a 2mm checkerboard ceramic calibration plate, a total of 15 calibration plate images were collected at different positions and angles. The MATLAB Camera Calibrator tool was used for camera calibration. The correspondence between pixel coordinates and world coordinates can be calculated from the formula, that is, the conversion relationship between pixel coordinates p = [u, v]^T in the camera imaging model and world coordinates P = [X, Y, Z]^T. Coordinate mapping is achieved through the intrinsic parameter matrix K, rotation matrix R, and translation vector T (Formula 1):
[0049] p=K[RP+T] (1)
[0050] At the same time, the calculation method of the physical size of the unit pixel in the X direction is defined (Formula 2):
[0051]
[0052] Among them, d x The unit pixel represents the true value in the world coordinate system, d unit is the actual unit size of the chessboard (set to 2mm), N X is the number of pixels in the X direction of the checkerboard in the calibration plate image. After calculation, the present invention d x =0.892857mm.
[0053] 2) d x Multiply it by the Y-axis coordinate of the upper-left pixel to get the Y-direction displacement value of the upper-left pixel in the world coordinate system.
[0054] Furthermore, the present invention compares the second vibration curve with the camera shake signal to determine that the second vibration curve can represent the camera shake signal. Specifically, after comparing the second vibration curve with the camera shake signal, the absolute value of the difference between the two is within 0.1 mm, indicating that the second vibration curve can represent the camera shake signal collected by the second eddy current sensor, such as Figure 7 As shown, the second vibration curve obtained by the model of the present invention is highly consistent with the camera shake signal (standard) collected by the second eddy current sensor.
[0055] Step 5: Subtract the first vibration curve from the second vibration curve to obtain the true value of the vibration displacement of the rotating body in the Y direction after the error is eliminated.
[0056] Furthermore, the obtained true value of the vibration displacement of the rotating body in the Y direction is quantitatively compared with the reference vibration signal of the rotating body collected by the first eddy current sensor.
[0057] According to the second aspect of an embodiment of the present invention, a system for compensating for vibration errors of rotating bodies in the case of high-speed camera shake is provided, comprising a module of any one of the above-mentioned methods for compensating for vibration errors of rotating bodies in the case of high-speed camera shake. Specifically comprising: a first module for executing step 1: acquiring a rotating body image by a high-speed camera; the rotating body image at least includes a rotating body target and a first eddy current sensor probe target; a second module for executing step 2: constructing an unlabeled training data set and an unlabeled verification data set based on the rotating body image; automatically labeling the rotating body target and the first eddy current sensor probe target in the rotating body image in the unlabeled training data set using a multi-target automatic labeling method to obtain a labeled training data set; a third module for executing step 3: constructing a camera shake error compensation network model based on deep learning; a fourth module for executing step 4: training the camera shake compensation network model based on deep learning based on the labeled training data set to obtain the optimal weight; and training the unlabeled data set based on the optimal weight. Note: Each frame of the rotating body image in the verification data set is detected to obtain the Y-axis coordinate of the upper left corner pixel point of the detection frame of the rotating body target and the first eddy current sensor probe target in each frame of the rotating body image; the Y-axis coordinate of the upper left corner pixel point of the detection frame of the two targets in each frame of the rotating body image is converted into the Y-direction displacement value in the world coordinate system based on the camera calibration result; the Y-direction displacement value of the upper left corner pixel point of the rotating body target in each frame of the rotating body image in the world coordinate system is used to draw a first vibration curve, and the Y-direction displacement value of the upper left corner pixel point of the first eddy current sensor probe target in each frame of the rotating body image in the world coordinate system is used to draw a second vibration curve; the fifth module is used to execute step 5: subtract the first vibration curve from the second vibration curve to obtain the true value of the Y-direction vibration displacement of the rotating body after the error is eliminated.
[0058] According to a third aspect of an embodiment of the present invention, a processor is provided, configured to execute an operation, wherein the operation includes executing any one of the aforementioned methods for compensating for rotational body vibration errors in the presence of high-speed camera shake. When executing the operation, the processor implements steps 1 to 5 above.
[0059] Example 2: Figure 1 As shown in FIG, a method for compensating the vibration error of a rotating body under high-speed camera shaking is implemented as follows:
[0060] The image data of the rotating body and the baseline vibration signal collected by the first eddy-current sensor were collected on a high-speed rotor vibration test bench (Nanjing Dongda Z-03). The rotor speed was controlled by adjusting different voltage values, and the output voltage was kept constant after being adjusted to 20V. A high-speed industrial camera (Qianyanlang 5F01) was fixed to a lifting platform. The high-speed camera shake signal was acquired using a second eddy-current sensor (probe OD900803-03-04-20-00, preamplifier OD900800-20-05) and a signal collector (sampling frequency set to 1000Hz). The high-speed camera was set to a high frame rate (sampling image resolution set to 640×640, sampling frame rate set to 1000 frames / second) for synchronous image acquisition, and a light source (Jinbei EF-200LED) was added for illumination compensation.
[0061] 2. Select 300 images from the captured video and automatically label the rotor and the first eddy current sensor probe according to the multi-target automatic labeling method to obtain 300 labeled training data sets; use 7400 consecutive images as unlabeled verification data sets.
[0062] 3. Construct a camera shake error compensation network model based on deep learning; train the camera shake compensation network model based on deep learning according to the labeled training data set to obtain the optimal weight; based on the optimal weight, detect each frame of the rotating body image in the unlabeled verification data set to obtain the Y-axis coordinate of the upper left corner pixel point of the detection frame of the rotating body target and the first eddy current sensor probe target in each frame of the rotating body image; convert the Y-axis coordinate of the upper left corner pixel point of the detection frame of the two targets in each frame of the rotating body image into the Y-direction displacement value in the world coordinate system based on the camera calibration result; draw a first vibration curve with the Y-direction displacement value of the upper left corner pixel point of the rotating body target in each frame of the rotating body image in the world coordinate system, and draw a second vibration curve with the Y-direction displacement value of the upper left corner pixel point of the first eddy current sensor probe target in each frame of the rotating body image in the world coordinate system.
[0063] To demonstrate the effectiveness of the proposed model, the performance of the improved network was gradually verified through ablation experiments. The root mean square error (RMSE) shown in formula (3) was used as an evaluation metric to measure the similarity between the second vibration curve extracted by the algorithm and the camera shake signal (standard) collected by the second eddy current sensor. All algorithms were verified on the same validation dataset, with the batch size set to 1 and the other parameters unchanged by default. Because the experiment carried out monitoring tasks for two types of targets (Rotor represents the rotor target and Sensor represents the first eddy current sensor probe target), their RMSE results were compared simultaneously.
[0064]
[0065] Among them, Gi is the true value measured by the eddy current sensor, P i Represents the predicted value of the algorithm.
[0066] Table 1 Ablation experiment
[0067]
[0068] As can be seen from Table 1, the model of the present invention improves the original network by introducing the AKConv module and the RepCSP-CIB module, significantly improving the displacement extraction performance of rotating objects and stationary background reference objects. As can be seen from Table 1, AKConv effectively filters background interference with its deformable convolution kernel, reducing the average root mean square error (mRMSE) of Rotor and Sensor targets from 127.69 to 117.71; the RepCSP-CIB module strengthens feature extraction through a dual compact inversion structure, reducing the errors of the two types of targets by 40.54 and 6.18 respectively. When the two modules work together, the network further reduces the average RMSE to 68.65 while maintaining the computational efficiency FPS, which is nearly half the error of the original network, verifying the effectiveness of the improved scheme in accurately extracting target objects in complex scenes.
[0069] The proposed algorithm is compared and analyzed with other deep learning-based object detection algorithms (YOLOv9) and traditional video stabilization algorithms. Evaluation metrics in the field of visual vibration measurement (RMSE, m-RMSE, and FPS) are used as comparison indicators. The comparison results are shown in Table 2.
[0070] Table 2 Comparative experiment
[0071]
[0072] As shown in Table 2, all algorithms were tested using the same dataset. The video processed by the video stabilization algorithm has jitter interference eliminated by default, so only the video processed by the video stabilization algorithm is input into the original YOLOv9 network to extract the rotor vibration displacement, which is then compared with the camera shake signal collected by the second eddy current sensor. Figure 7The camera shake signal (standard) collected by the second eddy current sensor and the shake curve regressed by the algorithm proposed in the present invention for the vertical direction of the camera show that compared with the standard shake signal collected by the second eddy current sensor, the second vibration curve regressed by the present algorithm is the best fit to the standard signal. Table 2 also shows that for sensor-type targets, the RMSE result of the curve obtained by the present algorithm is 36.4293, which is the lowest among all algorithms. The present invention uses the rotor as the visual measurement target, simulates the shooting of camera translation shake, constructs a camera shake compensation network and a multi-target automatic annotation dataset, introduces a specific structure to improve the network feature extraction capability, corrects the distortion of the high-speed camera, and calculates the true vibration value of the rotor, effectively solving the measurement error problem caused by slight camera shake.
[0073] 5. Subtract the first vibration curve from the second vibration curve to obtain the true value of the Y-direction vibration displacement of the rotating body after error elimination; quantitatively compare the obtained true value of the Y-direction vibration displacement of the rotating body with the rotating body reference vibration signal collected by the first eddy current sensor.
[0074] After obtaining the first and second vibration curves regressed by each algorithm, de-jittering is performed to obtain the true value of the Y-direction vibration displacement of the rotating body, that is, the vibration signal of the rotor itself obtained by the algorithm. The present invention visualizes the true value of the Y-direction vibration displacement of the rotating body extracted by each algorithm, and the results are as follows: Figure 8 Specifically: Figure 8 Shown are time domain diagrams comparing the prediction results of different algorithms with the rotating body reference vibration signal (standard) collected by the first eddy current sensor. In the figure, (a), (b), (c), and (d) are respectively compared with the rotating body reference vibration signal (standard) collected by the first eddy current sensor and the traditional video stabilization algorithm, YOLOv9, YOLOv 10, and the algorithm of the present invention. The blue curve represents the rotating body reference vibration signal (standard) collected by the first eddy current sensor, and the other curves represent the true value of the Y-direction vibration displacement of the rotating body including camera shake predicted by each algorithm. After comparison, it can be found that the signal obtained by the traditional video stabilization algorithm has a significant upward offset compared to the standard signal, while the mixed signal curve obtained by the YOLOv 9 algorithm has a large deviation. These problems affect the accuracy of the true result of the rotor after eliminating the shake. Compared with other algorithms, the algorithm proposed in the present invention shows a smoother curve in the visualization results and has no obvious error.
[0075] In order to compare the results of different algorithms more intuitively, the present invention intercepts 1500 consecutive images in the whole video and displays the corresponding curve results for analysis and comparison. The specific results are as follows: Figure 9 As shown; After performing fast Fourier transform (FFT) on the time domain signals obtained by the above algorithms, the following is obtained: Figure 10 The frequency domain diagram is shown. Figure 9 、 10 The images in the figure are from top to bottom: the comparison of this algorithm, traditional video stabilization algorithm, YOLOv 9 and YOLOv 10 with eddy current sensor signals. Figure 9 From the time domain diagram, it can be observed that the time domain curve regressed by the traditional video stabilization algorithm has obvious noise, and the subsequent prediction curve period gradually becomes disordered; the time domain diagram of the rotor vibration signal obtained by the YOLOv9 algorithm has an obvious upward offset compared with the standard time domain diagram. In addition to the algorithm of the present invention, the best algorithm is YOLOv10. Although its curve is smoother, the regressed curve has an error of about 80μm in amplitude compared with the standard signal. In contrast, the algorithm of the present invention performs best in the task of regressing the rotating body reference vibration signal (standard) collected by the first eddy current sensor. The error between its regression result and the standard signal is only about 16μm, which basically meets the accuracy requirements of the vibration measurement task. Taking into account factors such as amplitude, phase and period, although the curve of the algorithm of the present invention is slightly insufficient in smoothness, its fit with the rotating body reference vibration signal collected by the first eddy current sensor is the best among all algorithms. The accuracy of other camera pose regression algorithms is usually at the millimeter level, while the algorithm proposed in the present invention reaches the highest level.
[0076] from Figure 10 The frequency domain plot shows that the YOLOv9 algorithm performs the worst, primarily due to periodic errors in its time-domain regression curve, which results in a difference in the inherent frequency domain between the frequency domain plot and the eddy current standard curve. There is a significant difference in amplitude between the YOLOv9 and YOLOv10 algorithms. Furthermore, after calibration, the rotor vibration signal regressed by the YOLOv9 algorithm exhibits a significant deviation between peak and valley values, affecting the accuracy of the frequency domain plot. Aside from the algorithm of the present invention, the algorithm with the best performance in both the time and frequency domain plots is YOLOv10. However, the algorithm of the present invention performs even better in terms of eddy current peak comparison.
[0077] The specific embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.
Claims
1. A method for compensating for vibration errors of rotating bodies in the case of high-speed camera shake, characterized in that: The method comprises: Step 1: Capturing a rotating body image by a high-speed camera; the rotating body image at least includes a rotating body target and a first eddy current sensor probe target; Step 2: construct an unlabeled training dataset and an unlabeled verification dataset based on the rotating body image; automatically label the rotating body target and the first eddy current sensor probe target in the rotating body image in the unlabeled training dataset using a multi-target automatic labeling method to obtain a labeled training dataset; Step 3: Build a camera shake error compensation network model based on deep learning; Step 4. Train the camera shake compensation network model based on deep learning according to the labeled training data set to obtain the optimal weight; based on the optimal weight, detect each frame of the rotating body image in the unlabeled verification data set to obtain the Y-axis coordinate of the upper left corner pixel of the detection frame of the rotating body target and the first eddy current sensor probe target in each frame of the rotating body image; convert the Y-axis coordinate of the upper left corner pixel of the detection frame of the two targets in each frame of the rotating body image into a Y-direction displacement value in the world coordinate system according to the camera calibration result; draw a first vibration curve with the Y-direction displacement value of the upper left corner pixel of the rotating body target in each frame of the rotating body image in the world coordinate system, and draw a second vibration curve with the Y-direction displacement value of the upper left corner pixel of the first eddy current sensor probe target in each frame of the rotating body image in the world coordinate system; Step 5: Subtract the first vibration curve from the second vibration curve to obtain the true value of the vibration displacement of the rotating body in the Y direction after the error is eliminated.
2. The method for compensating for rotating body vibration errors in the case of high-speed camera shake according to claim 1, characterized in that: The high-speed camera is installed on a lifting platform, and a first eddy current sensor is installed directly above the rotating body, and the installation height of the first eddy current sensor is within the effective range of the first eddy current sensor; a second eddy current sensor is installed directly above the high-speed camera, and within the effective range of the second eddy current sensor, the high-speed camera is driven by the lifting platform to perform irregular up and down movement to realize the up and down jitter simulation of the high-speed camera; the rotating body image and the rotating body reference vibration signal are synchronously collected by the high-speed camera and the second eddy current sensor is used to collect the camera jitter signal.
3. The method for compensating for rotating body vibration errors in the case of high-speed camera shake according to claim 2, wherein: By comparing the second vibration curve with the camera shake signal, it is determined that the second vibration curve obtained in step 4 can represent the camera shake signal collected by the second eddy current sensor.
4. The method for compensating for rotating body vibration errors in the case of high-speed camera shake according to claim 1, wherein: The multi-target automatic labeling method specifically comprises the following steps: obtaining a binary edge contour map of the original rotating body image to be labeled according to an edge detection algorithm; detecting the straight lines of the two targets in the region of interest using the Hough line detection method based on the binary edge contour map; constructing circumscribed rectangles for the four straight lines of the rotating body target and the four straight lines of the first eddy current sensor probe target respectively; and recording the coordinates of the upper left and lower right pixel points of the circumscribed rectangle and mapping the circumscribed rectangle back to the original rotating body image coordinate system to obtain the labeled rotating body image.
5. The method for compensating for rotating body vibration errors in the case of high-speed camera shake according to claim 1, wherein: The camera shake error compensation network model based on deep learning is based on YOLOv9 as the framework. This framework is composed of a backbone network and an auxiliary reversible branch network in parallel. The AKConv module is further introduced at the input end of the third decoupling head of the auxiliary reversible branch network of YOLOv9, and the RepCSP-CIB module is designed to replace all RepNCSPELAN4 modules in the YOLOv9 framework; the backbone network receives the original image input, and performs multi-scale feature extraction through the embedded RepCSP-CIB module. The extracted features output preliminary detection results through the prediction head; the auxiliary reversible branch network and the backbone network process the input data in parallel: first, the global stable features of the first eddy current sensor probe target are extracted through the RepCSP-CIB module. Before the features are passed to the decoupling head, the multi-dimensional features of the rotating body and the first eddy current sensor probe target are further integrated through the introduced AKConv module to generate a feature map containing multiple feature information.
6. The method for compensating for rotating body vibration errors in the case of high-speed camera shake according to claim 5, characterized in that: The RepCSP-CIB module uses the RepNCSPELAN4 module as its framework and replaces the RepNCSP module in the RepNCSPELAN4 module with the DCIB module. The DCIB module uses the RepNCSP module as its framework and replaces the n repeated bottleneck modules in the first branch of the original RepNCSP module with the CIB module. At the same time, n CIB modules are introduced after the convolution module in the second branch of the RepNCSP module.
7. A system for compensating for vibration errors of rotating bodies in the case of high-speed camera shake, characterized in that: A module comprising the method for compensating for rotational body vibration errors in the case of high-speed camera shake according to any one of claims 1 to 6.
8. A processor, characterized in that: The processor is configured to execute operations including executing the method for compensating for rotating body vibration errors in the case of high-speed camera shake according to any one of claims 1 to 6.