UAV-based gimbal measurement and control method and system

Abstract

This application relates to a gimbal measurement and control method and system based on an unmanned aerial vehicle (UAV). The method includes: reproducing the UAV's flight jitter attitude using a 6-axis programmable control vibration table; collecting vibration table motion parameters, gimbal output images, and gimbal contact force data through sensors to form a physical test dataset; constructing a coupled digital twin model of the vibration table and gimbal; injecting the physical test dataset into the digital twin model using the OPCUA protocol and event flow-driven mapping mechanism to achieve bidirectional synchronization between physical and virtual states, generating a virtual simulation dataset; fusing the virtual simulation dataset output by the digital twin model with the physical test dataset to form a multi-source dataset; constructing a transfer learning prediction model based on the digital twin model; and fusing image features and gimbal motor data through a CNN-LSTM network based on the multi-source dataset to output a gimbal stabilization performance score and confidence interval.

Description

Technical Field

[0001] This application relates to the field of gimbal testing technology, and in particular to a gimbal measurement and control method and system based on unmanned aerial vehicles (UAVs). Background Technology

[0002] In recent years, with the rapid development of IoT, AI and cloud computing technologies, the construction of smart campuses has gradually become more common. The mechanical gimbal of drones mainly serves to stabilize video images. When a drone is flying in the air, its attitude adjustments can cause shaking in aerial videos. The mechanical gimbal collects signals of the drone's shaking and uses motors to adjust the camera's attitude in the opposite direction, thus achieving video stabilization.

[0003] Currently, many companies that design and manufacture mechanical gimbals for drones need to mount the mechanical gimbal on the drone and conduct actual flight tests outdoors when testing the mechanical gimbal. This is because the aircraft environment is complex and variable, and only on-board testing can truly reflect the image stabilization effect of the mechanical gimbal.

[0004] Customers with high quality requirements often require each gimbal to undergo actual flight testing. Each gimbal flight test takes approximately 10-15 minutes, which is extremely inefficient, and drones cannot take off in rainy weather. Testers rely on subjective impressions to judge video image stability, making it difficult to guarantee consistent gimbal stabilization results. In summary, testing drone mechanical gimbals manually is costly and slow.

[0005] Based on this, this application provides a gimbal measurement and control method and system based on unmanned aerial vehicles (UAVs). Summary of the Invention

[0006] To address the issue of testers relying on subjective impressions to judge video image stability, the consistency of gimbal stabilization results in the final tests is difficult to guarantee. In summary, the manual testing of UAV mechanical gimbals suffers from high costs and slow testing speed. This application provides a UAV-based gimbal measurement and control method and system.

[0007] Firstly, this application provides a gimbal monitoring and control method based on an unmanned aerial vehicle (UAV), which adopts the following technical solution: including:

[0008] The 6-axis programmable vibration table is used to reproduce the flight jitter attitude of the UAV, and the vibration table motion parameters, gimbal output images and gimbal contact force data are collected by sensors to form a physical test dataset.

[0009] A digital twin model of a shaking table-gimbal coupling was constructed based on SolidWorks and ANSYS. The physical test dataset was injected into the digital twin model using the OPCUA protocol and event flow-driven mapping mechanism to achieve bidirectional synchronization between physical and virtual states and generate a virtual simulation dataset.

[0010] The virtual simulation dataset output by the digital twin model is called and fused with the physical test dataset to form a multi-source dataset;

[0011] Based on the digital twin model, a transfer learning prediction model is constructed. Based on the multi-source dataset, image features and gimbal motor data are fused through a CNN-LSTM network to output the gimbal stabilization effect score and confidence interval.

[0012] The preset optimal test parameters are invoked, and the gimbal stabilization effect score and the confidence interval are combined to dynamically adjust the vibration table parameters through a reinforcement learning algorithm.

[0013] Preferably, the method of reproducing the UAV's flight jitter attitude using a 6-axis programmable vibration table, and collecting vibration table motion parameters, gimbal output images, and gimbal contact force data through sensors to form a physical test dataset, including:

[0014] IMU data is used as vibration source data and input into a 6-axis programmable vibration table to reproduce the aircraft's flight shaking attitude and frequency. Sensors are used to collect the motion parameters of the vibration table and the contact force data of the gimbal.

[0015] Simultaneously, based on a preset black and white checkerboard pattern used for camera calibration, the gimbal captures images of the black and white checkerboard pattern to obtain the gimbal output image;

[0016] The vectors between the corner points of the chessboard are calculated by the output image of the gimbal. If the difference between the vectors between the corner points and the preset stationary vector is less than a threshold, it is determined that the gimbal stabilization effect has met the initial requirements.

[0017] If the difference between the vector between the corner points and the preset stationary vector is not less than a threshold, then the gimbal stabilization effect is deemed unsatisfactory.

[0018] Preferably, the construction of the digital twin model of the shaking table-gimbal coupling based on SolidWorks and ANSYS includes:

[0019] A three-dimensional geometric model of the vibration table-panel system was created using SolidWorks, and the panel motor was parametrically modeled, including rotor inertia, stator winding resistance, and permanent magnet flux linkage parameters, and exported in Parasolid format.

[0020] Import the 3D geometric model into ANSYS Workbench, mesh it, apply material properties, perform modal analysis using ANSYS Mechanical, extract the first 20 mode shapes, and generate a reduced-order model using the Craig-Bampton method.

[0021] The reduced-order model is used to simulate the dynamic response of the vibration table-gimbal in ANSYS Twin Builder; a two-dimensional finite element model of the gimbal motor is established using ANSYS Maxwell to calculate the electromagnetic torque under different currents; and a PID controller model is embedded to realize closed-loop control of the motor rotation angle.

[0022] The digital twin model is implemented by defining multiphysics interface equations using the Modelica language.

[0023] Preferably, the step of using the OPCUA protocol and event-driven mapping mechanism to inject the physical test dataset into the digital twin model to achieve bidirectional synchronization of physical and virtual states and generate a virtual simulation dataset includes:

[0024] A dedicated information model is built based on the OPCUA standard. Physical-virtual data mapping is achieved by defining objects, variables and event nodes. A secure communication link is established using TLS 1.3 encryption and X.509 certificate authentication to realize the real-time injection of the physical test dataset.

[0025] By using a pre-defined physical-virtual event rule base and employing a synchronization strategy of timestamp alignment and state feedback, the physical state and the virtual model are calibrated bidirectionally.

[0026] A virtual perturbation parameter set is set in the digital twin model. For each physical test sample, the digital twin model is driven to perform parameter perturbation simulation to generate a comparison dataset of physical and virtual data.

[0027] Based on the dynamic response data and performance prediction data in the comparison dataset, they are stored in a unified format as a virtual simulation dataset.

[0028] Preferably, the virtual simulation dataset output by the digital twin model is fused with the physical test dataset to form a multi-source dataset, including:

[0029] The physical test dataset and the virtual simulation dataset are cleaned, aligned, and dimensionality reduced by edge computing. Based on industrial metadata standards, the physical and virtual data are standardized, and the physical and virtual data are associated to achieve data mapping and obtain structured data with unified semantic labels.

[0030] Based on the structured data, a time series alignment at a sampling rate of 1kHz is achieved through a dynamic time warping algorithm, generating a one-to-one correspondence between physical test samples and virtual perturbation samples. Then, a dual-channel feature fusion network is used to extract joint time-frequency features, outputting fused feature data that can be directly used for model training.

[0031] The fused feature data is organized into a three-level structure of spatiotemporal index, feature type, and operating condition label, and stored as the multi-source dataset containing physical measured features, virtual simulation features, and fused derived features.

[0032] Preferably, the step of constructing a transfer learning prediction model based on the digital twin model for simulation, and based on the multi-source dataset, fusing image features and gimbal motor data through a CNN-LSTM network to output a gimbal stabilization performance score and confidence interval, includes:

[0033] Based on the multi-source dataset, a CNN-LSTM dual-branch transfer learning network model is constructed: the CNN branch receives the image features of the multi-source dataset and extracts 512-dimensional spatial features through 3 layers of convolution; the LSTM branch receives the time-series data of the gimbal motor and extracts 256-dimensional temporal features through 2 layers of bidirectional LSTM, outputting an initial model architecture that includes a feature extraction layer and a fusion interface.

[0034] The spatial features and the temporal features are weighted and fused using an attention mechanism, and the output dimension is a cross-modal joint feature vector.

[0035] Using the cross-modal joint feature vector as input and the image stabilization effect score in the physical test as the supervision label, the CNN-LSTM dual-branch transfer learning network model is trained using a transfer learning strategy. The mean square error between the predicted score and the real score is minimized through the Adam optimizer, and the converged model parameters are output.

[0036] The real-time acquired gimbal image data and motor data are input into the trained model, and the gimbal stabilization effect score and the confidence interval are output. The confidence interval is calculated by the bootstrap resampling algorithm to achieve quantitative evaluation and uncertainty quantification of the gimbal stabilization performance.

[0037] Preferably, the step of calling preset optimal test parameters, combining the gimbal stabilization effect score and the confidence interval, and dynamically adjusting the vibration table parameters through a reinforcement learning algorithm includes:

[0038] Read the preset optimal test parameters stored in the PLC controller, and combine them with the gimbal stabilization effect score to construct a reinforcement learning environment: define the vibration table amplitude, frequency and waveform type as the continuous action space, define the gimbal stabilization effect score and the width of the confidence interval as the state space, and output the environment initialization configuration file.

[0039] Based on the gimbal stabilization effect and the confidence interval, a dual-objective reward function is constructed: the basic reward is positively correlated with the gimbal stabilization effect, the penalty term is positively correlated with the width of the confidence interval, the comprehensive reward value = basic reward + penalty term, and the real-time reward value is output.

[0040] A deep deterministic policy gradient algorithm is adopted, with the state space as input and the real-time reward value as feedback signal, to iteratively optimize the vibration table parameters. The new parameters are sent to the vibration table controller through the OPCUA protocol, and the optimized parameter records with timestamps are output.

[0041] Secondly, this application discloses a gimbal monitoring and control device based on an unmanned aerial vehicle (UAV), which adopts the following technical solution, including:

[0042] The physical testing module is used to reproduce the flight jitter attitude of the UAV through a 6-axis programmable vibration table, and to collect the motion parameters of the vibration table, the output image of the gimbal, and the contact force data of the gimbal through sensors to form a physical testing dataset.

[0043] The digital twin module is used to build a vibration table-gimbal coupled digital twin model based on SolidWorks and ANSYS. It adopts the OPCUA protocol and event flow driven mapping mechanism to inject the physical test dataset into the digital twin model, realize bidirectional synchronization of physical and virtual states, and generate a virtual simulation dataset.

[0044] The multi-source data module is used to call the virtual simulation dataset output by the digital twin model and merge it with the physical test dataset to form a multi-source dataset;

[0045] The gimbal testing module is used to build a transfer learning prediction model based on the digital twin model. Based on the multi-source dataset, it fuses image features and gimbal motor data through a CNN-LSTM network to output the gimbal stabilization effect score and confidence interval.

[0046] The feedback optimization module is used to call the preset optimal test parameters, and dynamically adjust the vibration table parameters by combining the gimbal stabilization effect score and the confidence interval through reinforcement learning algorithm.

[0047] Thirdly, this application also provides a control device, the device comprising:

[0048] It includes a memory and a processor, wherein the memory stores a computer program that can be loaded and executed by the processor, such as the above-described UAV-based gimbal measurement and control method.

[0049] Fourthly, this application also provides a computer-readable storage medium storing a computer program that can be loaded by a processor and executed as described above regarding the gimbal control method based on unmanned aerial vehicles.

[0050] In summary, this application achieves efficient testing and optimization of gimbal performance through physical testing and digital twin simulation. First, a 6-axis vibration table is used to reproduce the flight jitter of a UAV, collecting a physical test dataset. Then, a coupled digital twin model of the vibration table and gimbal is constructed, and bidirectional synchronization of virtual and real states is achieved through the OPCUA protocol, generating a virtual simulation dataset. Next, edge computing is used to clean, align, and standardize the physical and virtual data, forming a multi-source dataset. Based on this dataset, a CNN-LSTM dual-branch transfer learning model is constructed, fusing image and motor data to output a stabilization performance score and confidence interval. Finally, a reinforcement learning algorithm is combined to dynamically adjust the vibration table parameters to optimize the test conditions. This solution expands the test scenario coverage through digital twin technology, enhances the diversity of model training samples through multi-source data fusion, improves prediction accuracy through transfer learning, and achieves adaptive optimization of test parameters through reinforcement learning, significantly improving gimbal testing efficiency and the accuracy of stabilization performance evaluation. It is applicable to the intelligent measurement and control field of precision equipment such as UAV gimbals. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating a gimbal monitoring and control method based on unmanned aerial vehicles (UAVs).

[0052] Figure 2 This is a structural block diagram of a gimbal control device based on unmanned aerial vehicles (UAVs). Detailed Implementation

[0053] The following combination Figure 1 - Figure 2 This application will be described in further detail.

[0054] With the widespread application of drone technology in aerial surveying, power line inspection, and emergency rescue, the gimbal system on board, as a core component, needs to maintain high-precision image stabilization performance under complex flight conditions. Traditional gimbal testing methods rely on repeated debugging of physical prototypes, which has problems such as long testing cycles (7-15 days per test), high cost of reproducing extreme conditions (such as strong winds and high-frequency vibration scenarios), and insufficient data sample coverage, thus limiting the accuracy of image stabilization performance evaluation and optimization efficiency.

[0055] To address the aforementioned pain points, this invention proposes a gimbal monitoring and control method based on digital twins and transfer learning. Through physical testing and virtual simulation, it improves testing efficiency and achieves accurate performance evaluation. The innovative approach combines a 6-axis vibration table physical test dataset with a SolidWorks-ANSYS coupled digital twin model. Using the OPCUA protocol, it achieves bidirectional synchronization between virtual and real states, generating a virtual simulation dataset covering over 100 disturbance conditions. Furthermore, a CNN-LSTM network is employed to fuse image features and motor data, constructing a transfer learning prediction model that outputs a stabilization performance score and confidence interval. Finally, reinforcement learning is used to dynamically optimize the vibration table parameters, forming an intelligent monitoring and control system driven by data, predictive models, and self-optimizing parameters.

[0056] Reference Figure 1 The embodiments of this application include at least steps S10 to S50.

[0057] The S10 uses a 6-axis programmable vibration table to reproduce the flight jitter attitude of a drone, and uses sensors to collect vibration table motion parameters, gimbal output images and gimbal contact force data to form a physical test dataset.

[0058] S20 is based on SolidWorks and ANSYS to build a digital twin model of vibration table-gimbal coupling. It adopts the OPCUA protocol and event flow driven mapping mechanism to inject physical test datasets into the digital twin model, realize bidirectional synchronization of physical and virtual states, and generate virtual simulation datasets.

[0059] S30 calls the virtual simulation dataset output by the digital twin model and merges it with the physical test dataset to form a multi-source dataset.

[0060] S40 uses a digital twin model as the basis for simulation to build a transfer learning prediction model. Based on a multi-source dataset, it fuses image features and gimbal motor data through a CNN-LSTM network to output the gimbal stabilization effect score and confidence interval.

[0061] The S50 calls the preset optimal test parameters and, combined with the gimbal stabilization effect score and confidence interval, dynamically adjusts the vibration table parameters through a reinforcement learning algorithm.

[0062] Specifically, a 6-axis vibration table is used to reproduce the flight vibrations of a UAV and collect physical data. A digital twin model built using SolidWorks and ANSYS is then used to achieve bidirectional synchronization between the virtual and real worlds, generating a virtual simulation dataset with extended operating conditions. After fusing multi-source data through edge computing, a transfer learning model is constructed using a CNN-LSTM network, fusing image and motor data to output a stable image score and confidence interval. Finally, reinforcement learning is used to dynamically optimize the vibration table parameters, forming a closed-loop control. This architecture overcomes the limitations of traditional testing, expanding operating condition coverage through digital twins, improving model accuracy through multi-source data, and achieving parameter self-optimization through reinforcement learning. This significantly shortens the testing cycle and improves the accuracy of gimbal stabilization performance evaluation.

[0063] In some embodiments, step S10 specifically includes the following steps: using IMU data as vibration source data, inputting it into a 6-axis programmable vibration table to reproduce the aircraft's flight shaking attitude and frequency; collecting vibration table motion parameters and gimbal contact force data through sensors; simultaneously, based on a preset black and white checkerboard pattern used for camera calibration, taking pictures of the black and white checkerboard pattern through the gimbal to obtain the gimbal output image; calculating the vectors between the corner points of the checkerboard pattern through the gimbal output image; if the difference between the vectors between the corner points and the preset stationary vector is less than a threshold, then it is determined that the gimbal stabilization effect has met the initial requirements; if the difference between the vectors between the corner points and the preset stationary vector is not less than the threshold, then it is determined that the gimbal stabilization effect has not met the standards.

[0064] In practice, a programmable 6-axis vibration table (forward and backward motion axis, left and right motion axis, up and down motion axis, roll motion axis, pitch motion axis, yaw motion axis).

[0065] In flight environments where the image stabilization effect of the mechanical gimbal can be effectively measured, such as in random gusts of wind, IMU data is collected during actual drone flight for approximately 10-15 minutes. This IMU data is then used as vibration source data and input into a programmable vibration table, allowing the table to reproduce the aircraft's flight vibration attitude and frequency.

[0066] Multiple mechanical pan-tilt units are mounted on a vibration table and connected to power and testing equipment. The power supply ensures the mechanical pan-tilt units function properly, while the testing equipment connects to cameras inside the mechanical pan-tilt units, transmitting the video images captured by the cameras to a computer for display and evaluation.

[0067] A standard black and white checkerboard pattern for camera calibration is created. The mechanical gimbal to be tested on the vibration table is aligned with the checkerboard to acquire images, ensuring the entire checkerboard is captured. OpenCV, an open computer vision library, has a dedicated interface for checkerboard detection, which can detect the corner points of the completed checkerboard using image data. In a static state, the checkerboard image is acquired, the corner points are detected, and the vector relationships between the corner points are calculated to form a feature descriptor d0. The vibration table is then started to simulate drone flight vibration. In this state, the checkerboard corner points are detected, the vector relationships between the corner points are calculated, and a feature descriptor di is formed. The difference between di and d0 is calculated. If the difference exceeds a given threshold T0, the image is considered to have shake, and the gimbal's stabilization effect is not up to standard; if the difference is less than the given threshold T0, the gimbal's stabilization effect is considered OK, and the initial test is passed.

[0068] If multiple mechanical pan-tilt units are used in the vibration table test, that number of threads are activated simultaneously to evaluate the pan-tilt unit stabilization effect. The evaluation results are then output to the screen for easy viewing by the tester.

[0069] In some embodiments, step S20 specifically includes the following steps: establishing a three-dimensional geometric model of the vibration table-gimbal system using SolidWorks, and parametrically modeling the gimbal motor, including rotor inertia, stator winding resistance, and permanent magnet flux linkage parameters, and exporting it in Parasolid format; importing the three-dimensional geometric model into ANSYS Workbench, meshing, applying material properties, performing modal analysis using ANSYS Mechanical, extracting the first 20 mode shapes, and generating a reduced-order model using the Craig-Bampton method; calling the reduced-order model to simulate the dynamic response of the vibration table-gimbal in ANSYS Twin Builder; establishing a two-dimensional finite element model of the gimbal motor using ANSYS Maxwell, and calculating the electromagnetic torque under different currents; embedding a PID controller model to achieve closed-loop control of the motor rotation angle; and defining multiphysics interface equations using Modelica language to realize a digital twin model.

[0070] Furthermore, step S20 also includes the following steps: constructing a dedicated information model based on the OPCUA standard, realizing physical-virtual data mapping by defining objects, variables, and event nodes, and establishing a secure communication link using TLS 1.3 encryption and X.509 certificate authentication to achieve real-time injection of the physical test dataset; performing bidirectional calibration of the physical state and the virtual model through a preset physical-virtual event rule base and a synchronization strategy of timestamp alignment and state feedback; setting a virtual perturbation parameter set in the digital twin model, driving the digital twin model to perform parameter perturbation simulation for each physical test sample, and generating a physical and virtual comparison dataset; and storing the dynamic response data and performance prediction data in the comparison dataset in a unified format as a virtual simulation dataset.

[0071] Specifically, a high-precision digital twin model is constructed through multi-tool collaboration and cross-platform integration to achieve real-time linkage between the physical and virtual systems. First, SolidWorks is used to complete the 3D geometric modeling of the vibration table and gimbal, focusing on the parametric design of the gimbal motor, accurately including core electromagnetic parameters such as rotor inertia, laying the foundation for subsequent multiphysics simulations. The ANSYS toolchain is used to complete the entire process from modal analysis to reduced-order model generation, employing the Craig-Bampton method to retain key dynamic characteristics while reducing computational complexity. Combined with Maxwell motor simulation and PID control models, multi-domain coupling of mechanical, electromagnetic, and control systems is achieved.

[0072] Building upon this foundation, a secure communication architecture is constructed based on the OPCUA standard. Real-time injection of physical test datasets is ensured through TLS 1.3 encryption and certificate authentication, while microsecond-level virtual-physical synchronization is achieved using timestamp alignment and state feedback mechanisms. A virtual perturbation parameter set is innovatively introduced to generate multi-dimensional perturbation conditions for each physical sample, significantly expanding the test boundaries. The resulting digital twin model not only accurately reproduces the dynamic response of the physical system but also generates scarce data such as extreme conditions and fault injections through virtual simulation. This provides a high-quality sample library for subsequent multi-source data fusion and transfer learning, effectively addressing the industry pain points of high cost and incomplete condition coverage in traditional physical testing.

[0073] In some embodiments, step S30 specifically includes the following steps: physical test datasets and virtual simulation datasets are cleaned, aligned, and dimensionality reduced using edge computing; physical and virtual data are standardized based on industrial metadata standards; physical and virtual data are associated to achieve data mapping, resulting in structured data with unified semantic labels; based on the structured data, time series alignment at a 1kHz sampling rate is achieved using a dynamic time warping algorithm, generating a one-to-one correspondence between physical test samples and virtual disturbance samples; and time-domain-frequency domain joint features are extracted using a dual-channel feature fusion network, outputting fused feature data that can be directly used for model training; the fused feature data is organized according to a three-level structure of spatiotemporal index, feature type, and operating condition label, and stored as a multi-source dataset containing physical measured features, virtual simulation features, and fused derived features.

[0074] Specifically, edge computing enables deep integration of physical and virtual data: First, the physical test and virtual simulation datasets are cleaned, aligned, and dimensionality reduced. Standardization is then performed based on industrial metadata standards to establish structured data with unified semantic labels. A dynamic time warping algorithm is used to achieve precise alignment of time series data at a 1kHz sampling rate. A dual-channel feature fusion network extracts joint time-domain and frequency-domain features. Finally, the data is organized into a three-level structure of "spatiotemporal index - feature type - operating condition label," forming a multi-source dataset containing physical measurements, virtual simulations, and fused derivative features. This process solves the problem of heterogeneity between physical and virtual data, improves sample time synchronization accuracy and feature representation capabilities, provides high-quality training data for transfer learning models, and supports accurate prediction of gimbal stabilization effects.

[0075] In some embodiments, step S40 specifically includes the following steps: Based on a multi-source dataset, a CNN-LSTM dual-branch transfer learning network model is constructed: the CNN branch receives image features from the multi-source dataset and extracts 512-dimensional spatial features through 3 layers of convolution; the LSTM branch receives time-series data of the gimbal motor and extracts 256-dimensional temporal features through 2 layers of bidirectional LSTM, outputting an initial model architecture containing a feature extraction layer and a fusion interface; the spatial features and temporal features are weighted and fused through an attention mechanism, and the output dimension is a cross-modal joint feature vector; the cross-modal joint feature vector is used as input, and the image stabilization effect score in the physical test is used as the supervision label, and a transfer learning strategy is used to train the CNN-LSTM dual-branch transfer learning network model, minimizing the mean square error between the predicted score and the true score through the Adam optimizer, and outputting converged model parameters; the real-time acquired gimbal image data and motor data are input into the trained model, and the gimbal image stabilization effect score and confidence interval are output, wherein the confidence interval is calculated through the bootstrap resampling algorithm to achieve quantitative evaluation and uncertainty quantification of the gimbal image stabilization performance.

[0076] Specifically, a CNN-LSTM dual-branch transfer learning network is used to extract image spatial features (512 dimensions) and motor temporal features (256 dimensions) respectively. An attention mechanism is employed to dynamically weight and fuse cross-modal features. Physical test scores are used as supervisory labels, and transfer learning is used to optimize model parameters and minimize prediction error. Finally, a real-time image stabilization performance score and a confidence interval based on the bootstrap algorithm are output. This design overcomes the limitations of traditional single-data evaluation, integrates visual and control domain information to improve prediction robustness, and quantifies uncertainty using confidence intervals, providing a reliable decision-making basis for subsequent reinforcement learning parameter optimization, thus achieving intelligent transformation from data to performance evaluation.

[0077] In some embodiments, step S50 specifically includes the following steps: reading the preset optimal test parameters stored in the PLC controller, and constructing a reinforcement learning environment by combining the gimbal stabilization effect score: defining the vibration table amplitude, frequency, and waveform type as the continuous action space, defining the gimbal stabilization effect score and the width of the confidence interval as the state space, and outputting the environment initialization configuration file; constructing a bi-objective reward function based on the gimbal stabilization effect and the confidence interval: the basic reward is positively correlated with the gimbal stabilization effect, the penalty term is positively correlated with the width of the confidence interval, the comprehensive reward value = basic reward + penalty term, and outputting the real-time reward value; using a deep deterministic policy gradient algorithm, taking the state space as input and the real-time reward value as feedback signal, iteratively optimizing the vibration table parameters, sending the new parameters to the vibration table controller through the OPCUA protocol, and outputting the optimized parameter record with a timestamp.

[0078] Specifically, a reinforcement learning environment is first constructed based on PLC preset parameters and gimbal stabilization score. Key parameters such as vibration table amplitude and frequency are defined as continuous action space. The stabilization effect score and confidence interval width are used as state observation dimensions to achieve dynamic representation of the test conditions. A dual-objective reward function is innovatively designed to incentivize high-scoring stabilization effects through basic rewards, while introducing a confidence interval penalty term to suppress evaluation uncertainty.

[0079] Subsequently, the Deep Deterministic Policy Gradient Algorithm (DDPG) is used for parameter iteration. The state-space data drives the policy network to output the optimal control quantity, which is then transmitted to the physical vibration table in real time via the OPCUA protocol, forming a "perception-decision-execution" closed loop. This mechanism breaks through the limitations of traditional fixed-parameter testing, and can adaptively adjust the excitation conditions according to the dynamic response of the gimbal. It can quickly converge to the optimal parameter combination under extreme conditions, significantly improving testing efficiency and image stabilization performance optimization accuracy, and providing an intelligent solution for the robustness verification of UAV gimbals.

[0080] The implementation principle of the UAV-based gimbal measurement and control method in this application is as follows: Efficient testing and optimization of gimbal performance is achieved through physical testing and digital twin simulation. First, a 6-axis vibration table is used to reproduce the flight jitter of the UAV, collecting a physical test dataset. Then, a coupled digital twin model of the vibration table and gimbal is constructed, and bidirectional synchronization of virtual and real states is achieved through the OPCUA protocol, generating a virtual simulation dataset. Next, edge computing is used to clean, align, and standardize the physical and virtual data, forming a multi-source dataset. Based on this dataset, a CNN-LSTM dual-branch transfer learning model is constructed, fusing image and motor data to output a stabilization performance score and confidence interval. Finally, a reinforcement learning algorithm is combined to dynamically adjust the vibration table parameters to optimize the test conditions. This solution expands the test scenario coverage through digital twin technology, enhances the diversity of model training samples through multi-source data fusion, improves prediction accuracy through transfer learning, and achieves adaptive optimization of test parameters through reinforcement learning, significantly improving gimbal testing efficiency and the accuracy of stabilization performance evaluation. It is applicable to the intelligent measurement and control field of precision equipment such as UAV gimbals.

[0081] Figure 1 This is a flowchart illustrating a gimbal monitoring and control method based on an unmanned aerial vehicle (UAV) in one embodiment. It should be understood that, although... Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows; unless explicitly stated otherwise, there is no strict order requirement for the execution of these steps, and they can be executed in other orders; and Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0082] Based on the same technical concept, referring to Figure 2 This application also provides a gimbal monitoring and control device based on an unmanned aerial vehicle (UAV), which adopts the following technical solution: The device includes:

[0083] The physical testing module is used to reproduce the flight jitter attitude of the UAV through a 6-axis programmable vibration table, and to collect the motion parameters of the vibration table, the output image of the gimbal, and the contact force data of the gimbal through sensors to form a physical testing dataset.

[0084] The digital twin module is used to build a coupled digital twin model of shaking table and gimbal based on SolidWorks and ANSYS. It adopts the OPCUA protocol and event flow driven mapping mechanism to inject physical test datasets into the digital twin model, realize bidirectional synchronization of physical and virtual states, and generate virtual simulation datasets.

[0085] The multi-source data module is used to call the virtual simulation dataset output by the digital twin model and merge it with the physical test dataset to form a multi-source dataset;

[0086] The gimbal testing module is used to build a transfer learning prediction model based on a digital twin model. Based on a multi-source dataset, it fuses image features with gimbal motor data through a CNN-LSTM network to output the gimbal stabilization effect score and confidence interval.

[0087] The feedback optimization module is used to call the preset optimal test parameters, and dynamically adjust the vibration table parameters by combining the gimbal stabilization effect score and confidence interval through reinforcement learning algorithm.

[0088] In some embodiments, the physical testing module is specifically used to input IMU data as vibration source data into a 6-axis programmable control vibration table, so that the vibration table can reproduce the aircraft's flight shaking attitude and frequency, and collect the vibration table's motion parameters and gimbal contact force data through sensors.

[0089] Simultaneously, based on the preset black and white checkerboard pattern used for camera calibration, the gimbal captures images of the black and white checkerboard pattern to obtain the gimbal output image;

[0090] The vectors between the corner points of the chessboard are calculated by outputting the image from the gimbal. If the difference between the vectors between the corner points and the preset stationary vector is less than a threshold, it is determined that the gimbal stabilization effect has met the initial requirements.

[0091] If the difference between the vector between corner points and the preset stationary vector is not less than the threshold, then the gimbal stabilization effect is deemed unsatisfactory.

[0092] In some embodiments, the digital twin module is specifically used to create a three-dimensional geometric model of the vibration table-gimbal system using SolidWorks, and to perform parametric modeling of the gimbal motor, including rotor inertia, stator winding resistance and permanent magnet flux linkage parameters, and export them in Parasolid format.

[0093] Import the 3D geometric model into ANSYS Workbench, mesh it, apply material properties, perform modal analysis using ANSYS Mechanical, extract the first 20 mode shapes, and generate a reduced-order model using the Craig-Bampton method.

[0094] The dynamic response of the vibration table-gimbal was simulated in ANSYS Twin Builder using a reduced-order model; a two-dimensional finite element model of the gimbal motor was established using ANSYS Maxwell to calculate the electromagnetic torque under different currents; and a PID controller model was embedded to realize closed-loop control of the motor rotation angle.

[0095] Digital twin models are realized by defining multiphysics interface equations using the Modelica language.

[0096] In some embodiments, the digital twin module is specifically used to build a dedicated information model based on the OPCUA standard, realize physical-virtual data mapping by defining objects, variables and event nodes, and establish a secure communication link using TLS1.3 encryption and X.509 certificate authentication to realize the real-time injection of physical test datasets;

[0097] By using a pre-defined physical-virtual event rule base and employing a synchronization strategy of timestamp alignment and state feedback, the physical state and the virtual model are calibrated bidirectionally.

[0098] A virtual perturbation parameter set is set in the digital twin model. For each physical test sample, the digital twin model is driven to perform parameter perturbation simulation to generate a comparison dataset of physical and virtual data.

[0099] Based on the dynamic response data and performance prediction data in the comparison dataset, the data is stored in a unified format as a virtual simulation dataset.

[0100] In some embodiments, the multi-source data module is specifically used to perform physical data cleaning, virtual data alignment and data dimensionality reduction processing on physical test datasets and virtual simulation datasets through edge computing, and to standardize physical data and virtual data based on industrial metadata standards, associate physical data with virtual data, realize data mapping, and obtain structured data with unified semantic labels.

[0101] Based on structured data, a dynamic time warping algorithm is used to align time series at a sampling rate of 1kHz, generating a one-to-one correspondence between physical test samples and virtual perturbation samples. A dual-channel feature fusion network is used to extract joint time-frequency features, outputting fused feature data that can be directly used for model training.

[0102] The fused feature data is organized into a three-level structure based on spatiotemporal index, feature type, and operating condition label, and stored as a multi-source dataset containing physical measured features, virtual simulation features, and fused derived features.

[0103] In some embodiments, the gimbal testing module is specifically used to construct a CNN-LSTM dual-branch transfer learning network model based on a multi-source dataset: the CNN branch receives image features from the multi-source dataset and extracts 512-dimensional spatial features through 3 layers of convolution; the LSTM branch receives time-series data from the gimbal motor and extracts 256-dimensional temporal features through 2 layers of bidirectional LSTM, outputting an initial model architecture that includes a feature extraction layer and a fusion interface.

[0104] Spatial and temporal features are weighted and fused using an attention mechanism, and the output dimension is a cross-modal joint feature vector.

[0105] Using the cross-modal joint feature vector as input and the image stabilization effect score in the physical test as the supervision label, a CNN-LSTM dual-branch transfer learning network model is trained using a transfer learning strategy. The mean square error between the predicted score and the true score is minimized through the Adam optimizer, and the converged model parameters are output.

[0106] The real-time acquired gimbal image data and motor data are input into the trained model, which outputs the gimbal stabilization performance score and confidence interval. The confidence interval is calculated using the bootstrap resampling algorithm to achieve quantitative evaluation and uncertainty quantification of the gimbal stabilization performance.

[0107] In some embodiments, the feedback optimization module is specifically used to read the preset optimal test parameters stored in the PLC controller, combine them with the gimbal stabilization effect score, and construct a reinforcement learning environment: the vibration table amplitude, frequency and waveform type are defined as the continuous action space, the gimbal stabilization effect score and the width of the confidence interval are defined as the state space, and the environment initialization configuration file is output.

[0108] A dual-objective reward function is constructed based on the gimbal stabilization effect and the confidence interval: the basic reward is positively correlated with the gimbal stabilization effect, the penalty term is positively correlated with the width of the confidence interval, the comprehensive reward value = basic reward + penalty term, and the real-time reward value is output.

[0109] A deep deterministic policy gradient algorithm is adopted, which takes the state space as input and the real-time reward value as feedback signal to iteratively optimize the vibration table parameters. The new parameters are sent to the vibration table controller through the OPCUA protocol, and the optimized parameter records with timestamps are output.

[0110] This application also discloses a control device.

[0111] Specifically, the control device includes a memory and a processor. The memory stores a computer program that can be loaded and executed by the processor to implement the aforementioned UAV-based gimbal control method.

[0112] This application also discloses a computer-readable storage medium.

[0113] Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and executed, such as the above-described UAV-based gimbal control method. The computer-readable storage medium includes, for example, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0114] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A gimbal monitoring and control method based on unmanned aerial vehicles (UAVs), characterized in that, include: The 6-axis programmable vibration table is used to reproduce the flight jitter attitude of the UAV, and the vibration table motion parameters, gimbal output images and gimbal contact force data are collected by sensors to form a physical test dataset. A digital twin model of a shaking table-gimbal coupling was constructed based on SolidWorks and ANSYS. The physical test dataset was injected into the digital twin model using the OPCUA protocol and event flow-driven mapping mechanism to achieve bidirectional synchronization between physical and virtual states and generate a virtual simulation dataset. The virtual simulation dataset output by the digital twin model is called and fused with the physical test dataset to form a multi-source dataset; Based on the digital twin model, a transfer learning prediction model is constructed. Based on the multi-source dataset, image features and gimbal motor data are fused through a CNN-LSTM network to output the gimbal stabilization effect score and confidence interval. The preset optimal test parameters are called, and the gimbal stabilization effect score and the confidence interval are combined to dynamically adjust the vibration table parameters through reinforcement learning algorithm; The method involves reproducing the drone's flight jitter attitude using a 6-axis programmable vibration table, and collecting vibration table motion parameters, gimbal output images, and gimbal contact force data through sensors to form a physical test dataset, including: IMU data is used as vibration source data and input into a 6-axis programmable vibration table to reproduce the drone's flight shaking attitude and frequency. Sensors are used to collect the motion parameters of the vibration table and the contact force data of the gimbal. Simultaneously, based on a preset black and white checkerboard pattern used for camera calibration, the gimbal captures images of the black and white checkerboard pattern to obtain the gimbal output image; The vectors between the corner points of the chessboard are calculated by the output image of the gimbal. If the difference between the vectors between the corner points and the preset stationary vector is less than a threshold, it is determined that the gimbal stabilization effect has met the initial requirements. If the difference between the vector between the corner points and the preset stationary vector is not less than a threshold, then the gimbal stabilization effect is deemed unsatisfactory.

2. The gimbal monitoring and control method based on an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, The digital twin model of the shaking table-gimbal coupling constructed based on SolidWorks and ANSYS includes: A three-dimensional geometric model of the vibration table-panel system was created using SolidWorks, and the panel motor was parametrically modeled, including rotor inertia, stator winding resistance, and permanent magnet flux linkage parameters, and exported in Parasolid format. Import the 3D geometric model into ANSYS Workbench, mesh it, apply material properties, perform modal analysis using ANSYS Mechanical, extract the first 20 mode shapes, and generate a reduced-order model using the Craig-Bampton method. The reduced-order model is called to simulate the dynamic response of the vibration table-gimbal in ANSYS Twin Builder; a two-dimensional finite element model of the gimbal motor is established using ANSYS Maxwell, the electromagnetic torque under different currents is calculated, and a PID controller model is embedded to realize closed-loop control of the motor rotation angle. The digital twin model is implemented by defining multiphysics interface equations using the Modelica language.

3. The gimbal monitoring and control method based on an unmanned aerial vehicle (UAV) according to claim 2, characterized in that, The OPCUA protocol and event-driven mapping mechanism are used to inject the physical test dataset into the digital twin model, achieving bidirectional synchronization between physical and virtual states and generating a virtual simulation dataset, including: A dedicated information model is built based on the OPCUA standard. Physical-virtual data mapping is achieved by defining objects, variables and event nodes. A secure communication link is established using TLS 1.3 encryption and X.509 certificate authentication to realize the real-time injection of the physical test dataset. By using a pre-defined physical-virtual event rule base and employing a synchronization strategy of timestamp alignment and state feedback, the physical state and the virtual model are calibrated bidirectionally. A virtual perturbation parameter set is set in the digital twin model. For each physical test sample, the digital twin model is driven to perform parameter perturbation simulation to generate a comparison dataset of physical and virtual data. Based on the dynamic response data and performance prediction data in the comparison dataset, they are stored in a unified format as a virtual simulation dataset.

4. The gimbal monitoring and control method based on an unmanned aerial vehicle (UAV) according to claim 3, characterized in that, The virtual simulation dataset output by the digital twin model is fused with the physical test dataset to form a multi-source dataset, including: The physical test dataset and the virtual simulation dataset are cleaned, aligned, and dimensionality reduced by edge computing. Based on industrial metadata standards, the physical and virtual data are standardized, and the physical and virtual data are associated to achieve data mapping and obtain structured data with unified semantic labels. Based on the structured data, a time series alignment at a sampling rate of 1kHz is achieved through a dynamic time warping algorithm, generating a one-to-one correspondence between physical test samples and virtual perturbation samples. Then, a dual-channel feature fusion network is used to extract joint time-frequency features, outputting fused feature data that can be directly used for model training. The fused feature data is organized into a three-level structure of spatiotemporal index, feature type, and operating condition label, and stored as the multi-source dataset containing physical measured features, virtual simulation features, and fused derived features.

5. The gimbal monitoring and control method based on an unmanned aerial vehicle (UAV) according to claim 4, characterized in that, The transfer learning prediction model, constructed based on the digital twin model for simulation, uses the multi-source dataset and a CNN-LSTM network to fuse image features with gimbal motor data to output a gimbal stabilization performance score and confidence interval, including: Based on the multi-source dataset, a CNN-LSTM dual-branch transfer learning network model is constructed: the CNN branch receives the image features of the multi-source dataset and extracts 512-dimensional spatial features through 3 layers of convolution; the LSTM branch receives the time-series data of the gimbal motor and extracts 256-dimensional temporal features through 2 layers of bidirectional LSTM, outputting an initial model architecture that includes a feature extraction layer and a fusion interface. The spatial features and the temporal features are weighted and fused using an attention mechanism, and the output dimension is a cross-modal joint feature vector. Using the cross-modal joint feature vector as input and the image stabilization effect score in the physical test as the supervision label, the CNN-LSTM dual-branch transfer learning network model is trained using a transfer learning strategy. The mean square error between the predicted score and the real score is minimized through the Adam optimizer, and the converged model parameters are output. The real-time acquired gimbal image data and motor data are input into the trained model, and the gimbal stabilization effect score and the confidence interval are output. The confidence interval is calculated by the bootstrap resampling algorithm to achieve quantitative evaluation and uncertainty quantification of the gimbal stabilization performance.

6. The gimbal monitoring and control method based on an unmanned aerial vehicle (UAV) according to claim 5, characterized in that, The step of calling the preset optimal test parameters, combining the gimbal stabilization effect score and the confidence interval, and dynamically adjusting the vibration table parameters through a reinforcement learning algorithm includes: Read the preset optimal test parameters stored in the PLC controller, and combine them with the gimbal stabilization effect score to construct a reinforcement learning environment: define the vibration table amplitude, frequency and waveform type as the continuous action space, define the gimbal stabilization effect score and the width of the confidence interval as the state space, and output the environment initialization configuration file. Based on the gimbal stabilization effect score and the confidence interval, a bi-objective reward function is constructed: the basic reward is positively correlated with the gimbal stabilization effect score, the penalty term is positively correlated with the width of the confidence interval, the comprehensive reward value = basic reward + penalty term, and the real-time reward value is output. A deep deterministic policy gradient algorithm is adopted, with the state space as input and the real-time reward value as feedback signal, to iteratively optimize the vibration table parameters. The new parameters are sent to the vibration table controller through the OPCUA protocol, and the optimized parameter records with timestamps are output.

7. A gimbal control device based on an unmanned aerial vehicle (UAV), characterized in that, The device includes: The physical testing module is used to reproduce the flight jitter attitude of the UAV through a 6-axis programmable vibration table, and to collect the motion parameters of the vibration table, the output image of the gimbal, and the contact force data of the gimbal through sensors to form a physical testing dataset. The digital twin module is used to build a vibration table-gimbal coupled digital twin model based on SolidWorks and ANSYS. It adopts the OPCUA protocol and event flow driven mapping mechanism to inject the physical test dataset into the digital twin model, realize bidirectional synchronization of physical and virtual states, and generate a virtual simulation dataset. The multi-source data module is used to call the virtual simulation dataset output by the digital twin model and merge it with the physical test dataset to form a multi-source dataset; The gimbal testing module is used to build a transfer learning prediction model based on the digital twin model. Based on the multi-source dataset, it fuses image features and gimbal motor data through a CNN-LSTM network to output the gimbal stabilization effect score and confidence interval. The feedback optimization module is used to call the preset optimal test parameters, and dynamically adjust the vibration table parameters by combining the gimbal stabilization effect score and the confidence interval through reinforcement learning algorithm. The physical testing module is specifically used to input IMU data as vibration source data into a 6-axis programmable control vibration table, so that the vibration table can reproduce the drone's flight shaking attitude and frequency. Through sensors, the motion parameters of the vibration table and the contact force data of the gimbal are collected. Simultaneously, based on a preset black and white checkerboard pattern used for camera calibration, the gimbal captures images of the black and white checkerboard pattern to obtain the gimbal output image; The vectors between the corner points of the chessboard are calculated by the output image of the gimbal. If the difference between the vectors between the corner points and the preset stationary vector is less than a threshold, it is determined that the gimbal stabilization effect has met the initial requirements. If the difference between the vector between the corner points and the preset stationary vector is not less than a threshold, then the gimbal stabilization effect is deemed unsatisfactory.

8. A control device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer program is stored that can be loaded by a processor and executed as described in any one of claims 1 to 6.

Images