Training Method and System for Gimbal Heading Lock Control Parameter Scheduling Model
By training a multilayer perceptron model and using a teacher model to generate supervisory labels, the problem of gimbal heading lock control parameters being difficult to take into account the differences between aircraft models was solved, and adaptive gimbal heading lock control was achieved, improving control stability and adaptability.
Patent Information
- Application Number
- CN202611548401.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-08
- Publication Date
- 2026-11-17
AI Technical Summary
In existing technologies, the gimbal heading lock control parameters are difficult to take into account the differences between different models, and manually set rules are difficult to cover multiple operating conditions, affecting control stability. Furthermore, the output of the learning model directly affects the motor, which may lead to instability.
By training a multilayer perceptron model and generating supervised labels using a teacher model, adaptive parameter scheduling is performed based on common state feature vectors. Combined with consistency, smoothness, and security loss functions, it is applicable to multiple gimbal models and reduces the need for manual annotation.
It achieves adaptive gimbal heading lock control, applicable to multiple models, improving control stability and adaptability, and reducing the workload of redesigning scheduling rules.
Smart Images

Figure CN123113991A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image stabilization technology, and in particular relates to a training method and system for a gimbal heading lock control parameter scheduling model. Background Technology
[0002] A gimbal (also known as an electro-optical pod) is a device used to mount cameras, sensors, and other equipment, and to achieve attitude stabilization or pointing tracking of the equipment through multiple rotation axes. In heading lock mode, the gimbal needs to keep the equipment's heading fixed to achieve heading stabilization or heading tracking.
[0003] In such heading lock control loops, multiple control parameters typically need to be set. For example, to suppress low-frequency drift of the heading angle signal, the time constant on which drift estimation is based needs to be set; to distinguish between the actual rotation of the aircraft and the applicable timing of drift estimation, the threshold on which rotation determination is based and the stabilization waiting time after rotation need to be set; to generate the angle control error of the heading axis, the upper limit of drift correction gain, fusion gain, and compensation amount also needs to be set. These parameters together determine the tracking speed of drift estimation, the ability to distinguish between aircraft movements and drift, and the strength of compensation.
[0004] In existing technologies, the aforementioned control parameters are typically set by designers based on experience or according to fixed rules. However, fixed parameters are difficult to take into account the differences in load inertia, motor response, motor position sensing resolution, and focal length among different gimbal models. If parameter scheduling is performed according to manually designed rules, the design and maintenance of these rules depend on design experience, and the number of rules required increases with the increase in the dimensions of the operating conditions, making it difficult to fully cover situations such as telephoto shooting, camera rotation, and high position sensing noise.
[0005] On the other hand, if the driving commands for the yaw axis motor are directly output from the learning model, the model's output will directly affect the yaw axis motor, and its abnormal output may affect the stability of the yaw lock control. Furthermore, ideal control parameters applicable to various states are difficult to label manually, and the control data required for training must cover multiple gimbal models and various operating states. Therefore, how to train a parameter scheduling model that can adaptively schedule yaw lock control parameters according to real-time states and is easy to deploy in embedded processors has become a technical problem that urgently needs to be solved in this field. Summary of the Invention
[0006] To address the shortcomings of the existing technology, this invention provides a training method for a gimbal heading lock control parameter scheduling model, comprising: The control data records of multiple gimbal models are obtained. The control data records are the operational data generated and recorded when the gimbal is running in heading lock mode. Each of the control data records is parsed and aligned to the same control cycle sequence to convert each of the control data records into the same set of state features, resulting in a common state feature vector. The common state feature vector includes the absolute value of the heading lock error, the absolute value of the drift observation residual, the absolute value of the change in the drift observation residual, the absolute value of the change in the second heading angle, the absolute value of the heading axis angular velocity, the amount of motor position sensing noise, and the zoom parameters. The common state feature vector does not contain the model identification information of the multiple gimbal models. Teacher labels are generated based on the common state feature vector using the teacher model. The teacher labels include drift estimation time constant, rotation decision threshold, stabilization waiting time, drift correction gain, fusion gain, and maximum compensation amount. Using the common state feature vector as input and the teacher label as the supervision target, a multilayer perceptron model is trained. The output layer of the multilayer perceptron model is six-dimensional, and the six dimensions correspond sequentially to the drift estimation time constant, the rotation determination threshold, the stabilization waiting time, the drift correction gain, the fusion gain, and the maximum compensation amount.
[0007] On the other hand, the present invention also provides a training system for a gimbal heading lock control parameter scheduling model, comprising: a data recording unit for acquiring control data records of multiple gimbal models, wherein the control data records are operational data generated and recorded by the gimbal during operation in heading lock mode; and a feature conversion unit for parsing and aligning the control data records to the same control cycle sequence, thereby converting the control data records into the same set of state features and calculating the features included in a common state feature vector, wherein the common state feature vector includes the absolute value of the heading lock error, the absolute value of the drift observation residual, the absolute value of the change in the drift observation residual, the absolute value of the change in the second heading angle, the absolute value of the heading axis angular velocity, the motor position sensing noise, and the zoom parameters, and the common state feature vector does not include the aforementioned features. The system includes: model identification information for multiple gimbal models; a teacher model unit for generating teacher labels based on a teacher model, the teacher labels including drift estimation time constant, rotation judgment threshold, stabilization waiting time, drift correction gain, fusion gain, and maximum compensation amount; a training unit for training a multilayer perceptron model with the common state feature vector as input and the teacher labels as the supervision target, the multilayer perceptron model having a seven-dimensional input layer, a six-dimensional output layer, and including multiple sequentially connected hidden layers; a model verification unit for verifying the trained multilayer perceptron model; and a model versioning unit for performing at least one of quantization, pruning, compression, and format conversion on the verified multilayer perceptron model, and generating multiple model versions with version identifiers based on the multiple gimbal models.
[0008] This invention uses a teacher model to generate teacher labels based on a common state feature vector, and then uses these teacher labels as the supervised target to train a multilayer perceptron model. The teacher model can provide corresponding parameter conclusions for any input, obtaining training labels without manually labeling each piece of control data. The common state feature vector does not contain model identification information for the multiple gimbal models, thus the control data for different gimbal models are represented within the same feature space. Based on this design, the same trained model can be applied to multiple gimbal models, and multiple model versions with version identifiers can be generated for each model, thereby reducing the workload required to redesign scheduling rules or readjust control parameters for each model. Furthermore, this invention can also perform semi-supervised training on unlabeled data without teacher labels. This unlabeled data includes real-world operation records under conditions such as telephoto shooting, camera rotation, and high sensing noise, improving the trained model's adaptability under these conditions. During training, this invention incorporates consistency loss, smoothing loss, and safety loss into the weight update of the multilayer perceptron model, making the output of the trained model more continuous in time series and constrained within a safe range. Furthermore, the multilayer perceptron model uses a seven-dimensional common state feature vector as input, and its small size makes it suitable for deployment in embedded processors of gimbal control systems. Attached Figure Description
[0009] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 and Figure 2 This is a flowchart illustrating a preferred embodiment of the training method for a gimbal heading lock control parameter scheduling model according to the present invention.
[0010] Figure 3 This is a schematic diagram of a preferred embodiment of a training system for a gimbal heading lock control parameter scheduling model according to the present invention.
[0011] Figure 4 for Figure 3 A schematic diagram of the preferred implementation of the teacher model unit. Detailed Implementation
[0012] The following specific examples illustrate the implementation of the present invention. The illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0013] This invention discloses a training method and system for a gimbal heading lock control parameter scheduling model, applicable to gimbals or optoelectronic pods requiring heading lock. The gimbal includes a heading axis control system capable of achieving heading lock mode. The gimbal drives the body to rotate around the heading axis via a heading axis motor to achieve heading stability or heading tracking. After training, the parameter scheduling model is deployed in the embedded processor of the gimbal control system and outputs adaptive control parameters based on real-time state characteristics in heading lock mode, thereby scheduling drift estimation, body motion determination, and compensation constraints. For ease of understanding, the preferred implementation method in the training phase is described first, followed by the preferred implementation method in the operation phase of applying the trained model.
[0014] Please refer to Figure 1 As shown, a preferred embodiment of the training method for a gimbal heading lock control parameter scheduling model according to the present invention includes the following steps: Step S1: Obtain control data records for multiple PTZ camera models.
[0015] In this step, the control data record refers to the operational data generated and recorded when the gimbal is running in heading lock mode. The control data record can be obtained from multiple gimbal models, which may differ in aspects such as motor torque, motor position sensing resolution, load inertia, convergence characteristics of the proportional-integral-derivative (PID) controller, focal length range, motor position sensing noise level, and inertial sensing noise level. The control data record can be obtained in at least one of the following ways: performing a live hover test on at least one gimbal model and recording its control data; performing a long-term heading lock test on at least one gimbal model and recording its control data; performing a test on at least one gimbal model in telephoto shooting mode and recording its control data; performing slow rotation tests and fast rotation tests on at least one gimbal model and recording its control data; performing a test on at least one gimbal model with high motor position sensing noise and recording its control data; and using simulated control data generated from simulated control of the gimbal model and synthetic control data generated based on synthetic drift and synthetic noise. The control data records can also be taken from the operation process of the same model under different load inertia conditions or different motor response conditions, so as to obtain the control data records corresponding to the differences in load inertia and motor response.
[0016] The control data record may include at least one of the following: a first heading angle signal θ_q, a second heading angle signal θ_m, the raw value of the second heading angle signal θ_m,raw, the heading axis angular velocity ω_yaw, the zoom parameter zoom, the heading lock reference angle θ_lock, the first drift estimation reference θ_q,ref, the second drift estimation reference θ_m,ref, the heading lock error e_q, the drift observation residual disc, the drift estimation value drift_est, the freeze timer, and the motor position sensing noise value jitter_m. Among these items, the heading lock reference angle θ_lock, the first drift estimation reference θ_q,ref, the second drift estimation reference θ_m,ref, the heading lock error e_q, the drift observation residual disc, the drift estimation value drift_est, and the freeze timer are intermediate quantities in the control process. Including them in the control data record allows subsequent feature calculations to reproduce the runtime state on the training side.
[0017] In one example, the unit for recording control data can be installed within the gimbal control system. After the gimbal enters heading lock mode, it continuously records the aforementioned items according to the control cycle. The recorded data is correlated with each other using time stamps, thus forming a time-series control data record. In another example, for aircraft models for which actual flight data has not yet been obtained, control data records can be obtained by running and recording the data under simulated conditions using a gimbal dynamics model. The names and arrangement of the recorded items are consistent with the actual flight records, so that they can be processed together with the actual flight data later.
[0018] Step S2: Convert each control data record into the same set of state characteristics.
[0019] In this step, the control data records output by different PTZ camera models are converted into feature records of the same format. Since the data from different models may differ in signal names, dimensions, sampling times, and resolutions, they are first parsed and aligned, then the feature quantities are calculated based on the same set of state feature definitions, and written into the feature records in the same order.
[0020] In one example, the multiple gimbal models may include small gimbals, medium gimbals, long-focus optical pods, gimbals with high motor position sensing noise levels, and gimbals using high-resolution encoders, which can be referred to as model A, model B, model C, model D, and model E, respectively. The analysis includes the following processes: identifying the first heading angle signal θ_q, the second heading angle signal θ_m, the heading axis angular velocity ω_yaw, the zoom parameter zoom, and the corresponding data columns from each control data record; aligning each data column to the same control cycle sequence according to the time stamp; processing missing values according to predetermined interpolation or discard rules; and converting each feature quantity to the same dimension. After the above processing, the control data records of multiple gimbal models are represented in the same space defined by a common state feature vector, enabling the model to learn the mapping from state features to adaptive control parameters.
[0021] Step S3: Calculate the features included in the common state feature vector.
[0022] In this step, each characteristic quantity is calculated based on the same set of state characteristic definitions. The heading lock error e_q is determined based on the first heading angle signal θ_q and the heading lock reference angle θ_lock. That is, in the absence of propulsion control or external control commands, the heading lock error e_q is equal to the difference between the first heading angle signal θ_q and the heading lock reference angle θ_lock. In the presence of propulsion control or external control commands, the heading lock error e_q may also include the difference between the heading angle component θ_push corresponding to the propulsion control or external control command and its locking reference angle θ_push,lock.
[0023] The drift observation residual disc is determined based on the difference between the first movement Δq and the second movement Δm. The first movement Δq is the movement of the first heading angle signal θ_q relative to the first drift estimation reference θ_q,ref, which can be expressed as the difference between the first heading angle signal θ_q and the first drift estimation reference θ_q,ref. The second movement Δm is the movement of the second heading angle signal θ_m relative to the second drift estimation reference θ_m,ref, which can be expressed as the difference between the second heading angle signal θ_m and the second drift estimation reference θ_m,ref. Through the above calculation, the drift observation residual disc represents the difference between the degree of movement of the two heading angle signals relative to their respective references. Therefore, it is unnecessary to determine which signal is more accurate, nor is it necessary to replace one signal with the other for control. That is, the heading lock error e_q, the first movement Δq, the second movement Δm, and the drift observation residual disc are obtained according to the following formula: e_q = θ_q θ_lock, Δq=θ_q θ_q,ref, Δm=θ_m θ_m,ref,disc=Δq Δm=(θ_q θ_q,ref) (θ_m θ_m,ref). In the case of propulsion control or external control commands, the heading lock error e_q can also be obtained by the following formula: e_q=(θ_q) θ_lock)+(θ_push θ_push,lock).
[0024] The drift observation residual change, `disc_dot`, is the difference between the drift observation residual of the current control cycle and the drift observation residual of the previous control cycle. The second heading angle change, `mrate`, is the change in the second heading angle signal of the current control cycle relative to the initial reference angle θ_m,rate_ref, which is the second heading angle signal of the previous control cycle. The initial reference angle θ_m,rate_ref is updated to the second heading angle signal θ_m of the previous control cycle in each control cycle, and is set to the current second heading angle signal θ_m during initialization or a new snapshot.
[0025] The motor position sensing noise level jitter_m can be determined based on the high-frequency components of the second heading angle signal θ_m, which are obtained from the difference between the second heading angle signal θ_m and its low-pass filter result, for example, expressed as the root mean square of the difference between the second heading angle signal θ_m and its low-pass filter result; alternatively, the motor position sensing noise level jitter_m can be determined based on the difference between the maximum and minimum values of the second heading angle signal θ_m over a time period, where the time period consists of multiple continuous control cycles. The drift observation residual change disc_dot, the second heading angle change mrate, and the motor position sensing noise level jitter_m are obtained according to the following formula: disc_dot = disc[k]. disc[k 1], mrate=θ_m[k] θ_m,rate_ref,jitter_m=RMS(θ_m LPF(θ_m)), or, jitter_m=max(θ_m). min(θ_m).
[0026] After obtaining the above-mentioned characteristic quantities, the absolute values of the heading lock error e_q, drift observation residual disc, drift observation residual change disc_dot, second heading angle change mrate, and heading axis angular velocity ω_yaw are taken respectively, and arranged together with the motor position sensing noise jitter_m and zoom parameter zoom to obtain a common state feature vector, hereinafter denoted as X. The common state feature vector X is expressed by the following formula: X=[|e_q|, |disc|, |disc_dot|, |mrate|, |ω_yaw|, jitter_m, zoom]. The common state feature vector includes the absolute values of the heading lock error e_q, drift observation residual disc, drift observation residual change disc_dot, second heading angle change mrate, heading axis angular velocity ω_yaw, motor position sensing noise jitter_m, and zoom parameter zoom, and its dimension is seven.
[0027] In one example, the zoom parameter *zoom* is determined by at least one of the lens focal length, zoom ratio, or lens control position; the yaw axis angular velocity *ω_yaw* is obtained by an inertial sensing unit; and the second yaw angle signal *θ_m* is obtained by a motor position sensing unit, which is at least one of a Hall sensor, magnetic encoder, optical encoder, or other motor position sensor. In another example, to improve the comparability of feature quantities across different models, each feature quantity can be normalized according to the same conversion relationship before being written into the feature record. This conversion relationship is set to be the same during both the training and runtime phases.
[0028] Step S4: Generate teacher labels using the teacher model.
[0029] In this step, the teacher model generates teacher labels based on the common state feature vector. The teacher model is configured to generate the teacher labels according to fuzzy rules, which may include a fuzzification sub-step, a rule deduction sub-step, and a defuzzification sub-step.
[0030] The fuzzification sub-step is used to convert each feature quantity included in the common state feature vector into a corresponding semantic level; the rule deduction sub-step is used to generate control conclusions based on the semantic levels and fuzzy rules; the defuzzification sub-step is used to convert the control conclusions into the teacher labels. The teacher labels include the drift estimation time constant τ_adapt, the rotation decision threshold R_th,adapt, the stabilization waiting time T_settle,adapt, the drift correction gain G_drift,adapt, the fusion gain G_fusion,adapt, and the maximum compensation amount MaxComp_adapt, with a dimension of six.
[0031] The semantic levels used in the fuzzification sub-step can be set according to the feature quantities. For example, the absolute values of the heading lock error and drift observation residual can be classified into levels such as near zero, small, medium, and large; the absolute values of the change in drift observation residual can be classified into levels such as slow, medium, and fast; the absolute values of the change in the second heading angle can be classified into levels such as stationary, slow, medium, and fast; the absolute values of the heading axis angular velocity can be classified into levels such as low, medium, and high; the motor position sensing noise can be classified into levels such as low, medium, and high; and the zoom parameters can be classified into levels such as wide-angle, normal, and telephoto. The above semantic levels are determined by the membership function to which each feature quantity belongs. The membership function can be triangular, trapezoidal, or Gaussian, and its parameters can be set according to the specific operating conditions of the gimbal.
[0032] The fuzzy rules include at least one of the following rules: The first approach is as follows: when the semantic level corresponding to the absolute value of the second heading angle change mrate is low, the semantic level corresponding to the absolute value of the heading axis angular velocity ω_yaw is low, the semantic level corresponding to the absolute value of the drift observation residual disc is medium or high, and the semantic level corresponding to the absolute value of the drift observation residual change disc_dot is slow, the drift estimation time constant τ_adapt is reduced and the drift correction gain G_drift,adapt is increased. This means that the aircraft is approximately stationary while the drift continues to accumulate, so it is necessary to speed up the drift estimation and strengthen the compensation.
[0033] The second approach is to increase the stabilization waiting time T_settle,adapt and freeze the drift estimation value drift_est when the absolute value of the second heading angle change mrate is high or the absolute value of the heading axis angular velocity ω_yaw is high. This means that when the aircraft is in a rotating state, the two heading angle signals may contain real motion components at the same time, and drift estimation needs to be temporarily suspended to avoid absorbing the real rotation as drift.
[0034] The third approach is to increase the rotation judgment threshold R_th,adapt or decrease the drift correction gain G_drift,adapt when the semantic level corresponding to the motor position sensing noise jitter_m is high. This means that when the sensing noise is too high, the threshold for motion judgment needs to be increased and the compensation intensity weakened to reduce the possibility of misjudgment and overcompensation.
[0035] The fourth method involves adjusting the drift estimation time constant τ_adapt, the drift correction gain G_drift,adapt, and the maximum compensation amount MaxComp_adapt when the semantic level corresponding to the zoom parameter is telephoto and the camera is approximately stationary. This means that the image is more sensitive to angle changes in telephoto mode, and it is necessary to balance the smoothness of drift suppression with the limitation of compensation amount.
[0036] In one example, the rule deduction sub-step can be implemented using a lookup table, that is, using the semantic level of each feature as an index to retrieve the corresponding control conclusion from a pre-established rule table. In another example, the rule deduction sub-step can also be implemented using a composition operation, that is, first calculating the influence strength of each rule on the current input, and then weighting the control conclusions of each rule according to the influence strength. The defuzzification sub-step can be implemented using a weighted average method, a centroid method, or a lookup table interpolation method, and the obtained result is restricted to a preset parameter range to obtain the teacher label.
[0037] The dimension of the teacher label corresponds to the dimension of the output layer of the subsequent multilayer perceptron model, so that the teacher label can be directly used as a supervision target.
[0038] Step S5: Train a multilayer perceptron model with a supervised objective.
[0039] In this step, a multilayer perceptron model is trained using the common state feature vector as input and the teacher label as the supervision target. The multilayer perceptron model includes an input layer, multiple hidden layers, and an output layer. The input layer is seven-dimensional, corresponding to the seven features included in the common state feature vector; the output layer is six-dimensional, with its six output values corresponding sequentially to the drift estimation time constant τ_adapt, the rotation decision threshold R_th,adapt, the stabilization waiting time T_settle,adapt, the drift correction gain G_drift,adapt, the fusion gain G_fusion,adapt, and the maximum compensation value MaxComp_adapt. These values are converted into physically meaningful parameter values via output mapping, and the converted results are referred to as the predicted parameter value Y_pred. The output mapping uses the same conversion relationship in both the training and runtime phases. The multiple hidden layers are connected sequentially, for example, a first hidden layer, a second hidden layer, and a third hidden layer. The number of nodes in each hidden layer can be selected according to the requirements of accuracy and computational complexity. The predicted parameter value Y_pred has the same dimension as the teacher label, and each dimension corresponds sequentially to the same adaptive control parameter. Therefore, the teacher label can be directly used as a supervision target. In one example, the input layer of the multilayer perceptron model is seven-dimensional, the number of nodes in the first hidden layer is 16 or 32, the number of nodes in the second hidden layer is 32 or 64, the number of nodes in the third hidden layer is 16 or 32, and the output layer is six-dimensional. The number of nodes can be selected according to the computing resources configured in the gimbal control system and the required scheduling accuracy.
[0040] The loss function used in supervised training includes the mean squared error between the predicted parameter value Y_pred and the teacher label. The mean squared error is the average of the differences between the predicted parameter value Y_pred and the teacher label in six dimensions.
[0041] In one example, the loss function used for supervised training can be expressed as follows: L_supervised=MSE(Y_pred, Y_teacher), where L_supervised is the supervised loss, MSE represents the mean squared error calculation, Y_pred is the predicted parameter value, and Y_teacher is the teacher label.
[0042] In another example, the loss function can also be expressed as: L = L_supervised + λ1·L_smooth + λ2·L_limit, where L is the total loss, L_supervised is the supervised loss, L_smooth is the smoothing loss, L_limit is the range limiting loss, and λ1 and λ2 are non-negative weight coefficients. λ1 is used to adjust the weight of the smoothing loss L_smooth in the total loss, and λ2 is used to adjust the weight of the range limiting loss L_limit in the total loss. λ1 and λ2 can be set according to the degree of jump in the model output, the strength of the safety range constraint, the model convergence result, or the validation result in the training data. In one embodiment, the smoothing loss L_smooth can be determined based on the difference in the predicted parameter values between adjacent control cycles.
[0043] In another embodiment, the loss function may further include a smoothing loss corresponding to the output jumps of the multilayer perceptron model in the time series, or a range-limiting loss corresponding to the output of the multilayer perceptron model relative to a preset safety control range. Either of these two options may be used, or they may be used together. During training, the weights of the multilayer perceptron model are updated according to the gradient of the loss function until the loss function converges or reaches a preset number of training rounds.
[0044] In one example, the training data is arranged such that common state feature vectors within the same control cycle are paired with the corresponding teacher labels for that control cycle, while samples from adjacent control cycles are organized into a sequence according to time order, facilitating the calculation of the smoothing loss. In another example, the hidden layers of the multilayer perceptron model employ nonlinear activation functions, while the output layer uses linear activation functions, ensuring that the output layer values cover the numerical range corresponding to each adaptive control parameter. The trained multilayer perceptron model is then used as a learning-based parameter scheduling model in subsequent runtime phases to output the adaptive control parameters.
[0045] Step S6: Perform semi-supervised training using unlabeled data.
[0046] In this step, semi-supervised training is further performed when there is a large amount of unlabeled control data. The portion of the common state feature vector that has been labeled by the teacher is treated as labeled data, hereinafter referred to as X_labeled; the remaining portion is treated as unlabeled data, hereinafter referred to as X_unlabeled. The unlabeled data may include at least one of the following: actual flight operation records, heading lock records, telephoto shooting records, records of high motor position sensing noise, aircraft rotation records, simulated control data, and synthetic control data.
[0047] The loss function used in semi-supervised training includes the sum of the products of supervised loss, consistency loss, smoothing loss, and safety loss with their respective weight coefficients.
[0048] In one example, the total loss used in semi-supervised training can be expressed as: L = L_supervised + λ10·L_consistency + λ20·L_smooth + λ30·L_safety, where L is the total loss, L_supervised is the supervised loss, L_consistency is the consistency loss, L_smooth is the smoothing loss, L_safety is the safety loss, and λ10, λ20, and λ30 are non-negative weight coefficients. λ10 is used to adjust the weight of the consistency loss L_consistency in the total loss, λ20 is used to adjust the weight of the smoothing loss L_smooth in the total loss, and λ30 is used to adjust the weight of the safety loss L_safety in the total loss. λ10, λ20, and λ30 can be set according to the proportion of unlabeled data, output smoothing requirements, the strength of safety range constraints, and model validation results. In one embodiment, the consistency loss L_consistency can be determined based on the model output difference between the unlabeled state feature vector and its augmented state feature vector.
[0049] The smoothing loss L_smooth can be determined based on the difference in model output between adjacent control periods in the time series; the safety loss L_safety can be determined based on the degree to which the model output exceeds the preset safety control range.
[0050] The supervised loss is the mean squared error between the predicted parameter value Y_pred and the teacher label; the consistency loss is used to ensure that similar inputs or perturbed inputs produce similar outputs; the smoothing loss is used to limit the output jumps of the multilayer perceptron model over time; and the safety loss is used to ensure that the output of the multilayer perceptron model falls within a preset safety control range. Through these losses, unlabeled data can participate in weight updates, and the model's adaptability to situations such as telephoto shooting, camera rotation, and high sensor noise is enhanced.
[0051] In one example, the consistency loss is determined by the output difference between the common state feature vector within the same control cycle and its perturbed version, whereby the perturbation can be a small offset of the feature quantity. In another example, the security loss only produces a non-zero value when the output of the multilayer perceptron model exceeds a preset security control range, thereby guiding the parameter values back within the security control range.
[0052] Step S7: Perform validation on the trained model.
[0053] In this step, the trained multilayer perceptron model is validated. The validation includes at least one of the following: the low-frequency drift suppression capability of multiple gimbal models in heading lock mode; whether the drift estimation value (drift_est) is correctly frozen during body rotation; whether the freeze timer (freeze_timer) is adjusted according to the operating state; whether the angle control error after limiting in telephoto mode is overcompensated; whether the misjudgment when the motor position sensing noise is too high is reduced; whether the output of the multilayer perceptron model falls within the preset safety control range; and whether the output of adjacent control cycles is smooth.
[0054] If the above verification fails, additional control data records can be added, fuzzy rules can be adjusted, the weight coefficients of the loss function can be adjusted, or the multilayer perceptron model can be retrained. If the verification passes, subsequent processing can proceed. In one example, the verification is performed on multiple gimbal models to confirm that the same set of common state feature definitions can support parameter scheduling output on multiple models. If the verification result of a certain model does not meet the requirements, additional control data records are added for that model, and the model is retrained.
[0055] Step S8: Quantization, compression, and versioning.
[0056] In this step, at least one of quantization, pruning, compression, or format conversion is performed on the validated multilayer perceptron model to make it suitable for embedded deployment. Multiple model versions are generated based on the multiple gimbal models, each assigned a corresponding version identifier. These multiple model versions may include a base version trained using data from multiple models, and model versions retrained or fine-tuned for specific models. The differences between versions lie in their weight values, while their input and output layer definitions remain consistent, allowing them to be driven by the same set of real-time state features. In one example, the multiple model versions may include a base model trained using data from multiple models, a model version corresponding to a small gimbal (Model_A_small_gimbal), a model version corresponding to a medium gimbal (Model_B_medium_gimbal), a model version corresponding to a long-focus optical pod (Model_C_tele_pod), and a model version corresponding to a model with high motor position sensing noise levels (Model_D_high_jitter).
[0057] In one example, the model version is selected by pre-setting or burning the corresponding model version to the gimbal control system at the factory, so that the gimbal does not need to select a model based on the model identification information during operation. In another example, the gimbal control system can also store a basic version and multiple model versions, and select one of them based on the configuration items during power-on self-test.
[0058] The above describes a preferred embodiment of the training phase in the training method of the parameter scheduling model described in this invention. After the trained multilayer perceptron model has undergone at least one of the following processes: quantization, pruning, compression, or format conversion, it is deployed to the embedded processor of the gimbal control system and is run when the gimbal enters the heading lock mode.
[0059] The inputs received and adaptive control parameters output by the multilayer perceptron model during the runtime phase correspond to the common state feature vector and teacher labels used during the training phase. The composition and order of their features remain consistent, ensuring that the parameter scheduling relationship established during training can be continued during runtime. Furthermore, since the common state feature vector does not contain model identification information for the multiple gimbal models, the trained model can be driven by the same set of real-time state features without requiring model identification information during runtime, allowing the same trained model to be applied to the multiple gimbal models.
[0060] Please continue to refer to this. Figure 2 As shown, a preferred implementation of the running phase in the training method of the parameter scheduling model of the present invention includes the following steps.
[0061] Step S9: Enter heading lock mode and perform initialization.
[0062] In this step, when the user or the upper-level controller puts the gimbal into heading lock mode, the gimbal control system performs initialization. This initialization includes setting the current first heading angle signal θ_q as the heading lock reference angle θ_lock, setting the current first heading angle signal θ_q as the first drift estimation reference θ_q,ref, setting the current second heading angle signal θ_m as the second drift estimation reference θ_m,ref, clearing the drift estimation value drift_est to zero, and clearing the freeze timer freeze_timer to zero. The initialization also includes setting the current second heading angle signal θ_m as the starting reference angle θ_m,rate_ref.
[0063] In the case of propulsion control or external control commands, the initialization further includes setting the heading angle component θ_push corresponding to the current propulsion control or external control command to the corresponding locking reference angle θ_push,lock. That is, the initialization is performed according to the following formula: θ_lock=θ_q, θ_q,ref=θ_q, θ_m,ref=θ_m, drift_est=0, freeze_timer=0. In the case of propulsion control or external control commands, the initialization can also be obtained according to the following formula: θ_push,lock=θ_push.
[0064] In one example, the initialization is performed the moment the gimbal switches from cruise mode to heading lock mode. Its purpose is to start the drift observation residual disc from zero and to make the heading lock error e_q based on the attitude at the moment of entering heading lock mode.
[0065] Step S10: Acquire the sensing signal and perform filtering in each control cycle.
[0066] In this step, during each control cycle, the gimbal control system acquires the first heading angle signal θ_q, the raw value of the second heading angle signal θ_m,raw, the heading axis angular velocity ω_yaw, and the zoom parameter zoom. In the case of push control or external control commands, it also acquires the heading angle component θ_push corresponding to the push control or external control command. Specifically, the first heading angle signal θ_q can be obtained from the signal of the inertial sensing unit through quaternion attitude calculation, Euler angle calculation, or other attitude estimation methods; the raw value of the second heading angle signal θ_m,raw can be obtained from the motor position sensing unit; the heading axis angular velocity ω_yaw can be obtained from a gyroscope, attitude estimator, or heading axis angular velocity estimator; the zoom parameter zoom can be obtained from the lens focal length, zoom ratio, or lens control position; and in the case of push control or external control commands, the heading angle component θ_push can be obtained from the push control or external control commands. To reduce motor position sensing noise, the control system can perform low-pass filtering on the raw value θ_m,raw of the second heading angle signal to obtain the second heading angle signal θ_m, where θ_m=LPF(θ_m_raw), and LPF represents the low-pass filtering operation.
[0067] In one example, the low-pass filter is a first-order low-pass filter, and its filter coefficients are selected based on the resolution and noise level of the motor position sensing unit. In another example, the cutoff frequency of the low-pass filter is set below the control frequency, thereby suppressing position sensing noise while preserving the true rotation information of the machine body.
[0068] Step S11: Calculate the heading lock error and drift observation residual.
[0069] In this step, the heading lock error e_q is equal to the difference between the first heading angle signal θ_q and the heading lock reference angle θ_lock. In the case of push control or external control commands, the heading lock error e_q may also include the difference between the heading angle component θ_push corresponding to the push control or external control command and its lock reference angle θ_push,lock. The drift observation residual disc is equal to the difference between the first movement Δq and the second movement Δm. The first movement Δq is the movement of the first heading angle signal θ_q relative to the first drift estimation reference θ_q,ref, and the second movement Δm is the movement of the second heading angle signal θ_m relative to the second drift estimation reference θ_m,ref. That is, the heading lock error e_q and the drift observation residual disc are obtained according to the following formula: e_q = θ_q θ_lock, disc=Δq Δm=(θ_q θ_q,ref) (θ_m θ_m,ref). In the case of propulsion control or external control commands, the heading lock error e_q can also be obtained by the following formula: e_q=(θ_q) θ_lock)+(θ_push θ_push,lock).
[0070] In one example, when the aircraft is approximately stationary, if the drift observation residual disc gradually accumulates, and neither the second heading angle change mrate nor the heading axis angular velocity ω_yaw indicates that the aircraft is rotating, it can be inferred that the accumulation of the drift observation residual disc mainly comes from the low-frequency drift component of the first heading angle signal θ_q. In this case, the drift observation residual disc is neither used to directly determine the drift of the first heading angle signal θ_q, nor is it used to replace the first heading angle signal θ_q in control; instead, it is presented as the difference in the relative degree of movement between the second heading angle signal θ_m and the first heading angle signal θ_q.
[0071] Step S12: Extract the instantaneous state feature vector.
[0072] In this step, based on the same state feature definitions as in the training phase, the drift observation residual change disc_dot, the second heading angle change mrate, and the motor position sensing noise jitter_m are calculated. The absolute values of the heading lock error e_q, drift observation residual disc, drift observation residual change disc_dot, second heading angle change mrate, and heading axis angular velocity ω_yaw are taken to form an instantaneous state feature vector, hereinafter denoted as X_runtime. The instantaneous state feature vector X_runtime is expressed as follows: X_runtime=[|e_q|, |disc|, |disc_dot|, |mrate|, |ω_yaw|, jitter_m, zoom]. This common state feature vector serves as the input to the multilayer perceptron model during the training phase, hereinafter denoted as X; the instantaneous state feature vector X_runtime serves as the inference input to the trained multilayer perceptron model during the runtime phase. The instantaneous state feature vector has the same feature definition, the same arrangement order, and the same dimensions as the common state feature vector used in the training phase, and its dimension is seven.
[0073] In one example, the instantaneous state feature vector does not include gimbal model identification information, motor torque level, position sensing resolution, load weight, proportional-integral-derivative controller gain level, control cycle setpoint, and gyroscope noise level, etc., so the model has fewer input dimensions at runtime.
[0074] Step S13: Input the instantaneous state feature vector into the trained model.
[0075] In this step, the instantaneous state feature vector is input into the trained multilayer perceptron model to obtain the model's raw output y_raw. The raw output y_raw includes six raw parameters corresponding to the drift estimation time constant τ_adapt, the rotation decision threshold R_th,adapt, the stabilization waiting time T_settle,adapt, the drift correction gain G_drift,adapt, the fusion gain G_fusion,adapt, and the maximum compensation amount MaxComp_adapt, respectively, and has a six-dimensional dimension. The result of the output mapping transformation of these six raw parameters is the predicted parameter value Y_pred, referred to in the training phase. The trained multilayer perceptron model does not output pulse width modulation signals, current commands, torque commands, or drive commands for the yaw axis motor; its output only affects the subsequent parameter scheduling stage. In this embodiment, the six raw parameters included in the raw output y_raw are raw_τ_adapt, raw_R_th,adapt, raw_T_settle,adapt, raw_G_drift,adapt, raw_G_fusion,adapt, and raw_MaxComp_adapt, respectively. They correspond to the drift estimation time constant τ_adapt, the rotation determination threshold R_th,adapt, the stabilization waiting time T_settle,adapt, the drift correction gain G_drift,adapt, the fusion gain G_fusion,adapt, and the maximum compensation amount MaxComp_adapt.
[0076] In one example, the multilayer sensor model is stored in the embedded processor of the gimbal control system and performs forward calculations according to the control cycle in the heading lock mode; in the non-heading lock mode, manual steering mode, or high-speed motion mode, the multilayer sensor model stops outputting adaptive control parameters and instead uses preset control parameters or pauses the update of drift estimation.
[0077] Step S14: Perform mapping, limiting, rate limiting and smoothing on the original output.
[0078] In this step, the raw output y_raw of the multilayer perceptron model is sequentially processed by output mapping, safety limiting, rate limiting, and parameter smoothing to obtain safe adaptive control parameters y_safe. The output mapping converts the raw parameters output by the model into physically meaningful parameter values. The safety limiting restricts each adaptive control parameter to a corresponding preset safety range. These preset safety ranges are set for the drift estimation time constant, rotation determination threshold, stabilization waiting time, drift correction gain, fusion gain, and maximum compensation amount, with an upper limit greater than a lower limit. In one example, the preset safety interval can be represented as follows: the drift estimation time constant τ_adapt is restricted to its lower limit τ_min and upper limit τ_max; the rotation determination threshold R_th,adapt is restricted to its lower limit R_min and upper limit R_max; the stabilization waiting time T_settle,adapt is restricted to its lower limit T_min and upper limit T_max; the drift correction gain G_drift,adapt is restricted to its lower limit Gd_min and upper limit Gd_max; the fusion gain G_fusion,adapt is restricted to its lower limit Gf_min and upper limit Gf_max; and the maximum compensation amount MaxComp_adapt is restricted to its lower limit C_min and upper limit C_max. The values of each of these adaptive control parameters thus fall within the closed intervals defined by their respective lower and upper limits.
[0079] The above relationships can also be expressed as: τ_adapt∈[τ_min,τ_max], R_th,adapt∈[R_min,R_max], T_settle,adapt∈[T_min,T_max], G_drift,adapt∈[Gd_min,Gd_max], G_fusion,adapt∈[Gf_min,Gf_max], MaxComp_adapt∈[C_min,C_max].
[0080] The rate limiting is used to restrict the parameter variation between adjacent control cycles to within a preset upper limit, and the parameter value it applies to is the parameter value after the safety limiting processing. In one example, the rate limiting is performed according to the following formula: |y_clamp[k] y_clamp[k 1]|<Δy_max, where Δy_max is the preset upper limit of the change range, y_clamp is the parameter after limiting, and k represents the sequence number of the control cycle. The rate limiting is used to limit the parameter change range between adjacent control cycles to within the preset upper limit of the change range, and the parameter value it acts on is the parameter value after the safety limiting processing. Specifically, the original output y_raw is mapped to the mapping parameter y_map, the mapping parameter y_map is safety-limited to the limiting parameter y_clamp, the limiting parameter y_clamp is then rate-limited to the rate limiting parameter y_rate, and the rate limiting parameter y_rate is then smoothed to obtain the safe adaptive control parameter y_safe for the current control cycle.
[0081] The parameter smoothing process weights the limited parameters and the safe adaptive parameters of the previous control cycle according to a preset time smoothing coefficient. The time smoothing coefficient represents the weight of the safe adaptive parameters of the previous control cycle, and its value is greater than 0 and less than 1. It can be set according to the specific operating conditions of the gimbal. The output mapping, the safe limiting, the rate limiting, and the parameter smoothing can be implemented using methods known in the art. The specific implementation parameters can be set according to the actual operating conditions. The above implementation methods do not change the characteristic of the multilayer perceptron model that takes the instantaneous state features as input and outputs the adaptive control parameters.
[0082] In one example, the parameter smoothing follows the formula: y_safe[k]=β·y_safe[k 1]+(1 β)·y_clamp[k].
[0083] Where y_safe[k] is the adaptive parameter for safety in the current control cycle, y_safe[k] 1] represents the safety adaptive parameter of the previous control cycle, y_clamp[k] represents the limited parameter of the current control cycle, and β represents the time smoothing coefficient. The original output y_raw is converted into a physically meaningful parameter value through output mapping. This parameter value is then processed by safety limiting and is called the limited parameter y_clamp. The rate limit is applied to the limited parameter y_clamp, which is then smoothed to become the safety adaptive control parameter y_safe of the current control cycle.
[0084] The time smoothing coefficients are set for the six adaptive control parameters respectively, and each time smoothing coefficient can be selected according to the role of the corresponding adaptive control parameter in the control loop.
[0085] When the output of the multilayer perceptron model is abnormal, its derivation fails, the instantaneous state feature vector exceeds the preset confidence range, the output change of adjacent control cycles is too large, the sensing data is abnormal, or the model version is mismatched, the gimbal control system performs rollback processing, that is, the adaptive control parameters are replaced with preset control parameters, the output of the teacher model, or the preset safety parameter table, and the update of drift estimation drift_est can be paused or the maximum output of the angle control error after amplitude limiting can be restricted.
[0086] Step S15: Execute the body action gate control according to the adaptive control parameters.
[0087] In this step, the aircraft motion is determined based on the rotation determination threshold R_th,adapt and the stabilization waiting time T_settle,adapt in the adaptive control parameters. When the absolute value of the second heading angle change mrate is greater than the rotation determination threshold R_th,adapt, or the absolute value of the heading axis angular velocity ω_yaw is greater than the angular velocity threshold ω_th, the aircraft is determined to be in a rotating state. At this time, the freeze timer freeze_timer is set to the stabilization waiting time T_settle,adapt, and the drift estimate drift_est is kept at the value of the previous control cycle, i.e., the update of the drift estimate is frozen. The freeze timer freeze_timer is decremented by one control cycle time Ts in each control cycle until it reaches zero. The angular velocity threshold ω_th can be set according to the rotation determination requirements of the aircraft.
[0088] In one example, rotation determination can be achieved by comparing two criteria in parallel. The criterion corresponding to the second heading angle change mrate is based on the gimbal's own position sensing signal, while the criterion corresponding to the heading axis angular velocity ω_yaw is based on the inertial sensing signal. The two criteria are compared with their respective thresholds. If either criterion is true, the machine is determined to be in a rotating state, thus enabling the machine to be identified in both low-speed and high-speed rotation scenarios.
[0089] Step S16: Update or freeze drift estimates.
[0090] When it is determined that the machine is not rotating and the freeze timer is zero, the drift estimate value `drift_est` is updated according to the drift estimation time constant `τ_adapt`. When it is determined that the machine is rotating or the freeze timer is not zero, the drift estimate value `drift_est` is maintained at the value of the previous control cycle. The update of the drift estimate value `drift_est` is performed according to the drift estimation update coefficient `α_adapt`, which is determined based on the drift estimation time constant `τ_adapt` and the control cycle time `Ts`.
[0091] In one example, the drift estimate drift_est is updated according to the following formula: drift_est[k] = drift_est[k] 1]+α_adapt·(disc[k] drift_est[k 1]), where drift_est[k] is the drift estimate for the current control cycle, drift_est[k] 1] represents the drift estimate of the previous control cycle, disc[k] represents the drift observation residual of the current control cycle, α_adapt represents the drift estimate update coefficient, and k represents the sequence number of the control cycle.
[0092] In one example, the drift estimation update coefficient α_adapt increases with the control cycle time Ts and decreases with the drift estimation time constant τ_adapt. That is, the drift estimation update coefficient can be obtained by the following formula: α_adapt = Ts / (τ_adapt + Ts), where Ts is the control cycle time and τ_adapt is the drift estimation time constant. In this case, when the value of the drift estimation time constant τ_adapt is small, the drift estimation measurement drift_est tracks faster; when the value of the drift estimation time constant τ_adapt is large, the drift estimation measurement drift_est is smoother and less affected by sensing noise. The drift estimation time constant τ_adapt is output by the multilayer perceptron model based on the instantaneous state characteristics, therefore, the value of the drift estimation time constant τ_adapt can be adjusted according to changes in operating conditions.
[0093] Step S17: Perform a new snapshot and baseline reset.
[0094] In this step, when the freeze timer decreases from greater than zero to zero, it is determined that the aircraft has completed a stabilization waiting time after rotation, and a new snapshot is performed. The new snapshot includes incorporating the drift estimate 'drift_est' into the heading lock reference angle 'θ_lock', making the updated heading lock reference angle 'θ_lock' equal to the sum of the original heading lock reference angle 'θ_lock' and the drift estimate 'drift_est'; resetting the first drift estimate reference 'θ_q,ref' to the current first heading angle signal 'θ_q'; resetting the second drift estimate reference 'θ_m,ref' to the current second heading angle signal 'θ_m'; resetting the starting reference angle 'θ_m,rate_ref' to the current second heading angle signal 'θ_m'; and clearing the drift estimate 'drift_est' to zero.
[0095] After the resnapshot is executed, the drift observation residual disc is recalculated from zero, and the heading lock reference angle θ_lock on which the heading lock error e_q is based has been incorporated into the previously accumulated drift estimate drift_est. The heading lock error e_q is thus subtracted from the same drift estimate drift_est. Therefore, when the drift correction gain G_drift,adapt is 1, the control error before and after the resnapshot remains continuous. Even when the drift correction gain is not equal to 1 or the limiting conditions change, the resnapshot can still reduce the sudden change in angle control error caused by the reference reset, thereby reducing the risk of screen jump.
[0096] Step S18: Generate the angle control error after amplitude limiting and output it to the heading axis controller.
[0097] In this step, a pre-limiting angle control error is generated based on the heading lock error e_q, drift estimate drift_est, drift correction gain G_drift,adapt, and fusion gain G_fusion,adapt. This pre-limiting angle control error is then limited according to the maximum compensation amount MaxComp_adapt to obtain the final-limiting angle control error FinalAngle. The pre-limiting angle control error is the product of the fusion gain G_fusion,adapt and a first difference, where the first difference is the difference between the heading lock error e_q and a second difference, and the second difference is the product of the drift correction gain G_drift,adapt and the drift estimate drift_est.
[0098] In one example, the pre-limiting angle control error and the post-limiting angle control error can be obtained by the following formula: FinalAngle=G_fusion,adapt·(e_q) G_drift,adapt·drift_est),FinalAngle=limit(FinalAngle, MaxComp_adapt, +MaxComp_adapt), where FinalAngle is the angle control error after limiting, G_fusion,adapt is the fusion gain, e_q is the heading lock error, G_drift,adapt is the drift correction gain, drift_est is the drift estimate, MaxComp_adapt is the maximum compensation amount, and limit indicates that the former is limited to the interval defined by the negative maximum compensation amount and the positive maximum compensation amount.
[0099] The angle control error FinalAngle after the amplitude limit is output to the yaw axis PID controller, which calculates the control quantity and then drives the yaw axis motor controller to perform compensation.
[0100] In one example, the control output u_yaw of the yaw axis motor consists of an angular velocity term and a control error term. The angular velocity term is the product of the angular velocity term gain G_ω and the yaw axis angular velocity ω_yaw, and the control error term is the product of the control error term gain G_e and the angle control error after limiting, FinalAngle. The control output u_yaw can be expressed as: u_yaw = G_ω·ω_yaw + G_e·FinalAngle, where u_yaw is the control output, G_ω is the angular velocity term gain, ω_yaw is the yaw axis angular velocity, G_e is the control error term gain, and FinalAngle is the angle control error after limiting.
[0101] The trained multilayer perceptron model does not output the aforementioned control output u_yaw; it only outputs the parameters required to generate the angle control error after amplitude limiting.
[0102] In other implementations, the trained multilayer perceptron model can be deployed in a microcontroller, digital signal processor, neural network processor, central processing unit, embedded control system, or gimbal master processor.
[0103] In one implementation, the multilayer sensor model is enabled only in heading lock mode; in non-heading lock mode, manual steering mode, or high-speed motion mode, the gimbal control system uses preset control parameters, fixed control rules, or pauses the update of drift estimation drift_est to reduce computational load, control power consumption, and improve the feasibility of embedded deployment.
[0104] Please continue to refer to this. Figure 3 As shown, corresponding to the training method of the gimbal heading lock control parameter scheduling model described above, the present invention also provides a training system for the gimbal heading lock control parameter scheduling model, which includes a data recording unit, a feature conversion unit, a teacher model unit, a training unit, a model verification unit, a model versioning unit, an initialization unit, a sensing acquisition unit, an error calculation unit, a feature extraction unit, a model inference unit, a parameter processing unit, an action gate control unit, a drift estimation unit, a baseline reset unit, and a control error output unit.
[0105] The data recording unit is used to acquire control data records from multiple PTZ camera models. The feature transformation unit is used to convert each control data record into the same set of state features and calculate the features included in the common state feature vector. The teacher model unit is used to generate teacher labels based on the teacher model. The training unit is used to train a multilayer perceptron model with a supervised objective and to perform semi-supervised training with unlabeled data. The multilayer perceptron model has a seven-dimensional input layer, a six-dimensional output layer, and includes multiple sequentially connected hidden layers.
[0106] The model validation unit is used to validate the trained model. The model versioning unit is used to quantize, compress, and version the trained model.
[0107] In one implementation, please refer to Figure 4 As shown, the teacher model unit includes a fuzzification module, a rule deduction module, and a defuzzification module, which are used to perform fuzzification processing, rule deduction processing, and defuzzification processing, respectively. The training unit includes a semi-supervised training unit, which is used for semi-supervised training with unlabeled data. The functions of the above units are implemented by steps S1 to S8 in the aforementioned training method embodiment, and their specific implementation methods are the same as those described above, and will not be repeated here.
[0108] The initialization unit is used to set the current first heading angle signal as the heading lock reference angle and the first drift estimation reference angle, and the current second heading angle signal as the second drift estimation reference angle and the starting reference angle when the gimbal enters the heading lock mode, and to clear the drift estimation measurement and the freeze timer. The sensing acquisition unit is used to acquire the first heading angle signal, the original value of the second heading angle signal, the heading axis angular velocity, and the zoom parameters in each control cycle, and to perform low-pass filtering on the original value of the second heading angle signal to obtain the second heading angle signal. The error calculation unit is used to calculate the heading lock error based on the difference between the first heading angle signal and the heading lock reference angle, and to calculate the drift observation residual based on the difference between the first movement and the second movement. The feature extraction unit is used to calculate the change in drift observation residual, the change in the second heading angle, and the motor position sensing noise based on the same state feature definition as in the training phase, and to form an instantaneous state feature vector by taking the absolute value of each feature quantity. The model inference unit is used to input the instantaneous state feature vector into the trained multilayer perceptron model and obtain the raw output of the model. The raw output includes six raw parameters corresponding to the drift estimation time constant, rotation determination threshold, stabilization waiting time, drift correction gain, fusion gain, and maximum compensation amount, respectively. The parameter processing unit is used to sequentially perform output mapping, safety limiting, rate limiting, and parameter smoothing on the raw output to obtain safe adaptive control parameters for the current control cycle. The parameter processing unit is also used to revert the adaptive control parameters to preset control parameters, the output of the teacher model, or a preset safety parameter table when the output of the multilayer perceptron model is abnormal, the inference fails, the instantaneous state feature vector exceeds the preset confidence range, or the model version is mismatched. The action gate control unit is used to determine whether the machine is in a rotating state based on the rotation determination threshold and the stabilization waiting time, and when it is determined that the machine is in a rotating state, it sets the freeze timer to the stabilization waiting time. The freeze timer decrements in each control cycle until it reaches zero. The drift estimation unit updates the drift estimation value based on the drift estimation time constant when it determines that the aircraft is not rotating and the freeze timer is zero, and maintains the drift estimation value at the value of the previous control cycle when it determines that the aircraft is rotating or the freeze timer is not zero. The reference reset unit performs a new snapshot when the freeze timer decreases from greater than zero to zero, incorporates the drift estimation value into the heading lock reference angle, resets the first drift estimation reference, the second drift estimation reference, and the initial reference angle to the current heading angle signal, and clears the drift estimation value to zero. The control error output unit generates the pre-limiting angle control error based on the heading lock error, drift estimation value, drift correction gain, and fusion gain, limits it according to the maximum compensation amount to obtain the post-limiting angle control error, and then outputs the post-limiting angle control error to the heading axis controller.The functions of each of the above units are implemented by the methods and steps of the aforementioned operation phase. The specific implementation methods are the same as those described above, and will not be repeated here.
[0109] In one implementation, the multilayer perceptron model is enabled only in heading lock mode; in non-heading lock mode, manual steering mode, or high-speed motion mode, the model inference unit stops outputting the adaptive control parameters and instead uses preset control parameters or pauses the update of drift estimation.
[0110] It should be noted that the above division of units is only an example, and the function of each unit can also be implemented by different units or a single unit; those skilled in the art can implement the specific functions of the above units based on the foregoing method embodiments, so they will not be described again.
[0111] The training method and system for the gimbal heading lock control parameter scheduling model described in this invention uses a teacher model to generate teacher labels based on common state feature vectors, and uses these teacher labels as the supervision target to train a multilayer perceptron model. Since the teacher model can provide corresponding parameter conclusions for any state feature, training labels can be obtained without manually annotating each control data item. Furthermore, because the training target is the scheduling result of adaptive control parameters rather than the driving command of the heading axis motor, the trained model still performs closed-loop control through the heading axis PID controller and the heading axis motor controller during operation. Therefore, while introducing adaptive parameter scheduling capabilities, it avoids the abnormal output of the learning model directly affecting the actuator.
[0112] Meanwhile, this invention converts control data from multiple gimbal models under different loads, focal lengths, and sensing noise conditions into a common state feature vector that does not contain model identification information, allowing control data from different models to be represented within the same feature space. Since only the instantaneous state feature vector needs to be input during operation, and model identification information is not required, the same trained model can be applied to multiple gimbal models, and model versions can be generated for each model. This reduces the workload required to redesign scheduling rules or readjust control parameters for each model.
[0113] Furthermore, this invention can further perform semi-supervised training using unlabeled data without teacher labels, and update the weights of the multilayer perceptron model while considering consistency loss, smoothing loss, and safety loss. Since the unlabeled data can cover real-world operation records under conditions such as telephoto shooting, camera rotation, and high sensing noise, the trained model's adaptability under these conditions is improved. Because the smoothing loss and safety loss respectively limit the model's output jumps and safe output range in the time series, and the original parameters of the model output are sequentially processed by output mapping, safety limiting, rate limiting, and parameter smoothing, the output of the trained model is constrained within the stable range required for control. Moreover, since the input of the trained model is a seven-dimensional instantaneous state feature, its scale is relatively small, making it suitable for deployment in embedded processors of gimbal control systems.
[0114] The above embodiments merely illustrate preferred embodiments of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention should be determined by the appended claims.
Claims
1. A training method for a gimbal heading lock control parameter scheduling model, characterized in that, include: The control data records of multiple gimbal models are obtained. The control data records are the operational data generated and recorded when the gimbal is running in heading lock mode. Each control data record is parsed and aligned to the same control cycle sequence to convert each control data record into the same set of state features, resulting in a common state feature vector. The common state feature vector includes the absolute value of the heading lock error, the absolute value of the drift observation residual, the absolute value of the change in the drift observation residual, the absolute value of the change in the second heading angle, the absolute value of the heading axis angular velocity, the motor position sensing noise, and the zoom parameters. The common state feature vector does not contain the model identification information of the multiple gimbal models. Teacher labels are generated based on the common state feature vector using the teacher model. The teacher labels include drift estimation time constant, rotation decision threshold, stabilization waiting time, drift correction gain, fusion gain, and maximum compensation amount. Using the common state feature vector as input and the teacher label as the supervision target, a multilayer perceptron model is trained. The output layer of the multilayer perceptron model is six-dimensional, and the six dimensions correspond sequentially to the drift estimation time constant, the rotation determination threshold, the stabilization waiting time, the drift correction gain, the fusion gain, and the maximum compensation amount.
2. The training method for the gimbal heading lock control parameter scheduling model according to claim 1, wherein, The plurality of gimbal models differ in at least one of the following: motor torque, motor position sensing resolution, load inertia, convergence characteristics of the proportional-integral-derivative controller, focal length range, motor position sensing noise level, and inertial sensing noise level. The control data recording is obtained in at least one of the following ways: performing a live hover test on at least one of the plurality of gimbal models and recording its control data; performing a long-term heading lock test on at least one of the plurality of gimbal models and recording its control data; performing a test on at least one of the plurality of gimbal models in telephoto shooting mode and recording its control data; performing a slow rotation test and a fast rotation test on at least one of the plurality of gimbal models respectively and recording its control data; performing a test on at least one of the plurality of gimbal models in a state where the motor position sensing noise is high and recording its control data; and obtaining analog control data generated by performing analog control on the plurality of gimbal models and synthetic control data generated based on synthetic drift and synthetic noise. The control data record also includes the operation process of any one of the multiple gimbal models under different load inertia conditions or different motor response conditions.
3. The training method for the gimbal heading lock control parameter scheduling model according to claim 1, wherein, Parsing and aligning the control data records to the same control cycle sequence includes: identifying the first heading angle signal, the second heading angle signal, the heading axis angular velocity, the zoom parameters, and the data columns corresponding to each reference quantity from each of the control data records; Align the data columns to the same control cycle sequence based on the time stamp; The training method of the gimbal heading lock control parameter scheduling model also includes processing the missing values in each of the control data records according to the predetermined interpolation rules or discard rules, and converting each feature quantity to the same dimension according to the same conversion relationship, which is set to be the same in the training phase and the running phase.
4. The training method for the gimbal heading lock control parameter scheduling model according to claim 1, wherein, The heading lock error is determined based on the difference between the first heading angle signal and the heading lock reference angle; The drift observation residual is determined based on the difference between the first drift amount and the second drift amount, where the first drift amount is the drift of the first heading angle signal relative to the first drift estimation reference, and the second drift amount is the drift of the second heading angle signal relative to the second drift estimation reference. The change in the drift observation residual is the difference between the drift observation residual in the current control period and the drift observation residual in the previous control period. The second heading angle change is the change in the second heading angle signal of the current control cycle relative to the starting reference angle, where the starting reference angle is the second heading angle signal of the previous control cycle. The noise level of the motor position sensing is determined based on the high-frequency component of the second heading angle signal, which is obtained by the difference between the second heading angle signal and its low-pass filter result. The zoom parameters are determined by at least one of the lens focal length, zoom ratio, and lens control position.
5. The training method for the gimbal heading lock control parameter scheduling model according to claim 1, wherein, The teacher model is configured to generate the teacher labels based on fuzzy rules, and to perform fuzzification, rule deduction, and defuzzification. The fuzzification process converts each feature quantity included in the common state feature vector into a corresponding semantic level. The rule deduction process generates a control conclusion based on the semantic level and the fuzzy rules. The defuzzification process converts the control conclusion into the teacher labels. The semantic levels are set according to the feature quantities, wherein the absolute values of the heading lock error and the drift observation residual are classified as near zero, small, medium and large; the absolute values of the change in the drift observation residual are classified as slow, medium and fast; the absolute values of the change in the second heading angle are classified as stationary, slow, medium and fast; the absolute values of the heading axis angular velocity are classified as low, medium and high; the motor position sensing noise is classified as low, medium and high; and the zoom parameters are classified as wide-angle, normal and telephoto.
6. The training method for the gimbal heading lock control parameter scheduling model according to claim 1, wherein, The loss function used to train the multilayer perceptron model includes the mean square error between the predicted parameter values of the multilayer perceptron model and the teacher label, wherein the predicted parameter values are the result obtained by output mapping of the output of the multilayer perceptron model. The loss function also includes at least one of the following: the smoothing loss corresponding to the output jump of the multilayer perceptron model in the time series, and the range limitation loss corresponding to the output of the multilayer perceptron model relative to the preset safety control range. Alternatively, training the multilayer perceptron model may involve semi-supervised training using unlabeled data without teacher labels. The loss function used in the semi-supervised training includes the sum of the products of supervised loss, consistency loss, smoothing loss, and safety loss with their respective weight coefficients. The consistency loss is used to ensure that similar inputs or perturbed inputs produce similar outputs. The smoothing loss is used to limit the output jumps of the multilayer perceptron model over time. The safety loss is used to ensure that the output of the multilayer perceptron model falls within a preset safety control range.
7. The training method for the gimbal heading lock control parameter scheduling model according to claim 1, wherein, After training the multilayer sensor model, the training process includes verifying the trained multilayer sensor model. The verification is performed on the multiple gimbal models respectively, and the verification includes at least one of the following items: whether the low-frequency drift suppression capability of the multiple gimbal models in heading lock mode meets the preset requirements; whether the drift estimation is frozen when the body rotates; whether the freeze timer is adjusted according to the operating state; whether the angle control error after limiting in telephoto mode is overcompensated; whether the misjudgment when the motor position sensing noise is too high is reduced; whether the output of the multilayer sensor model falls within the preset safety control range; and whether the output of adjacent control cycles is smooth. If the verification fails, supplement the control data records, adjust the fuzzy rules on which the teacher labels are generated, adjust the weight coefficients of the loss function used to train the multilayer perceptron model, or retrain the multilayer perceptron model.
8. The training method for the gimbal heading lock control parameter scheduling model according to claim 7, wherein, It also includes performing at least one of quantization, pruning, compression, and format conversion on the verified multilayer perceptron model; Furthermore, multiple model versions are generated based on the multiple gimbal models, each assigned a version identifier. These multiple model versions include a base version jointly trained using control data records from the multiple gimbal models, and model versions retrained or fine-tuned for specific models. The differences between the model versions are reflected in their weight values, while the input and output layers of each model version are defined in the same way.
9. The training method for the gimbal heading lock control parameter scheduling model according to claim 8, wherein, The multiple model versions are configured to be either pre-set or burned into the gimbal control system at the factory, so that the gimbal does not need to select a model version based on the model identification information during operation; or stored in the gimbal control system and selected according to the configuration items during power-on self-test.
10. A training system for a gimbal heading lock control parameter scheduling model, characterized in that, include: The data recording unit is used to acquire control data records of multiple gimbal models. The control data records are the operational data generated and recorded when the gimbal is running in heading lock mode. The feature conversion unit is used to parse and align each of the control data records to the same control cycle sequence, so as to convert each of the control data records into the same set of state features and calculate the feature quantities included in the common state feature vector. The common state feature vector includes the absolute value of the heading lock error, the absolute value of the drift observation residual, the absolute value of the change in the drift observation residual, the absolute value of the change in the second heading angle, the absolute value of the heading axis angular velocity, the motor position sensing noise, and the zoom parameters. The common state feature vector does not contain the model identification information of the multiple gimbal models. The teacher model unit is used to generate teacher labels based on the teacher model. The teacher labels include drift estimation time constant, rotation determination threshold, stabilization waiting time, drift correction gain, fusion gain, and maximum compensation amount. The training unit is used to train a multilayer perceptron model with the common state feature vector as input and the teacher label as the supervision target. The multilayer perceptron model has a seven-dimensional input layer, a six-dimensional output layer, and includes multiple hidden layers connected in sequence. A model verification unit is used to perform verification on the trained multilayer perceptron model; And a model versioning unit, used to perform at least one of quantization, pruning, compression and format conversion on the verified multilayer perceptron model, and to generate multiple model versions respectively assigned version identifiers according to the multiple gimbal models.
