Visual pose sequence optimization processing method based on data driving

Abstract

The invention relates to the technical field of computer vision and electric digital data processing, and discloses a visual pose sequence optimization processing method based on data driving. In order to solve the problems of instruction oscillation and slow convergence caused by the fact that visual observation data of a controlled object is easily interfered by high-frequency noise under a complex working condition, the method comprises the following steps: firstly, extracting a noisy original target feature point coordinate; fusing system prior prediction and a Kalman dynamic gain matrix to carry out denoising so as to output a net visual error; according to the net visual error and the change rate thereof, adaptively adjusting the length of a historical data interception window and constructing a local data subset; constructing a least square model with numerical regularization constraint based on the data subset, and solving a current optimal pose adjustment instruction; and finally, converting the instruction into a digital driving parameter to guide the closed-loop action of the terminal. The method effectively filters observation noise, gets rid of dependence on an accurate system mapping model, and improves convergence stability in a complex environment.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and electronic digital data processing technology, specifically to a data-driven visual pose sequence optimization method. Background Technology

[0002] In computer vision-based target tracking and spatial alignment systems, adaptive pose state command calculation based on real-time extracted target feature point feedback is the core of the visual temporal data processing stage. Traditional visual alignment algorithms heavily rely on accurate analytical mapping models to establish the analytical relationship between two-dimensional pixel errors and multi-dimensional spatial motion states. However, in complex real-world applications, the depth information of the target is often unknown, and the camera's intrinsic and extrinsic parameters are easily affected by nonlinear factors such as complex lighting, dust occlusion, and high-frequency external vibrations. This leads to a significant decrease in the accuracy of theoretically constructed or online estimated analytical mapping matrices, making the model prone to failure.

[0003] To reduce the system's reliance on precise analytical identification models, the industry is gradually adopting time-series processing and optimization methods based on pure data-driven approaches. These methods utilize historical state adjustment commands and visual error increment data accumulated over recent system cycles to directly calculate and generate the optimal control law for the current moment. However, such sequence algorithms, which directly rely on real-time data acquisition, have certain computational and processing limitations in practical applications. The data input is susceptible to high-frequency observation noise, leading to oscillations in the output command sequence. Under complex conditions, optical reflections from the target surface and external dynamic interference can introduce a large amount of high-frequency observation noise into the extracted feature point pixel coordinates. If the algorithm directly performs differential processing on the noisy coordinate data of adjacent image frames to obtain error increments, it will greatly amplify the high-frequency noise signal, causing severe fluctuations in the visual error sequence input to the algorithm's core. State data that has not been time-filtered and smoothed will cause frequent jumps in the underlying digital adjustment commands output by the system's optimization calculation. This not only severely reduces the data output accuracy of the algorithm model but also leads to high-frequency oscillations and physical losses in downstream controlled hardware. Secondly, the fixed data window truncation mechanism is rigid and cannot adapt to nonlinear dynamic convergence rhythms. Approaching a target feature point is a spatially nonlinear and rapidly changing approximation process. During the large-scale movement phase, the visual projection distortion is significant. If the algorithm introduces too early historical data sequences, it can lead to numerical misleading and affect the generation of the current optimal solution. However, during the convergence phase with small errors at close range, a longer historical data sequence is needed to smooth out local disturbances. Existing data-driven processing methods generally use a fixed-length historical data truncation window, which cannot adapt to the dynamic rhythm of error changes. This may lead to data overshoot or reduced local convergence speed during the numerical optimization process. Summary of the Invention

[0004] This invention proposes a data-driven visual pose sequence optimization method. Addressing the issue that raw visual observation data is easily affected by high-frequency noise in complex operating conditions such as high-frequency vibration interference and nonlinear environmental changes, leading to oscillations and slow convergence speeds in the state adjustment commands output by conventional algorithms, this invention aims to provide a closed-loop optimization method for visual data. First, the coordinates of noisy raw target feature points are extracted. Then, the system's prior predictions and the Kalman dynamic gain matrix are fused for denoising, outputting the net visual error, providing a reliable data foundation for subsequent optimization. Based on the net visual error and its rate of change, the length of the historical data window is adaptively adjusted, and a local data subset is constructed. A least-squares model with numerical regularization constraints is constructed based on this data subset to obtain the optimal pose adjustment command at the current moment. Finally, the command is converted into underlying digital driving parameters to guide the closed-loop action of the controlled terminal. To achieve the above objectives, the following steps are followed:

[0005] Step 1: Initialize the system and set the desired feature coordinates of the target feature points in the image plane and the discrete control period. Continuously acquire target image data frames through the image acquisition device, and use image processing algorithms to extract the noisy original observation coordinates of the target feature points.

[0006] Step 2: Construct a state vector containing pixel position and velocity dimensions, and obtain the prior prediction state at the current moment based on the system prior mapping relationship. Use a dynamic filtering gain matrix to weight and fuse the noisy original observation coordinates and prior prediction values ​​to achieve optimal state correction and update the error covariance matrix. Finally, generate the denoised net visual error data.

[0007] Step 2.1: Construct a four-dimensional state vector containing the two-dimensional pixel positions and two-dimensional pixel motion velocities of the feature points. Substitute the pose adjustment command vector from the previous time step into the discrete-time state transition equation to obtain the prior predicted state at the current time step. The expressions involved are as follows:

[0008] ,

[0009] in For a moment The four-dimensional state vector, , These represent the horizontal and vertical pixel coordinates of the feature point in the image plane, respectively. , These represent the horizontal and vertical pixel movement velocities of the feature points within the image plane, respectively.

[0010] ,

[0011] in Here is the system state transition matrix. For discrete control cycles;

[0012] ,

[0013] in To control the input matrix, The initial value of the Jacobian matrix, which is the initial feature mapping matrix of the system, is roughly determined.

[0014] ,

[0015] for The prior predicted state estimate at time t. For the previous moment Complete the corrected optimal posterior state estimate. For the previous moment The output pose adjustment command vector;

[0016] Step 2.2: Based on the prior predicted state, sequentially calculate the prior error covariance matrix, Kalman gain matrix, state update, and posterior error covariance matrix update, fusing the noisy original observation coordinates into the state estimation sequence. The expressions involved are as follows:

[0017] ,

[0018] in For a moment The resulting prior error covariance matrix, For the previous moment The posterior error covariance matrix, The system process noise covariance matrix;

[0019] ,

[0020] in For a moment The dynamic filter gain matrix, To observe the noise covariance matrix, The observation matrix;

[0021] ,

[0022] in For a moment The output is the purified state estimate after weighted correction. For the current moment Original target feature observation coordinates;

[0023] ,

[0024] in An identity matrix adapted to the dimension of the state vector;

[0025] Step 2.3: Obtain the net visual error after denoising based on the purified state estimate, involving the following expressions:

[0026] ,

[0027] in For a moment Net visual error, The desired feature coordinates for the defined image plane.

[0028] Step 3: Store the net visual error obtained in the current sampling period and the corresponding pose adjustment command vector into the historical data buffer. Adaptively adjust the length of the historical data window used for data optimization according to the real-time net visual error and the rate of change of net visual error. Extract the corresponding net error increment sequence and historical pose adjustment command sequence from the historical data buffer to construct a local data subset.

[0029] Step 3.1: In the first... In each control cycle, the net visual error increment vector and the pose adjustment command vector are stored in the historical data cache. The relevant expressions are as follows:

[0030] ,

[0031] in For a moment The net visual error increment vector, For the previous moment Net visual error;

[0032] ,

[0033] in For a moment A historical cache queue maintained in memory to store net visual error increments;

[0034] ,

[0035] in For a moment The historical cache queue maintained in memory is used to store pose adjustment instruction vectors. For the previous moment The resulting pose adjustment command vector;

[0036] Step 3.2: Obtain the output composite error index based on the Euclidean norm and the rate of change norm of the net visual error at the current moment. Substitute this composite error index into the nonlinear exponential decay function to obtain the dynamic window length for output control of subsequent data truncation. The relevant expressions are as follows:

[0037] ,

[0038] in It is a dimensionless composite error state index. For the current moment The magnitude of the net visual error vector, For the current moment The magnitude of the rate of change of net visual error The initial static error magnitude reference constant set at the beginning of system operation. This is the normalized reference constant for the maximum permissible rate of change of error; , These are scalar weighting coefficients;

[0039] ,

[0040] in For a moment The length of the historical data window for the current control cycle output at any time. , These are the preset minimum data window length constant and maximum data window length constant, respectively. The morphological slope parameter is used to control the smooth switching rate of the exponential function. The threshold parameter is shifted by the exponential function;

[0041] Step 3.3: Read the continuously updated historical data cache queue from sub-step 3.1 and extract the corresponding net error increment sequence and historical pose adjustment instruction sequence to construct a local data subset. The expressions involved are as follows:

[0042] ,

[0043] in For a moment Local historical net error increment matrix;

[0044] ,

[0045] in For a moment Local historical pose adjustment instruction sequence matrix.

[0046] Step 4: Construct a least squares optimization model with numerical regularization constraints, solve for the optimal combination of weight coefficient vectors and apply it to the local historical adjustment instruction matrix, synthesize and generate the optimal system state adjustment instruction at the current moment;

[0047] Step 4.1: Receive the current net visual error output from Step 2 and the local data matrix constructed in Step 3. With the optimization objective of minimizing the residual between the expected error decrease and the linear superposition of historical error responses, construct a least-squares cost function with regularization constraints. Calculate the derivative and set the gradient to zero to obtain the optimal combined weight coefficient vector. Synthesize and output the optimal pose adjustment command for the current moment. The expressions involved are as follows:

[0048] ,

[0049] in The least squares optimization objective cost function is constructed as follows: The dimension to be solved is The combined weight coefficient vector, The proportional gain scalar parameter for error convergence. To numerically calculate the regularization penalty scalar factor, To be with the square array A dimensionally compatible identity matrix;

[0050] ,

[0051] in This is the optimal combination of weight coefficient vectors. This is the transpose of the local historical net error increment matrix;

[0052] ,

[0053] in For the current moment The current optimal pose adjustment command vector generated at that time.

[0054] Step 5: Receive the current optimal pose adjustment command output in Step 4, convert it into the digital drive parameter format corresponding to the underlying actuator, and send the digital drive parameters to the external controlled terminal to guide the external controlled terminal to complete the closed-loop alignment action.

[0055] Compared with the prior art, the advantages of the present invention are as follows:

[0056] This invention constructs a purely data-driven visual pose sequence optimization method based on temporal state filtering estimation and adaptive data truncation. Its key features are: First, addressing the issue of high-frequency noise interference in visual data under real-world conditions, this invention employs prior prediction and Kalman dynamic gain for temporal filtering, effectively filtering observation noise and extracting clean net visual error, overcoming the shortcomings of existing methods that rely on ideal data. Second, based on the net visual error and its rate of change, this invention constructs an adaptive adjustment strategy for historical data truncation windows and a least-squares optimization strategy with numerical regularization constraints, eliminating the dependence on precise system mapping models and improving the system's stability and anti-interference capability under complex nonlinear environments. Attached Figure Description

[0057] Figure 1 This is an overall flowchart of an embodiment of the present invention; Detailed Implementation

[0058] To more clearly illustrate the purpose, technical solution, and advantages of this invention, the following detailed description of the invention will be provided with the aid of the accompanying drawings and specific embodiments.

[0059] Figure 1 This is a flowchart of an embodiment. This embodiment provides a data-driven visual pose sequence optimization processing method. The specific process includes: extracting the original observation coordinates of target feature points, combining system prior state prediction and dynamic Kalman filtering for noise reduction, and outputting a net visual error sequence; then, based on the current net visual error and its rate of change, adaptively adjusting the truncation window length of historical data and constructing a local historical data subset; and constructing a least squares optimization model with numerical regularization constraints based on the local data subset to obtain the optimal pose adjustment command at the current moment; finally, converting the command into low-level driving parameters to guide the closed-loop action of the controlled terminal.

[0060] A data-driven visual pose sequence optimization method includes the following steps:

[0061] Step 1: Initialize the system and set the desired feature coordinates of the target feature points in the image plane and the discrete control period. Continuously acquire target image data frames through the image acquisition device, and use image processing algorithms to extract the noisy original observation coordinates of the target feature points.

[0062] Step 2: Construct a state vector containing pixel position and velocity dimensions, and obtain the prior prediction state at the current moment based on the system prior mapping relationship. Use a dynamic filtering gain matrix to weight and fuse the noisy original observation coordinates and prior prediction values ​​to achieve optimal state correction and update the error covariance matrix. Finally, generate the denoised net visual error data.

[0063] Step 2.1: Construct a four-dimensional state vector containing the two-dimensional pixel positions and two-dimensional pixel motion velocities of the feature points. Substitute the pose adjustment command vector from the previous time step into the discrete-time state transition equation to obtain the prior predicted state at the current time step. The expressions involved are as follows:

[0064] ,

[0065] in For a moment The four-dimensional state vector, , These represent the horizontal and vertical pixel coordinates of the feature point in the image plane, respectively. , These represent the horizontal and vertical pixel movement velocities of the feature points within the image plane, respectively.

[0066] ,

[0067] in Here is the system state transition matrix. For discrete control cycles;

[0068] ,

[0069] in To control the input matrix, The initial value of the Jacobian matrix, which is the initial feature mapping matrix of the system, is roughly determined.

[0070] ,

[0071] for The prior predicted state estimate at time t. For the previous moment Complete the corrected optimal posterior state estimate. For the previous moment The output pose adjustment command vector;

[0072] Step 2.2: Based on the prior predicted state, sequentially calculate the prior error covariance matrix, Kalman gain matrix, state update, and posterior error covariance matrix update, fusing the noisy original observation coordinates into the state estimation sequence. The expressions involved are as follows:

[0073] ,

[0074] in For a moment The resulting prior error covariance matrix, For the previous moment The posterior error covariance matrix, The system process noise covariance matrix;

[0075] ,

[0076] in For a moment The dynamic filter gain matrix, To observe the noise covariance matrix, The observation matrix;

[0077] ,

[0078] in For a moment The output is the purified state estimate after weighted correction. For the current moment Original target feature observation coordinates;

[0079] ,

[0080] in An identity matrix adapted to the dimension of the state vector;

[0081] Step 2.3: Obtain the net visual error after denoising based on the purified state estimate, involving the following expressions:

[0082] ,

[0083] in For a moment Net visual error, The desired feature coordinates for the defined image plane.

[0084] Step 3: Store the net visual error obtained in the current sampling period and the corresponding pose adjustment command vector into the historical data buffer. Adaptively adjust the length of the historical data window used for data optimization according to the real-time net visual error and the rate of change of net visual error. Extract the corresponding net error increment sequence and historical pose adjustment command sequence from the historical data buffer to construct a local data subset.

[0085] Step 3.1: In the first... In each control cycle, the net visual error increment vector and the pose adjustment command vector are stored in the historical data cache. The relevant expressions are as follows:

[0086] ,

[0087] in For a moment The net visual error increment vector, For the previous moment Net visual error;

[0088] ,

[0089] in For a moment A historical cache queue maintained in memory to store the net visual error increment;

[0090] ,

[0091] in For a moment The historical cache queue maintained in memory is used to store pose adjustment instruction vectors. For the previous moment The resulting pose adjustment command vector;

[0092] Step 3.2: Obtain the output composite error index based on the Euclidean norm and the rate of change norm of the net visual error at the current moment. Substitute this composite error index into the nonlinear exponential decay function to obtain the dynamic window length for output control of subsequent data truncation. The relevant expressions are as follows:

[0093] ,

[0094] in It is a dimensionless composite error state index. For the current moment The magnitude of the net visual error vector, For the current moment The magnitude of the rate of change of net visual error The initial static error magnitude reference constant set at the beginning of system operation. This is the normalized reference constant for the maximum permissible rate of change of error; , These are scalar weighting coefficients;

[0095] ,

[0096] in For a moment The length of the historical data window for the current control cycle output at any time. , These are the preset minimum data window length constant and maximum data window length constant, respectively. The morphological slope parameter is used to control the smooth switching rate of the exponential function. The threshold parameter is shifted by the exponential function;

[0097] Step 3.3: Read the continuously updated historical data cache queue from sub-step 3.1 and extract the corresponding net error increment sequence and historical pose adjustment instruction sequence to construct a local data subset. The expressions involved are as follows:

[0098] ,

[0099] in For a moment Local historical net error increment matrix;

[0100] ,

[0101] in For a moment Local historical pose adjustment instruction sequence matrix.

[0102] Step 4: Construct a least squares optimization model with numerical regularization constraints, solve for the optimal combination of weight coefficient vectors and apply it to the local historical adjustment instruction matrix, synthesize and generate the optimal system state adjustment instruction at the current moment;

[0103] Step 4.1: Receive the current net visual error output from Step 2 and the local data matrix constructed in Step 3. With the optimization objective of minimizing the residual between the expected error decrease and the linear superposition of historical error responses, construct a least-squares cost function with regularization constraints. Calculate the derivative and set the gradient to zero to obtain the optimal combined weight coefficient vector. Synthesize and output the optimal pose adjustment command for the current moment. The expressions involved are as follows:

[0104] ,

[0105] in The least squares optimization objective cost function is constructed as follows: The dimension to be solved is The combined weight coefficient vector, The proportional gain scalar parameter for error convergence. To numerically calculate the regularization penalty scalar factor, To be with the square array A dimensionally compatible identity matrix;

[0106] ,

[0107] in This is the optimal combination of weight coefficient vectors. This is the transpose of the local historical net error increment matrix;

[0108] ,

[0109] in For the current moment The current optimal pose adjustment command vector generated at that time.

[0110] Step 5: Receive the current optimal pose adjustment command output in Step 4, convert it into the digital drive parameter format corresponding to the underlying actuator, and send the digital drive parameters to the external controlled terminal to guide the external controlled terminal to complete the closed-loop alignment action.

Claims

1. A data-driven visual pose sequence optimization method, characterized in that, Includes the following steps: Step 1: Initialize the system and set the desired feature coordinates of the target feature points in the image plane and the discrete control period. Continuously acquire target image data frames through the image acquisition device, and use image processing algorithms to extract the noisy original observation coordinates of the target feature points. Step 2: Construct a state vector containing pixel position and velocity dimensions, and obtain the prior prediction state at the current moment based on the system prior mapping relationship. Use a dynamic filtering gain matrix to weight and fuse the noisy original observation coordinates and prior prediction values ​​to achieve optimal state correction and update the error covariance matrix. Finally, generate the denoised net visual error data. Step 3: Store the net visual error obtained in the current sampling period and the corresponding pose adjustment command vector into the historical data buffer. Adaptively adjust the length of the historical data window used for data optimization according to the real-time net visual error and the rate of change of net visual error. Extract the corresponding net error increment sequence and historical pose adjustment command sequence from the historical data buffer to construct a local data subset. Step 4: Construct a least squares optimization model with numerical regularization constraints, solve for the optimal combination of weight coefficient vectors and apply it to the local historical adjustment instruction matrix, synthesize and generate the optimal system state adjustment instruction at the current moment; Step 5: Receive the current optimal pose adjustment command output in Step 4, convert it into the digital drive parameter format corresponding to the underlying actuator, and send the digital drive parameters to the external controlled terminal to guide the external controlled terminal to complete the closed-loop alignment action.

2. The data-driven visual pose sequence optimization method according to claim 1, characterized in that, As described in step 2, a state vector containing pixel position and velocity dimensions is constructed, and the prior prediction state at the current moment is obtained based on the system prior mapping relationship. The noisy original observation coordinates and prior prediction values ​​are weighted and fused through a dynamic filtering gain matrix to achieve optimal state correction and update the error covariance matrix. Finally, the denoised net visual error data is generated. Step 2.1: Construct a four-dimensional state vector containing the two-dimensional pixel positions and two-dimensional pixel motion velocities of the feature points. Substitute the pose adjustment command vector from the previous time step into the discrete-time state transition equation to obtain the prior predicted state at the current time step. The expressions involved are as follows: ， in For a moment The four-dimensional state vector, , These represent the horizontal and vertical pixel coordinates of the feature point in the image plane, respectively. , These represent the horizontal and vertical pixel movement velocities of the feature points within the image plane, respectively. ， in Here is the system state transition matrix. For discrete control cycles; ， in To control the input matrix, The initial value of the Jacobian matrix, which is the initial feature mapping matrix of the system, is roughly determined. ， for The prior predicted state estimate at time t. For the previous moment Complete the corrected optimal posterior state estimate. For the previous moment The output pose adjustment command vector; Step 2.2: Based on the prior predicted state, sequentially calculate the prior error covariance matrix, Kalman gain matrix, state update, and posterior error covariance matrix update, fusing the noisy original observation coordinates into the state estimation sequence. The expressions involved are as follows: ， in For a moment The resulting prior error covariance matrix, For the previous moment The posterior error covariance matrix, The system process noise covariance matrix; ， in For a moment The dynamic filter gain matrix, To observe the noise covariance matrix, The observation matrix; ， in For a moment The output is the purified state estimate after weighted correction. For the current moment Original target feature observation coordinates; ， in An identity matrix adapted to the dimension of the state vector; Step 2.3: Obtain the net visual error after denoising based on the purified state estimate, involving the following expressions: ， in For a moment Net visual error, The desired feature coordinates for the defined image plane.

3. The data-driven visual pose sequence optimization method according to claim 1, characterized in that, As described in step 3, the net visual error obtained in the current sampling period and the corresponding pose adjustment command vector are stored in the historical data cache. The length of the historical data window used for data optimization is adaptively adjusted according to the real-time net visual error and the rate of change of net visual error. The corresponding net error increment sequence and historical pose adjustment command sequence are extracted from the historical data cache to construct a local data subset. Step 3.1: In the first... In each control cycle, the net visual error increment vector and the pose adjustment command vector are stored in the historical data cache. The relevant expressions are as follows: ， in For a moment The net visual error increment vector, For the previous moment Net visual error; ， in For a moment A historical cache queue maintained in memory to store the net visual error increment; ， in For a moment The historical cache queue maintained in memory is used to store pose adjustment instruction vectors. For the previous moment The resulting pose adjustment command vector; Step 3.2: Obtain the output composite error index based on the Euclidean norm and the rate of change norm of the net visual error at the current moment. Substitute this composite error index into the nonlinear exponential decay function to obtain the dynamic window length for output control of subsequent data truncation. The relevant expressions are as follows: ， in It is a dimensionless composite error state index. For the current moment The magnitude of the net visual error vector, For the current moment The magnitude of the rate of change of net visual error The initial static error magnitude reference constant set at the beginning of system operation. This is the normalized reference constant for the maximum permissible rate of change of error; , These are scalar weighting coefficients; ， in For a moment The length of the historical data window for the current control cycle output at any time. , These are the preset minimum data window length constant and maximum data window length constant, respectively. The morphological slope parameter is used to control the smooth switching rate of the exponential function. The threshold parameter is shifted by the exponential function; Step 3.3: Read the continuously updated historical data cache queue from sub-step 3.1 and extract the corresponding net error increment sequence and historical pose adjustment instruction sequence to construct a local data subset. The expressions involved are as follows: ， in For a moment Local historical net error increment matrix; ， in For a moment Local historical pose adjustment instruction sequence matrix.

4. The data-driven visual pose sequence optimization method according to claim 1, characterized in that, As described in step 4, a least squares optimization model with numerical regularization constraints is constructed, the optimal combination weight coefficient vector is solved and applied to the local historical adjustment instruction matrix, and the optimal system state adjustment instruction at the current moment is synthesized and generated. Step 4.1: Receive the current net visual error output from Step 2 and the local data matrix constructed in Step 3. With the optimization objective of minimizing the residual between the expected error decrease and the linear superposition of historical error responses, construct a least-squares cost function with regularization constraints. Calculate the derivative and set the gradient to zero to obtain the optimal combined weight coefficient vector. Synthesize and output the optimal pose adjustment command for the current moment. The expressions involved are as follows: ， in The least squares optimization objective cost function is constructed as follows: The dimension to be solved is The combined weight coefficient vector, The proportional gain scalar parameter for error convergence. To numerically calculate the regularization penalty scalar factor, To be with the square array A dimensionally compatible identity matrix; ， in This is the optimal combination of weight coefficient vectors. This is the transpose of the local historical net error increment matrix; ， in For the current moment The current optimal pose adjustment command vector generated at that time.

Images