Robust adaptive filtering prediction method and device, and storage medium

Abstract

The invention belongs to the technical field of visual servo, and particularly relates to a robust adaptive filtering prediction method and device and a storage medium, and the method comprises the following steps: measuring motion state information of an inertial stabilization platform; converting the rotation angle of the inertial stabilization platform into pixel coordinates in an image plane; measuring the inherent delay time t of the visual sensor, and carrying out delay processing on the measurement data of the inertial sensor according to the t, so that the time consistency of the two data is calibrated under the same time reference; defining a target state vector as a six-dimensional vector, and establishing a system state equation and a measurement equation based on a Singer model; robust adaptive filtering processing is carried out; carrying out time delay compensation; the tracking precision and stability of the visual servo system can be improved; through cooperative work of the inertial measurement unit and the visual sensor, high-precision prediction and tracking of a non-cooperative target are realized in combination with a robust adaptive filtering algorithm and a time delay compensation technology.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual servo, and particularly relates to an application of a robust adaptive filtering prediction method with time-delay compensation and noise resistance in a visual servo system, a computer device for executing the filtering prediction method, and a non-transitory computer-readable storage medium. Background Art

[0002] A visual servo system is a system that obtains target position information through a visual sensor and controls mechanical motion accordingly to achieve precise tracking. This system has wide applications in industrial automation, robot operation, unmanned aerial vehicle navigation, and other fields. However, the visual servo system faces two major challenges in practical applications: measurement time-delay and noise interference.

[0003] The measurement time-delay mainly comes from the image acquisition and processing process of the visual sensor. Due to the large amount of image data and the need for complex feature extraction and target recognition operations, there is a certain time delay between the actual change in the target position and the system obtaining the corresponding measurement information. This time-delay will cause the system to make control decisions based on outdated information, thereby reducing the tracking accuracy and even causing system instability. Noise interference is another key issue. The measurement results of the visual sensor are easily affected by factors such as environmental light changes, image noise, and sensor accuracy limitations, resulting in inaccurate measurement data. In addition, the random motion characteristics of the target (such as random changes in acceleration) will also introduce process noise. These noise interferences will further reduce the tracking accuracy and robustness of the system.

[0004] Traditional filtering methods, such as the Kalman filtering method, have certain limitations in dealing with time-delay and noise and require prior values of the noise; specifically, the Kalman filter assumes that both the process noise and the measurement noise are Gaussian white noise and the system model is precisely known, but in practical applications, these assumptions are often difficult to meet; especially when the target motion is complex or the environmental noise is large, traditional filtering methods are difficult to maintain stable tracking performance. Summary of the Invention

[0005] In view of this, the present invention aims to provide a robust adaptive filtering prediction method that can effectively compensate for time-delay and resist noise interference to improve the tracking accuracy and stability of the visual servo system; through the collaborative work of an inertial measurement unit and a visual sensor, combined with a robust adaptive filtering algorithm and time-delay compensation technology, high-precision prediction and tracking of non-cooperative targets are achieved.

[0006] To achieve the above object, the technical solution of the present invention is realized as follows: The present invention provides a robust adaptive filtering prediction method with time-delay compensation and noise resistance, and the robust adaptive filtering prediction method with time-delay compensation and noise resistance includes the steps: S1. The inertial stabilization platform includes an inertial measurement unit, and the inertial measurement unit is used to measure the motion state information of the inertial stabilization platform; S2. Convert the rotation angle of the inertial stabilization platform into pixel coordinates in the image plane through geometric relationships; S3. Measure the inherent delay time of the vision sensor in the visual servo system △t , delay the measurement data of the inertial sensor according to △t to make the inertial data and the visual data under the same time reference, and use the timestamp alignment algorithm to calibrate the time consistency of the inertial data and the visual data; S4. Define the target state vector as a six-dimensional vector, and based on the Singer model, establish a system state equation and a measurement equation; S5. Perform robust adaptive filtering processing; S6. According to the inherent delay time of the vision sensor △t and the filtering period T s , calculate the prediction step number N = ceil ( △t / T s ) ; After obtaining the state estimation k at time , perform N step prediction to obtain the target position information after compensating for the time lag; Use a robust adaptive Kalman filter predictor for multi-step prediction, and advance the state estimation value at the current time by the corresponding time step to compensate for the time delay in the visual servo system.

[0007] Further, the motion state information includes acceleration and angular velocity; The formula for the geometric relationship is ; Wherein, θ x ,θ y are respectively the rotation angles of the inertial stabilization platform on the x-axis and y-axis of the field of view; u, v are respectively the moving pixels on the x-axis and y-axis of the field of view, λ is the camera focal length.

[0008] Further, the six-dimensional vector includes the target position, the target velocity, and the target acceleration; the target state vector is: ; Wherein, u T,v T is the target position , is the target speed, is the target acceleration; The system state equation is x T ( k + 1) = Fx T ( k ) + w k , the measurement equation is z ( k ) = Hx T ( k ) + v k ; The covariance matrix is: ; Wherein, x T ( k ) is the target state vector at time k, x T ( k + 1) is the target state vector at time k + 1, F is the state transition matrix, w k is the process noise, is the reciprocal of the maneuver time constant, σ a 2 is the maneuver acceleration variance, H is the measurement matrix, v k is the measurement noise, represents the process noise weight of the Singer model after integral discretization.

[0009] Furthermore, the iterative formula for the robust adaptive filtering process is: ; Wherein, is the Kalman gain, is the variance of the previous state estimate.

[0010] Furthermore, the robust adaptive filtering process includes the steps of: S51. Calculate the prediction residual ; Wherein, ( k + 1) is the target state vector at the next moment predicted based on the current state, z ( k ) is the sensor measurement value at the current moment; S52. Dynamically adjust the adaptation factor based on the statistic of the prediction residual α T ; S53. Calculate the diagonal weight matrix using the IGG-III weight function .

[0011] Furthermore, in the diagonal weight matrix ω ( i ) the calculation formula is: Wherein, k 0 and k 1 are constants, is the standardized residual.

[0012] Furthermore, the calculation formula of the adaptation factor α T is: ; Wherein, c is a preset threshold, represents the normalized statistic of the residual, ; is the covariance matrix of the predicted state vector, R is the measurement noise covariance matrix.

[0013] The present invention also provides a visual servo system, and the visual servo system can implement the robust adaptive filtering prediction method with time delay compensation and noise resistance described in the present invention.

[0014] The present invention also provides a computer device, including: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the robust adaptive filtering prediction method with time delay compensation and noise resistance described above in the present invention.

[0015] The present invention also provides a non-transitory computer-readable storage medium storing computer instructions, and the computer instructions are used to cause the computer to execute the robust adaptive filtering prediction method with time delay compensation and noise resistance described above in the present invention.

[0016] Compared with the prior art, the present invention can achieve the following beneficial effects: The robust adaptive filtering prediction method with time-delay compensation and noise resistance provided by the present invention effectively solves the problems of reduced tracking accuracy and system stability caused by time delay and noise in the visual servo system through time-delay compensation technology and robust adaptive filtering algorithms. By compensating for the delay of the visual sensor through a prediction algorithm, it ensures that the system makes control decisions based on real-time information, significantly improving the real-time performance and control performance of the system. At the same time, by combining the robust adaptive filtering algorithm and the IGG-III weight function, it can effectively resist the interference of measurement noise and abnormal data, enhancing the robustness and stability of the system.

[0017] In addition, by comprehensively considering the position, velocity, and acceleration of the target, the motion state of the target is comprehensively described, further improving the tracking accuracy of the visual servo system. The design of the adaptation factor is dynamically adjusted to enable the system to adapt to the maneuverability of the target and environmental changes, ensuring high-performance tracking capabilities even under complex working conditions.

[0018] The robust adaptive filtering prediction method with time-delay compensation and noise resistance provided by the present invention is applicable to visual servo systems with high-precision tracking and stable control requirements, and has important application value especially in the fields of non-cooperative target tracking and robot vision control. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a schematic diagram of the overall process of the robust adaptive filtering prediction method with time-delay compensation and noise resistance provided by the embodiment of the present invention; Figure 2 is a schematic diagram for comparing the predicted curve of the motion trajectory of a non-cooperative target in Embodiment 1 of the present invention; Figure 3 is a schematic diagram for comparing the motion tracking error of a non-cooperative target in Embodiment 2 of the present invention; Figure 4 is a schematic diagram of a computer device for executing the robust adaptive filtering prediction method with time-delay compensation and noise resistance according to the embodiment of the present invention.

[0020] Description of the reference numerals: 12. Computer device; 14. External device; 16. Processing unit; 18. Bus; 20. Network adapter; 22. I / O interface; 24. Display; 28. System memory; 30. RAM; 32. Cache; 34. Storage system; 40. Program / utilities; 42. Program module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation to the present invention.

[0022] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0023] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "plurality" is two or more.

[0024] In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the terms "installed", "connected", "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific situations.

[0025] In a specific embodiment of the present invention, a robust adaptive filtering prediction method with time-delay compensation and noise resistance is provided. The robust adaptive filtering prediction method with time-delay compensation and noise resistance includes the following steps: S1. The inertial stabilization platform includes an inertial measurement unit, and the inertial measurement unit is used to measure the motion state information of the inertial stabilization platform; the motion state information includes acceleration (angular acceleration) and angular velocity. By measuring the angular velocity and angular acceleration, the angle can be obtained, and the angle can be converted into the information of the visual image plane.

[0026] S2. Convert the angular information into image information. Specifically, convert the rotation angle of the inertial stabilization platform into pixel coordinates in the image plane through geometric relationships; by converting the angular information of the inertial sensor itself into image plane information of its own movement, and then superimposing the image plane information of the error between itself and the target (which can be measured by a vision sensor), the information of the target movement trajectory can be obtained.

[0027] Specifically, the formula for the geometric relationship is (1); Wherein, θ x ,θ y are respectively the rotation angles of the inertial stabilization platform on the x-axis and y-axis of the field of view; u, v are respectively the mobile pixels on the x-axis and y-axis of the field of view; λ is the camera focal length, that is, the camera focal length in the visual servo system.

[0028] This conversion process provides a basis for the prediction of the movement trajectory of non-cooperative targets; in specific implementation, by accurately measuring the rotation angle of the inertial stabilization platform and combining camera parameters, calculate the pixel position of the target on the image plane, thereby realizing the mapping from the physical space to the image space.

[0029] S3. Since the measurement delay of the inertial sensor can be ignored, while there is a significant delay in the visual servo system, the present invention delays the measurement data of the inertial sensor to ensure the time consistency of the two sets of data; specifically, measure the inherent delay time △t of the vision sensor in the visual servo system, and delay the measurement data of the inertial sensor according to △t so that the inertial data and the visual data are on the same time basis, and use the timestamp alignment algorithm to calibrate the time consistency of the inertial data and the visual data to ensure the accuracy of subsequent processing; S4. Define the target state vector as a six-dimensional vector, and based on the Singer model, establish the system state equation and the measurement equation. Since the target movement is unknown, describe the movement trajectory of the target through a kinematic model; the Singer model assumes that the acceleration of the target is uniformly distributed within a certain range and is suitable for describing the movement of targets with random maneuver characteristics. Specifically, the six-dimensional vector includes the target position, the target velocity, and the target acceleration; the target state vector is as shown in formula (2): (2); Wherein, u T ,v T is the target position , is the target velocity, is the target acceleration; The system state equation is as shown in formula (3): x T ( k +1)= Fx T ( k ) + w k (3); The measurement equation is as shown in formula (4): z ( k ) = Hx T ( k ) + v k (4); The covariance matrix is as shown in formula (5): (5); Wherein, x T ( k ) is the target state vector at time k, x T ( k + 1) is the target state vector at time k + 1, F is the state transition matrix, the elements of which are determined by the parameters in the Singer model and reflect the transfer law of the target motion state between different times; w k is the process noise; is the reciprocal of the maneuver time constant, characterizing the rate of change of the target acceleration; σ a 2 is the maneuver acceleration variance, reflecting the fluctuation amplitude of the target acceleration; H is the measurement matrix, and the measurement matrix H is determined according to the measurement method of the sensor and the definition of the target state vector, and is usually used to extract position information from the state vector. Here, the measurement data of the sensor is the measurement data after the data fusion of the inertial sensor data and the visual sensor data; v k is the measurement noise, covering factors such as sensor errors and environmental disturbances, represents the process noise weight value of the Singer model after integral discretization.

[0030] S5. Perform robust adaptive filtering processing; specifically, the robust adaptive filtering processing includes the steps: S51. Calculate the prediction residual according to formula (6); (6); Among them, ( k +1) is the target state vector at the next moment predicted based on the current state, z ( k ) is the sensor measurement value at the current moment, specifically the measurement value after the data fusion of the inertial sensor and the vision sensor; the magnitude of the residual directly reflects the accuracy of the prediction model and the reliability of the sensor data; S52. Dynamically adjust the adaptation factor based on the statistic of the prediction residual α T ; specifically, evaluate the model accuracy by calculating the prediction residual and dynamically adjust the adaptation factor based on the residual statistic; the adaptation factor α T is calculated according to the formula shown in formula (7): (7); Among them, c is a preset threshold, usually selected between 1 and 2.5 according to experience; represents the normalized statistic of the residual, ; among them, is the covariance matrix of the predicted state vector, R is the measurement noise covariance matrix.

[0031] S53. Calculate the diagonal weight matrix using the IGG-III weight function ; in the diagonal weight matrix ω ( i ) is calculated according to the formula shown in formula (8): (8); Among them, k 0 and k 1 are constants, generally selected in the ranges of 1.0 to 2.5 and 3.5 to 8.0 respectively, used to determine the interval of weight value change, is the standardized residual; in this way, when the residual is small, a larger weight value is given, and the measurement data is considered more reliable; when the residual is large, the weight value is gradually reduced, or even given a very small weight value to reduce the influence of abnormal data on the filtering result.

[0032] In a specific implementation manner, the complete iterative formula for the robust adaptive filtering process is shown in formula (9): (9); Among them, is the Kalman gain, is the variance of the previous state estimate.

[0033] S6. According to the inherent delay time of the visual sensor △t and the filtering period T s , the formula for calculating the prediction steps is N = ceil ( △t / T s ) (10); After obtaining the state estimation k at the moment , perform N step prediction, (11); Obtain the target position information after compensating the time lag, where F is as follows: .

[0034] Use a robust adaptive Kalman filter predictor for multi-step prediction, advancing the state estimation value at the current moment by the corresponding number of time steps to compensate for the time lag in the visual servo system; specifically, after the processing of the previous steps, six-dimensional target motion information has been obtained, but this motion information contains a delay. At this time, multi-step prediction is to predict the motion state at the current moment based on the delayed motion state; generally, the sampling period is short, assumed to be 2 milliseconds, with a delay of 100 milliseconds, and the specific number of multi-steps for multi-step prediction is 50 steps; by using a robust adaptive Kalman filter predictor for multi-step prediction, it can effectively make up for the information lag caused by the time lag and improve the real-time performance and control performance of the visual servo system.

[0035] In a specific implementation manner, in the robust adaptive filtering prediction method with time lag compensation and noise resistance provided by the present invention, the specific implementation steps of S6 are as follows: (1) Real-time monitor the delay time △t of the visual sensor, and dynamically adjust the prediction steps N according to the filtering period Ts.

[0036] (2) Use a robust adaptive Kalman filter predictor for multi-step prediction, advancing the state estimation value at the current moment by the corresponding number of time steps to obtain the target position information after compensating the time lag.

[0037] (3) Feed back the compensated target position information to the control system to ensure that the system makes control decisions based on real-time information, improving the tracking accuracy and system stability.

[0038] The robust adaptive filtering prediction method with time lag compensation and noise resistance provided by the specific implementation manner of the present invention is applicable to visual servo systems with high-precision tracking and stable control requirements, and has important application value especially in the fields of non-cooperative target tracking and robot vision control.

[0039] The specific implementation manner of the present invention also provides an application of the robust adaptive filtering prediction method with time-delay compensation and noise resistance in a visual servo system, and specifically provides a visual servo system, which can implement the robust adaptive filtering prediction method with time-delay compensation and noise resistance of the present invention.

[0040] The specific implementation manner of the present invention also provides a computer device, including: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the above-mentioned robust adaptive filtering prediction method with time-delay compensation and noise resistance of the present invention.

[0041] The specific implementation manner of the present invention also provides a non-transitory computer-readable storage medium storing computer instructions, and the computer instructions are used to cause the computer to execute the above-mentioned robust adaptive filtering prediction method with time-delay compensation and noise resistance of the present invention.

[0042] The robust adaptive filtering prediction method with time-delay compensation and noise resistance provided by the present invention effectively solves the problems of reduced tracking accuracy and system stability caused by time delay and noise in a visual servo system through time-delay compensation technology and a robust adaptive filtering algorithm; compensates for the delay of the visual sensor through a prediction algorithm to ensure that the system makes control decisions based on real-time information, significantly improving the real-time performance and control performance of the system. At the same time, combining the robust adaptive filtering algorithm and the IGG-III weight function can effectively resist the interference of measurement noise and abnormal data, enhancing the robustness and stability of the system.

[0043] In addition, by comprehensively considering the position, speed, and acceleration of the target, the motion state of the target is fully described, further improving the tracking accuracy of the visual servo system, dynamically adjusting the design of the adaptation factor, enabling the system to adapt to the maneuverability of the target and environmental changes, and ensuring high-performance tracking capabilities even under complex working conditions.

[0044] The robust adaptive filtering prediction method with time-delay compensation and noise resistance provided by the present invention is applicable to visual servo systems with high-precision tracking and stable control requirements, and has important application values especially in fields such as non-cooperative target tracking and robot vision control.

[0045] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0046] Select an inertial measurement unit with high precision and fast response speed, and tightly integrate it with the inertial stabilization platform to accurately measure the motion state information of the target object, such as acceleration, angular velocity, etc. Specifically, as Figure 1 shown, it is a schematic diagram of the overall process of the robust adaptive filtering prediction method with time delay compensation and noise resistance described in the specific embodiment of the present invention. It can be seen from the figure that the robust adaptive filtering prediction method with time delay compensation and noise resistance includes the following steps: S1. The inertial stabilization platform includes an inertial measurement unit, and the inertial measurement unit is used to measure the motion state information of the inertial stabilization platform. Specifically, the inertial measurement unit can be integrated with the inertial stabilization platform to measure the motion state information of the inertial stabilization platform. The motion state information includes acceleration and angular velocity, and the angle information is obtained through acceleration and angular velocity.

[0047] S2. Convert the angle information into image information through the following formula (1): (1); where, θ x ,θ y are respectively the rotation angles of the inertial stabilization platform on the x-axis and y-axis of the field of view; u, v are respectively the moving pixels of the x-axis and y-axis of the field of view, λ is the camera focal length; by calculating the moving pixels of the inertial stabilization platform itself, it is prepared for obtaining the motion trajectory of the non-cooperative target.

[0048] S3. Artificially delay the inertial sensor measurement data according to a fixed delay time to ensure data synchronization and time consistency between the two. Specifically, delay the inertial sensor according to a fixed delay time; the visual servo system determines the relative deviation through the visual sensor, and the extension of the image acquisition and processing time will introduce significant delay to the system. In contrast, the measurement delay of the inertial measurement unit in the inertial stabilization platform can be ignored.

[0049] S4. Define the target state vector and establish the target kinematic model; define the state vector of the moving target as a six-dimensional vector including the target position, target velocity, and target acceleration, that is (2); where, u T ,v T is the target position , is the target velocity, is the target acceleration; this definition method can comprehensively describe the motion state of the target and provide a sufficient information basis for subsequent state estimation and prediction. Specifically, the position component directly gives the specific coordinates of the target in space and is a key parameter for the servo system to perform positioning and tracking; the velocity component can help the system understand the motion trend of the target and predict its approximate position at the next moment; the acceleration component reflects the change of the target's motion state and is of great significance for dealing with complex motions such as rapid direction change, acceleration or deceleration of maneuvering targets; by comprehensively considering the motion characteristics at these three levels, the state vector can more accurately depict the dynamic behavior of the target and improve the adaptability and tracking accuracy of the system to the target's motion.

[0050] According to the Singer model, it is assumed that the acceleration of the target is uniformly distributed within a certain range. This model is suitable for describing the motion of targets with random maneuver characteristics. In practical applications, the motion of many targets is not completely uniform or uniformly accelerated, but there are certain degrees of random changes. The Singer model can better fit the motion laws of such targets and provide a reasonable theoretical basis for the establishment of the system state equation.

[0051] Based on the Singer model, establish the state equation and measurement equation of the system.

[0052] The state equation describes the evolution relationship of the target state vector over time, and its specific form is shown in formula (3): x T ( k +1)= Fx T ( k )+ w k (3); The measurement equation is shown in formula (4): z ( k )= Hx T ( k )+ v k (4); Among them, x T ( k ) is the target state vector at time k, x T ( k +1) is the target state vector at time k + 1, F is the state transition matrix, and its elements are determined by the parameters in the Singer model, reflecting the transfer law of the target motion state between different moments; is the process noise, used to simulate the random factors in the target motion, such as the random change of acceleration, etc., and its covariance matrix is shown in formula (5): (5); α is the reciprocal of the maneuvering time constant, representing the rate of change of the target acceleration; σ a 2 is the maneuvering acceleration variance, reflecting the fluctuation amplitude of the acceleration, represents the process noise weight of the Singer model after integral discretization.

[0053] The measurement equation establishes the relationship between the sensor measurement value and the true state of the target; H is the measurement matrix, whose value is determined according to the measurement method of the sensor and the definition of the target state vector, and is usually used to extract the position information corresponding to the sensor measurement from the state vector; v k is the measurement noise, covering the influence of factors such as the error of the sensor itself and environmental interference on the measurement result.

[0054] S5. Perform robust adaptive filtering processing. First, give the iterative formula for the complete robust adaptive filtering: (9); where, is the Kalman gain, is the variance of the previous state estimate.

[0055] Calculating the prediction residual is a key step in the filtering prediction process. The residual reflects the difference between the predicted value and the actual measurement value, that is (6); where, ([[]] k + 1) is the target state vector at the next moment predicted based on the current state, z ([[]] k ) is the sensor measurement value at the current moment; the magnitude of the residual directly reflects the accuracy of the prediction model and the reliability of the sensor data, providing a basis for the calculation of the adaptation factor and the robust factor.

[0056] The adaptation factor is used to adjust the data weight in the filtering process. When the statistic of the residual exceeds the preset threshold, it reduces the influence of the predicted value in the filtering and decision-making processes, enhancing the adaptability of the system to target maneuverability and environmental changes. By reasonably setting the adaptation factor, the filtering algorithm can maintain good performance under different working conditions, avoiding problems such as filtering divergence caused by too large a deviation between the model and the actual situation.

[0057] Determine the adaptation factor according to the statistic of the residual, and the specific formula is: (7); where,c is a preset threshold, usually selected between 1 and 2.5 according to experience, and is used to determine whether the residual is within an acceptable range; represents the normalized statistic of the residual. By normalizing the residual vector with the trace of its covariance matrix, the influence of dimension and magnitude is eliminated, enabling unified comparison and evaluation of residuals in different dimensions; , where is the covariance matrix of the predicted state vector, R is the measurement noise covariance matrix, which jointly determines the covariance structure of the residual.

[0058] To further improve the resistance of the filtering algorithm to measurement noise and abnormal data, the IGG-III weight function is used to calculate the diagonal weight matrix ; in the diagonal weight matrix ω ( i ) is calculated as shown in formula (8): (8); where k 0 and k 1 are constants, generally selected in the ranges of 1.0 to 2.5 and 3.5 to 8.0 respectively, and are used to determine the interval of weight value change, is the standardized residual, representing the ratio of the i th residual component to the corresponding standard deviation; in this way, when the residual is small, a larger weight value is given, indicating that the measurement data is more reliable; when the residual is large, the weight value is gradually reduced, or even given a very small weight value, to reduce the influence of abnormal data on the filtering result, thereby improving the robustness and stability of the system.

[0059] S6. Non-cooperative target position time-delay compensation processing.

[0060] In a visual servo system, the time delay mainly comes from the image acquisition and processing process of the visual sensor; due to the large amount of image data and the need for complex operations such as feature extraction and target recognition, there is a certain time delay between the actual change of the target position and the system obtaining the corresponding measurement information. This time delay will cause the system to make control decisions based on outdated information, thereby reducing the tracking accuracy and even leading to system instability.

[0061] According to the visual sensor delay time △t and the filtering period T s , the formula for calculating the prediction step is N = ceil ( △t / T s ) (10); After obtaining k the state estimation at a moment perform N a multi-step prediction, (11); Obtain the target position information with compensated time lag, where F is as follows: .

[0062] After obtaining the target motion state estimation at each time node, use a robust adaptive Kalman filter predictor to perform an additional prediction step to determine the position of the moving target, thereby compensating for the time lag in the system. Specifically, according to the estimated value of the time lag, determine the number of steps that need to be predicted, and then use the recurrence formula of the filter predictor to advance the state estimation value forward by the corresponding time steps to obtain the target position information with compensated time lag; this method can effectively compensate for the information lag caused by the time lag, enabling the system to respond more timely and accurately to the motion changes of the target, and improving the real-time performance and control performance of the visual servo system.

[0063] According to the above conditions and methods, two groups of comparative analysis and verification are respectively carried out using the robust adaptive filtering prediction method and the traditional Kalman filtering method provided in the embodiments of the present invention.

[0064] Example 1 Filter and predict the motion trajectory of a non-cooperative target through the robust adaptive filtering prediction method, and compare it with the real curve at the same time. The results are as Figure 2 shown. In the left box of the figure, the three lines from left to right represent the predicted trajectory, the real trajectory, and the measured trajectory respectively; in the right box of the figure, the three lines from top to bottom represent the predicted trajectory, the real trajectory, and the measured trajectory respectively; it can be seen from the figure that the predicted trajectory is closer to the real trajectory, overcoming the influence of the time lag.

[0065] Example 2 Track a non-cooperative target through the robust adaptive wave prediction method, and compare the tracking errors of the traditional method and the method of the present invention. The results are as Figure 3 shown. It can be seen from the figure that the tracking effect of the prediction method proposed in the embodiment of the present invention is significantly better than that of the traditional method.

[0066] Correspondingly, according to the embodiments of the present invention, the present invention also provides a visual servo system, a computer device, a readable storage medium, and a computer program product.

[0067] Figure 4 It is a schematic structural diagram of a computer device 12 provided in the embodiment of the present invention. Figure 4A block diagram of an exemplary computer device 12 suitable for implementing embodiments of the present invention is shown. Figure 4 The computer device 12 shown is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present invention.

[0068] As Figure 4 shown, the computer device 12 is presented in the form of a general-purpose computing device. The computer device 12 is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, for example, personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0069] The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples different system components (including the system memory 28 and the processing unit 16).

[0070] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures. By way of example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0071] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0072] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory (cache) 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 34 may be used for reading and writing on non-removable, non-volatile magnetic media ( Figure 4 not shown, commonly referred to as a "hard disk drive"). Although Figure 4Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (such as a "floppy disk") and an optical disk drive for reading and writing a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM or other optical medium) can be provided. In these cases, each drive can be connected to the bus 18 through one or more data medium interfaces. The memory 28 may include at least one program product having a set (such as at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0073] The program / utility 40 having a set (at least one) of program modules 42 can be stored, for example, in the memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment. The program modules 42 generally perform the functions and / or methods in the embodiments described in the present invention.

[0074] The computer device 12 can also communicate with one or more external devices 14 (such as a keyboard, a pointing device, a display 24, etc.), and can also communicate with one or more devices that enable a user to interact with the computer device 12, and / or communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (such as a network card, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 22. Moreover, the computer device 12 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and / or a public network, such as the Internet) through the network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the computer device 12 through the bus 18. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0075] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing the robust adaptive filtering prediction method with time delay compensation and noise resistance provided by the embodiments of the present invention.

[0076] In the embodiments of the present invention, a non-transitory computer-readable storage medium storing computer instructions is also provided, on which a computer program is stored. When the program is executed by a processor, the robust adaptive filtering prediction method with time delay compensation and noise resistance provided by all the embodiments of the present application is implemented.

[0077] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable media may be computer-readable signal media or computer-readable storage media. More specific examples (non-exhaustive list) of the computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, the computer-readable storage media may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or device.

[0078] The computer-readable signal media may include data signals propagated in a baseband or as part of a carrier wave, which carry computer-readable program codes. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal media may also be any computer-readable medium other than the computer-readable storage media, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, device, or device.

[0079] The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0080] The embodiment of the present invention also provides a computer program product, including a computer program, and the computer program realizes the robust adaptive filtering prediction method with time-delay compensation and noise resistance according to the above when being executed by a processor.

[0081] It should be understood that the various forms of processes shown above can be used, with steps reordered, added or deleted. For example, the steps described in the disclosure of the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and no limitation is made herein.

[0082] The above specific embodiments do not constitute a limitation to the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub - combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A robust adaptive filtering prediction method with time-delay compensation and noise resistance, characterized in that: The robust adaptive filtering prediction method with time-delay compensation and noise resistance includes the following steps: S1. The inertial stabilization platform includes an inertial measurement unit, which is used to measure the motion state information of the inertial stabilization platform; S2. Convert the rotation angle of the inertial stabilization platform into pixel coordinates in the image plane through geometric relationships; S3. Measure the inherent delay time of the vision sensor in the visual servo system △t , delay the measurement data of the inertial sensor according to △t so that the inertial data and the visual data are on the same time basis, and use the timestamp alignment algorithm to calibrate the time consistency of the inertial data and the visual data; S4. Define the target state vector as a six-dimensional vector, and based on the Singer model, establish a system state equation and a measurement equation; S5. Perform robust adaptive filtering processing; S6. Calculate the prediction steps t according to the inherent delay time △ T s of the visual sensor and the filtering period N = ceil (△ t / T s ) ; After obtaining k instantaneous state estimation perform N a one-step prediction to obtain the target position information after the compensation time lag; Use a robust adaptive Kalman filter predictor for multi-step prediction, and advance the state estimate value at the current moment by the corresponding number of time steps to compensate for the time delay in the visual servo system.

2. The robust adaptive filtering prediction method with time delay compensation and noise resistance according to claim 1, characterized in that: The motion state information includes acceleration and angular velocity; The formula for the geometric relationship is ; Wherein, θ x ,θ y are respectively the rotation angles of the inertial stabilization platform about the x-axis and y-axis of the field of view; u,v are respectively the mobile pixels of the x-axis and y-axis of the field of view, λ is the camera focal length.

3. The robust adaptive filtering prediction method with time-delay compensation and noise resistance according to claim 1, characterized in that: The six-dimensional vector includes the target position, target velocity, and target acceleration; the target state vector is: ； Among them, u T ,v T is the target position, , is the target speed, is the target acceleration; The system state equation is x T ( k +1)= Fx T ( k )+ w k , and the measurement equation is z ( k )= Hx T ( k )+ v k ; The covariance matrix is: ； Among them, x T ( k ) is the target state vector at time k, x T ( k + 1) is the target state vector at time k+1, F is the state transition matrix, w k is the process noise, is the reciprocal of the maneuver time constant, σ a 2 is the maneuver acceleration variance, H is the measurement matrix, v k is the measurement noise, represents the process noise weight of the Singer model after integral discretization.

4. The robust adaptive filtering prediction method with time-delay compensation and noise resistance according to claim 1, characterized in that: The iterative formula for the robust adaptive filtering processing is: ； Among them, is the Kalman gain, is the variance of the previous state estimate.

5. The robust adaptive filtering prediction method with time-delay compensation and noise resistance according to claim 1, characterized in that: The steps of the robust adaptive filtering processing include: S51. Calculate the prediction residual ; Among them, ( k +1) is the target state vector at the next moment predicted based on the current state, z ( k ) is the sensor measurement value at the current moment; S52. Dynamically adjust the adaptation factor based on the statistic of the prediction residual α T ; S53. Calculate the diagonal weight matrix using the IGG-III weight function .

6. The robust adaptive filtering prediction method with time-delay compensation and noise resistance according to claim 5, characterized in that: In the diagonal weight matrix ω ( i ) is calculated as follows: ； wherein, k 0 and k 1 are constants, is the standardized residual.

7. The robust adaptive filtering prediction method with time delay compensation and noise resistance according to claim 5, characterized in that: The adaptation factor α T has the following calculation formula: ； where c is a preset threshold value, represents the normalized statistic of the residual, ; is the covariance matrix of the predicted state vector, R is the measurement noise covariance matrix.

8. A visual servo system, characterized in that, The visual servo system can implement the robust adaptive filtering prediction method with time-delay compensation and noise resistance according to any one of claims 1 to 7.

9. A computer device, characterized in that, It includes: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the robust adaptive filtering prediction method with time-delay compensation and noise resistance according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the robust adaptive filtering prediction method with time-delay compensation and noise resistance according to any one of claims 1 to 7.

Images