Mechanical arm nonlinear motion state estimation method based on hybrid neural network

Abstract

The invention provides a mechanical arm nonlinear motion state estimation method based on a hybrid neural network, and relates to the technical field of state estimation. The method comprises the steps that unknown input variables are introduced, a state space model of nonlinear motion of the mechanical arm is constructed and linearized, and an expansion state model is formed; performing prior prediction and posteriori updating on the state by using an adaptive extended Kalman filtering method, and constructing an estimation result data set; dynamically correcting process noise covariance through a maximum likelihood estimation technology, and optimizing filtering performance; constructing a hybrid neural network by combining a bidirectional LSTM layer, a maximum output layer and a linear output layer, and training a state estimation model by taking a state true value and an estimated value as input and a correction residual as output; and mechanical arm state data are obtained in real time, the trained neural network is used for optimizing the Kalman filtering result, and a final correction value is output. According to the method, the accuracy and robustness of motion state estimation of the nonlinear mechanical arm are improved, and the method is suitable for complex noise and dynamic change environments.

Description

Technical Field

[0001] The invention relates to the technical field of state estimation, and in particular to a method for estimating the nonlinear motion state of a manipulator based on a hybrid neural network and intermediate variables. Background Art

[0002] With the continuous advancement of science and technology, cyber-physical systems (CPS) have been widely used in the fields of autonomous driving, smart grids, robots, medical equipment, etc. In cyber-physical systems, accurate state estimation can help monitor and adjust system parameters in real time to ensure the stability and reliability of the system during operation. For example, for autonomous driving systems, real-time state estimation can provide accurate key information such as vehicle position, speed, acceleration, etc., thereby guiding autonomous driving decisions; in smart grids, accurate state estimation helps detect load fluctuations and grid failures, and make timely adjustments and repairs. Therefore, state estimation is the basis for improving system performance and optimizing control strategies. Especially in complex environments, the behavior of nonlinear systems is difficult to predict, so it is necessary to rely on accurate state estimation to optimize controller design. Furthermore, in terms of ensuring safety, state estimation can help identify potential anomalies or failures, issue alarms in time, and take corresponding measures to prevent system failure or catastrophic consequences. However, in many application fields, the cyber-physical system is nonlinear, has unknown inputs and non-Gaussian noise, and the state estimation of the system faces challenges. It is urgent to study effective and accurate state estimation methods to ensure that the system can still operate stably and safely in complex environments.

[0003] Existing state estimation methods for cyber-physical systems mainly include Kalman filtering, particle filtering, deep learning methods, and Bayesian estimation methods. Among them, the Kalman filtering method has low computational complexity and high estimation accuracy, and is one of the mainstream methods for state estimation of nonlinear cyber-physical systems. However, the traditional Kalman filtering method still cannot effectively handle the unknown input problem in the system. To this end, some studies have proposed state estimation algorithms based on intermediate variables, but these algorithms often assume that the noise is known, while the noise is usually unknown in actual operation. Therefore, some studies use online learning methods to estimate noise parameters, and use variational Bayesian technology to directly estimate and adjust the process noise covariance in the Kalman filter to improve the performance of state filtering, but its computational complexity and dependence on prior knowledge limit its application in real-time or resource-constrained systems. A robust Kalman filtering method based on variational Bayes has been proposed for the problem of unknown noise covariance in linear systems. The robustness of the filter is enhanced by introducing robust design and hierarchical priors. However, the application scope of this method is mainly limited to linear systems, and its effectiveness and adaptability are still insufficient when dealing with more complex nonlinear systems. In further research, a uniformly distributed Kalman filter framework was proposed to compensate for mismatched noise and uncertain dynamics in sensor networks. Although the algorithm has achieved effective results in a specific network environment, its strict requirements for prior knowledge of the system limit its breadth and universality in practical applications. Some studies have proposed a method to improve positioning accuracy and avoid state estimation error inflation by adding virtual noise to the measurement matrix at each time step. Although this method can effectively improve performance in some scenarios, its universality and effectiveness in various applications have not been widely verified. Some studies have directly estimated the correlated noise covariance through a subspace identification algorithm. Although the algorithm can provide accurate initial noise covariance estimates, further research is needed on the applicability and scalability of complex or highly nonlinear systems before it can be applied in more practical scenarios. In summary, although the traditional Kalman filter-based state estimation method performs well in Gaussian noise and linear information-physical system systems, when faced with complex nonlinear dynamics, unknown inputs and non-Gaussian noise information-physical system systems, the presence of non-Gaussian noise will cause the traditional filtering method to distort the estimation of the system state, and the unknown input will further increase the complexity of the system, and the accuracy and robustness of the state estimation are difficult to guarantee.

[0004] With the rapid development of deep learning and neural networks, data-driven methods provide new solutions for state estimation of nonlinear systems. Neural networks can effectively capture the complex nonlinear dynamics of the system. With the help of neural networks, the estimation results can be corrected, which can not only improve the estimation accuracy of unknown inputs and noise, but also significantly enhance the robustness of the system, overcoming the shortcomings of traditional methods in dealing with uncertainty and complex environments. However, it still faces challenges in real-time and applicability when dealing with unknown inputs and Gaussian noise at the same time. To this end, some scholars focus on combining neural networks and Kalman filtering to study system state estimation methods. For example, a study proposed an unscented trainable Kalman filtering algorithm combined with a deep learning prediction model, aiming to improve the accuracy of state prediction in power systems, especially in the face of data loss. This method can reduce prediction errors, but its performance still depends on the integrity of the data, and the loss in data transmission will have a negative impact on the overall prediction effect. A study explored the SOC (State of Charge) estimation problem of lithium-ion batteries and proposed a hybrid state algorithm based on deep learning and Kalman filtering. The algorithm can effectively capture the spatial and temporal characteristics of the input signal, reduce the impact of transient signal fluctuations, and improve the estimation accuracy. However, the algorithm has high computational complexity in real-time operation and is not suitable for practical environments with limited resources. Some studies have proposed two integrated algorithms for acoustic howling suppression by combining Kalman filtering and deep learning technology. These algorithms improve the feedback suppression effect, but due to their complex training and implementation processes, they may face certain challenges in practical applications. Some studies have proposed the DeepKalPose method, which uses a bidirectional Kalman filtering strategy and a learnable motion model to improve the accuracy and robustness of monocular vehicle posture estimation. Although this method has improved the accuracy of posture estimation, its high computational requirements limit its application in resource-constrained environments.

[0005] In summary, the state estimation problem of robot motion usually involves unknown inputs, and its dynamic performance is nonlinear. However, most of the current state estimation methods are mainly designed for linear systems, and do not consider the unknown input interference that may exist in nonlinear systems. At the same time, the characteristics of noise are often unknown. These methods are difficult to achieve efficient and accurate state estimation when dealing with complex and unpredictable noise characteristics. Therefore, existing technologies are difficult to meet the requirements for accurate and efficient prediction of the robot motion state under conditions with unknown inputs, nonlinear forms, and uncertain noise characteristics. Summary of the invention

[0006] In view of the above situation, the present invention provides a method for estimating the nonlinear motion state of a robotic arm based on a hybrid neural network and intermediate variables. This method combines the highly nonlinear curve of the hybrid neural network with the excellent dynamic tracking ability of the Kalman filter, which not only improves the accuracy of the robotic arm state estimation, but also enhances the adaptability to complex dynamic robotic arm systems. Especially in the face of incomplete and rapidly changing data, it can effectively improve the overall performance and robustness of the robotic arm motion, and provide a new solution for processing complex and dynamically changing robotic arm motion environments.

[0007] The technical solution provided by the present invention is as follows:

[0008] The present invention provides a method for estimating the nonlinear motion state of a robotic arm based on a hybrid neural network, comprising:

[0009] S1. According to the motion of the robot arm, considering that the robot arm may be disturbed by unknown input during operation, unknown input variables are introduced, the state space model of the nonlinear motion system of the robot arm is constructed, and the observation equation is obtained;

[0010] S2. Using the extended Kalman filter method, the observation equation is linearized using the Taylor first-order expansion principle, and then intermediate variables are introduced to construct the system extended state, so as to obtain a robot arm state estimation system model corresponding to the system extended state;

[0011] S3, setting the initial values ​​of the parameters of the robot arm state estimation system model at the initial moment, and constructing a state estimation result data set by an adaptive extended Kalman filter method;

[0012] S4, updating and calculating the posterior extended state, Kalman gain and state covariance matrix according to the Kalman filtering method;

[0013] S5. According to the maximum likelihood principle, the noise covariance of the estimation process is calculated, and then the excess noise covariance in the Kalman filter method is updated and returned to S3 until the specified number of cycles is completed, and a data set of the true value and estimated value of the state of the robot arm motion process is obtained;

[0014] S6, constructing a hybrid neural network consisting of two bidirectional LSTM layers, a maximum output layer and a linear output layer, and dividing the data set of the true value and estimated value of the state of the robot arm motion process into a training set and a verification set, using the true state value of the training set and the verification set and the state estimated value calculated by the adaptive extended Kalman filter method as the input of the hybrid neural network, correcting the residual of the estimation result as the output of the hybrid neural network, training the hybrid neural network, and obtaining a nonlinear motion state estimation model of the robot arm;

[0015] S7, entering the real-time state estimation stage of the robotic arm, setting the parameters to the initial values ​​of the parameters;

[0016] S8, read the real-time state data of the current robotic arm; calculate the prior state estimate and measurement value prediction value, the posterior extended state of the robotic arm state, calculate the Kalman gain and calculate the state covariance matrix, update the system process noise, input the obtained state data into the trained neural network, obtain the correction residual of the state estimate, add the correction residual to the state estimate, and finally output the corrected robotic arm state value, and repeat S8 until the robotic arm movement is completed.

[0017] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0018] The present invention effectively handles the unknown input and unknown Gaussian noise problems in the nonlinear information-physical system for the motion of the manipulator by introducing intermediate variables and maximum likelihood technology, and expands the application scope of the traditional Kalman filter method. Secondly, although the traditional Kalman filter provides an effective framework for state estimation, it faces limitations when dealing with highly nonlinear systems and complex noise environments. In order to overcome these challenges and improve the accuracy of the estimation, the present invention combines the training of hybrid neural networks with the adaptive extended Kalman filter technology, and uses the powerful nonlinear fitting ability of the neural network to correct and optimize the output of the adaptive extended Kalman filter. This not only enhances the dynamic adaptability of the filter to the estimation of complex manipulator motion, but also improves the accuracy of the manipulator state estimation in a complex environment, especially when dealing with non-Gaussian noise and dynamic changes in the motion state of the manipulator, showing good performance. In addition, the present invention also uses deep learning to perform real-time correction on the Kalman filter results, further improving the accuracy and robustness of the filter estimation. Combining these technologies, the present invention not only adapts to the actual application scenarios with dynamic and uncertain conditions, but also effectively solves the state estimation problem of nonlinear systems such as manipulators under complex noise backgrounds, significantly improving the operating efficiency and safety of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A schematic diagram of a flow chart of a method for estimating a nonlinear motion state of a robotic arm based on a hybrid neural network provided in an embodiment of the present invention;

[0021] Figure 2 A schematic flow chart of a method for estimating the nonlinear motion state of a robotic arm based on a hybrid neural network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0023] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0024] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0025] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0026] Reference Manual Attached Figure 1 , shows a flow chart of a method for estimating the nonlinear motion state of a robotic arm based on a hybrid neural network provided in an embodiment of the present invention.

[0027] The embodiment of the present invention provides a method for estimating the nonlinear motion state of a robotic arm based on a hybrid neural network. The processing flow may include the following steps:

[0028] S1. According to the motion of the robot arm, considering that the robot arm may be disturbed by unknown input during operation, unknown input variables are introduced, the state space model of the nonlinear motion system of the robot arm is constructed, and the observation equation is obtained;

[0029] S2. Use the extended Kalman filter method and Taylor's first-order expansion principle to linearize the observation equation, then introduce intermediate variables and construct the system extended state, so as to obtain the robot arm state estimation system model corresponding to the system extended state;

[0030] S3, setting the initial values ​​of the parameters of the robot arm state estimation system model at the initial moment, and constructing a state estimation result data set by an adaptive extended Kalman filter method;

[0031] S4, updating and calculating the posterior extended state, Kalman gain and state covariance matrix according to the Kalman filtering method;

[0032] S5. According to the maximum likelihood principle, the noise covariance of the estimation process is calculated, and then the excess noise covariance in the Kalman filter method is updated and returned to S3 until the specified number of cycles is completed, and a data set of the true value and estimated value of the state of the robot arm motion process is obtained;

[0033] S6. Construct a hybrid neural network consisting of two bidirectional LSTM layers, a maximum output layer, and a linear output layer, and divide the data set of the true value and estimated value of the state of the robot arm motion process into a training set and a validation set, use the true state value of the training set and the validation set and the state estimated value calculated by the adaptive extended Kalman filter method as the input of the hybrid neural network, correct the residual of the estimation result as the output of the hybrid neural network, perform hybrid neural network training, and obtain a nonlinear motion state estimation model of the robot arm;

[0034] S7, entering the real-time state estimation stage of the robot arm, setting the parameters to the initial values ​​of the parameters;

[0035] S8, read the real-time state data of the current robotic arm; calculate the prior state estimate and measurement value prediction value, the posterior extended state of the robotic arm state, calculate the Kalman gain and calculate the state covariance matrix, update the system process noise, input the obtained state data into the trained neural network, obtain the correction residual of the state estimate, add the correction residual to the state estimate, and finally output the corrected robotic arm state value, and repeat S8 until the robotic arm movement is completed.

[0036] In a possible implementation, S1 further includes:

[0037] Unknown inputs include sudden disturbances, faults, and disturbances caused by inaccurate modeling, which introduce unknown input variables m k and n k+1 , the nonlinear system model of the robotic arm is constructed as follows:

[0038]

[0039] Among them, x k =[p x,k p y,k α k θ k ω k ] T represents the state matrix of the robot at time k, x k+1 represents the state matrix of the robot at time k+1, p x,k represents the coordinate of the end of the robot in the X direction at time k, p y,k represents the Y-direction coordinate of the end of the robot at time k, α k represents the speed of the end of the robot at time k, θk represents the deflection angle of the end of the robot at time k, ω k represents the angular velocity of the end of the robot at time k, f(x k ) represents the state transition equation during the system operation, B k ∈R 5×2 represents the observation matrix of the control input, u k Represents the robot input matrix, m k represents the unknown input of the system state at time k, M k Represents unknown input m k The corresponding matrix, y k+1 =[p 1 p 2 p 3 ] T represents the observation matrix at the end of the robot at time k+1, p 1 represents the horizontal position of the end of the robot arm observed at time k+1, p 2 represents the longitudinal position of the end of the robot arm observed at time k+1, p 3 represents the vertical position of the end of the robot arm observed at time k+1, h(x k+1 ) represents the nonlinear function in the measurement process, obtained by binocular vision, n k+1 represents the unknown input observed by the system at time k+1, N k+1 Represents unknown input n k+1 The corresponding matrix, v k+1 Represents the system observation noise at time k+1, which is set to position noise.

[0040] In a possible implementation, S2 further includes:

[0041] After linearizing the observation equation, the obtained robotic arm system model is as follows:

[0042]

[0043] in, is the Jacobian matrix, is the Jacobian matrix, omitting x k and x k+1 The higher-order terms of , and then the intermediate variables are introduced Where μ is an adjustable parameter, we get

[0044]

[0045] in, Δm=m k -m k-1 , build system extended status in represents the transpose of the intermediate variable, represents the transpose of the unknown input, and the expanded state includes both the motion state of the robot and the unknown input, thereby obtaining the system expanded state z k The corresponding robot arm state estimation system model is:

[0046]

[0047] Among them, the intermediate variable

[0048]

[0049] In a possible implementation, S3 and S7 further include:

[0050] Set the initial values ​​of the parameters of the robot arm state system model at the initial time k = 1:

[0051] Initialize the motion state matrix A 1 =I(5);

[0052] Control input matrix B 1 =[1 1 1 0.01 0.01] T ;

[0053] Unknown input matrix M 1 =[0 cos(0.1) sin(0.1) 0 0 0];

[0054] Unknown input matrix M 1 = [1 1 0];

[0055] System observation noise covariance matrix R = I (3) × 0.1;

[0056] Intermediate variable τ 1 =0;

[0057] Unknown input m 1 =0;

[0058] Unknown input n 1 =0;

[0059] System Status x 1 =[113 -140 34.1 39.3 -41.7] T ;

[0060] Extended Status

[0061] z 1 =[x 1 τ 1 n 1 ] T=[113 -140 34.1 39.3 -41.7 0 0] T ;

[0062] Adjustable variable μ = 1.5;

[0063] Control input u=1;

[0064] Unknown noise covariance Q 1 =0;

[0065] The setting parameters in S7 are the same as the initial values ​​mentioned above.

[0066] In a possible implementation, S3 further includes:

[0067] The state estimation result data set is constructed by the adaptive extended Kalman filter method. First, the prior state estimation is calculated by the following formula: and the measured value predicted value

[0068]

[0069] in, represents the posterior estimate at time k, and the prior state error is

[0070]

[0071] Then the prior error covariance for

[0072]

[0073] Among them, E(·) is the expected value calculation function of the matrix, represents the state covariance matrix at time k-1, Q k represents the unknown noise covariance, considering e k-1 and Δr k-1 Independent of each other,

[0074]

[0075] in, is the unknown process noise covariance of the system.

[0076] In a possible implementation, S4 further includes:

[0077] A posteriori extended state Kalman gain K k+1 and the state covariance matrix The calculation formula is as follows:

[0078]

[0079] Where R represents the noise covariance during the measurement process.

[0080] In a possible implementation, S5 further includes:

[0081] According to the maximum likelihood principle, the estimated process noise covariance is:

[0082]

[0083] in, represents the estimated error covariance at time j, represents the prior estimate at time k-1, and the process noise covariance in the Kalman filter method is updated as And k=k+1 and return to S3 to reset the parameters to the initial values, and end after looping for a specified number of times to obtain a data set of the actual value and the estimated value of the state during the movement of the robot arm.

[0084] In a possible implementation, S8 further includes:

[0085] After reading the real-time state data of the current robot arm, the prior state estimate is calculated using the method in S3 and the measured value predicted value Then calculate the posterior extended state by the method in S4 Kalman gain K k+1 and the state covariance matrix Then, the system process noise is updated according to the estimated process noise covariance formula in S5, and the obtained state data is input into the trained neural network to obtain the corrected residual α of the state estimation k , then add the correction residual and the state estimate, and finally output the corrected robot state value The above step S8 is repeatedly executed until the movement of the robot arm is completed.

[0086] In one possible implementation, the process noise covariance in S5 is The estimation method is as follows:

[0087] S501, set the new information vector Let the difference of the innovation vector be:

[0088]

[0089] Get the measurement vector y k The conditional probability density p y,k It can be expressed as

[0090]

[0091] Where m is the dimension of the measurement vector, |·| represents the modulus of the determinant;

[0092] S502, calculate the likelihood function L based on the measured value:

[0093]

[0094] Where N is the window size of the fixed-length memory, and taking the logarithm of the likelihood function of the measured value, we can get

[0095]

[0096] Among them, const represents a constant;

[0097] S503. According to the principle of maximum likelihood, the process noise covariance is obtained by the following formula:

[0098]

[0099] Then substitute the logarithmic formula of the likelihood function of the measured value into the above formula, multiply both sides of the equation by -2, ignore the constant term, and further obtain In the maximum likelihood estimation, the following conditions are met:

[0100]

[0101] Where Q represents the set of all noise covariances that meet the conditions;

[0102] S504, for The conditions satisfied in the maximum likelihood estimation define the cost function J(Q) as

[0103]

[0104] make available

[0105]

[0106] Where tr{·} represents the trace of the matrix, Q il (i,l=1,2,…,n) represents the element in Q in the i-th row and l-th column;

[0107] S505, respectively estimate the covariance of the prior and the measurement error covariance Conduct Q il Partial derivative, we can get

[0108]

[0109] S506: When the filtering process in the estimation window reaches a steady state, The first term on the right side of the formula is 0. Formula Substitution Formula, let available

[0110]

[0111] S507, will Substitute the formula into S504 command The formula obtained can be obtained

[0112]

[0113] Simplified to

[0114]

[0115] S508, after substituting the Kalman gain into

[0116]

[0117] Considering that the filtering method works in a steady state, an explicit approximate estimate of Q can be obtained. That is, considering that the filtering process within the estimation window is in a steady state, For all j (j = k-N+1, k-N+2, ..., k), it is approximately a constant. Then the above formula can be converted to

[0118]

[0119] in, According to the Kalman filtering process, we can get

[0120]

[0121] S509, substituting the two formulas obtained through the Kalman filtering process in S508 into the converted formula in S508, the estimation formula of the process noise covariance Q can be obtained as follows:

[0122]

[0123] In a possible implementation, a hybrid neural network consisting of two bidirectional LSTM layers, a maximum output layer, and a linear output layer is constructed in S6. The method for constructing the hybrid neural network is as follows:

[0124] S601, two bidirectional LSTM layers are used. The first layer consists of 128 neurons and a noise activation function. The second layer consists of 256 neurons and a Tanh activation function. The Tanh activation function is multiplied by 0.5, so that the noise works in both saturated and unsaturated regions, thereby helping the gradient descent process avoid falling into the local optimal solution and improving the training effect of the overall model. Noise activation function φ n (θ,ξ) is defined as follows:

[0125] φ n (θ,ξ)=φ p (θ)+σ(θ)ξ

[0126] Among them, θ is the input of the noise activation function, ξ is the noise that follows the normal distribution, and φ p (θ) is the Tanh function, defined as

[0127]

[0128] σ(θ) is the scaling function of the noise ξ and can be expressed as:

[0129] σ(θ)=(sigmoid(φ p (θ)-θ)-0.5) 2

[0130] Among them, sigmoid() is the activation function, which can be expressed as

[0131] S602, use the maximum output layer. This layer uses the Maxout activation function, which activates by selecting the maximum value of all neuron outputs. At each time step k, a new tensor h is generated through a full link layer. k The tensor h k There are N nodes. Each node is denoted as h k (i)∈h k . Use the maximum output unit of the input feature to convert h k Mapped to the maximum output layer. h k The mapping is represented as:

[0132]

[0133] There are I subsets, each of which contains S nodes, and there are a total of I×S=N nodes. Represents the largest node in the subset, i∈[1:I].

[0134] S603, using a linear output layer. In order to obtain the expected output of the neural network, the residual value α k The prediction must go through another fully connected layer, which is responsible for extracting the features Mapped to the final prediction output α k . Using linear activation function φ o (u) = u, and the state correction residual α is obtained k .

[0135] In a possible implementation, S6 further divides the obtained data set into a training set and a validation set, with the first 80% being the training set and the last 20% being the validation set. k and the state estimate calculated by the adaptive extended Kalman filter method As the input of the hybrid neural network, the residual α of the corrected estimation result k As the output of the hybrid neural network, the hybrid neural network is trained to obtain the nonlinear motion state estimation model of the robot arm. The neural network training process is as follows:

[0136] S604, initializing the hybrid neural network. Each layer in the network is randomly initialized using normal distribution to assign a random initial value.

[0137] S605: Input the data of the training set and the validation set, and set a sliding window L to filter the input data to reduce the noise of the input data, wherein the sliding window L is a 7×7 matrix, which can be expressed as:

[0138]

[0139] In which, each element in the input data is filtered independently by each element in L, which can be expressed as:

[0140]

[0141] in, It represents the state value after filtering. When the vector index kn<0, the vector is set to 0. After obtaining the filtered training set and validation set data, the filtered training set data is first input into the hybrid neural network.

[0142] S606, the hybrid neural network performs forward propagation, and the input data is passed from the input layer to the output layer through each layer. In each LSTM layer, the input data is linearly transformed, and the hidden state of the current time step is generated through bidirectional propagation, and is passed to the second LSTM layer as the input of the next layer. After passing through the maximum output layer and the linear output layer, the predicted value is generated.

[0143] S607, start the back propagation phase, and calculate the loss function of the hybrid neural network through the root mean square error loss function of the following formula.

[0144]

[0145] Among them, γ represents the loss function value, K represents the number of samples, represents the state prediction value predicted by the hybrid neural network, α k Represents the true value of the state. When the value of the loss function is less than the threshold ε, jump to step S608, otherwise use the minimum gradient descent method to update the hybrid neural network parameters. The neural network calculates the gradient of each layer according to the result of the loss function, and then transfers these gradients back layer by layer. Based on the historical information of the gradient, the Adam algorithm is used to dynamically adjust the learning rate of each parameter, update the parameters of the network, and jump to step S606 again.

[0146] S608. Input the verification set into the trained hybrid neural network, calculate the robot arm state estimation error corresponding to the verification set, if the error is less than the specified threshold Ω, the trained hybrid neural network model is valid, output the neural network model, otherwise return to step S606, re-input the filtered training set data into the hybrid neural network, and retrain.

[0147] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0148] The present invention effectively handles the unknown input and unknown Gaussian noise problems in the nonlinear information-physical system for the motion of the manipulator by introducing intermediate variables and maximum likelihood technology, and expands the application scope of the traditional Kalman filter method. Secondly, although the traditional Kalman filter provides an effective framework for state estimation, it faces limitations when dealing with highly nonlinear systems and complex noise environments. In order to overcome these challenges and improve the accuracy of the estimation, the present invention combines the training of hybrid neural networks with the adaptive extended Kalman filter technology, and uses the powerful nonlinear fitting ability of the neural network to correct and optimize the output of the adaptive extended Kalman filter. This not only enhances the dynamic adaptability of the filter to the estimation of complex manipulator motion, but also improves the accuracy of the manipulator state estimation in a complex environment, especially when dealing with non-Gaussian noise and dynamic changes in the motion state of the manipulator, showing good performance. In addition, the present invention also uses deep learning to perform real-time correction on the Kalman filter results, further improving the accuracy and robustness of the filter estimation. Combining these technologies, the present invention not only adapts to the actual application scenarios with dynamic and uncertain conditions, but also effectively solves the state estimation problem of nonlinear systems such as manipulators under complex noise backgrounds, significantly improving the operating efficiency and safety of the system.

[0149] The above contents are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0150] There are a few points to note:

[0151] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention, and other structures may refer to the general design.

[0152] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0153] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0154] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A nonlinear motion state estimation method for a robotic arm based on a hybrid neural network, characterized in that: include: S1. According to the motion of the robot arm, considering that the robot arm may be disturbed by unknown input during operation, unknown input variables are introduced, the state space model of the nonlinear motion system of the robot arm is constructed, and the observation equation is obtained; S2. Using the extended Kalman filter method, the observation equation is linearized using the Taylor first-order expansion principle, and then intermediate variables are introduced to construct the system extended state, so as to obtain a robot arm state estimation system model corresponding to the system extended state; S3, setting the initial values ​​of the parameters of the robot arm state estimation system model at the initial moment, and constructing a state estimation result data set by an adaptive extended Kalman filter method; S4, updating and calculating the posterior extended state, Kalman gain and state covariance matrix according to the Kalman filtering method; S5. According to the maximum likelihood principle, the noise covariance of the estimation process is calculated, and then the excess noise covariance in the Kalman filter method is updated and returned to S3 until the specified number of cycles is completed, and a data set of the true value and estimated value of the state of the robot arm motion process is obtained; S6, constructing a hybrid neural network consisting of two bidirectional LSTM layers, a maximum output layer and a linear output layer, and dividing the data set of the true value and estimated value of the state of the robot arm motion process into a training set and a verification set, using the true state value of the training set and the verification set and the state estimated value calculated by the adaptive extended Kalman filter method as the input of the hybrid neural network, correcting the residual of the estimation result as the output of the hybrid neural network, training the hybrid neural network, and obtaining a nonlinear motion state estimation model of the robot arm; S7, entering the real-time state estimation stage of the robotic arm, setting the parameters to the initial values ​​of the parameters; S8, read the real-time status data of the current robot arm; Calculate the prior state estimate and measurement value prediction value of the robot state, the a posteriori extended state, calculate the Kalman gain and calculate the state covariance matrix, update the system process noise, input the obtained state data into the trained neural network, obtain the corrected residual of the state estimate, add the corrected residual to the state estimate, and finally output the corrected robot state value, and repeat S8 until the robot movement is completed.

2. The method for estimating the nonlinear motion state of a robotic arm based on a hybrid neural network according to claim 1, characterized in that: The S1 further comprises: The unknown input includes sudden disturbances, faults and disturbances caused by inaccurate modeling, and introduces unknown input variables m k and n k+1 , the nonlinear system model of the robotic arm is constructed as follows: Among them, x k =[p x,k p y,k α k θ k ω k ] T represents the state matrix of the robot at time k, x k+1 represents the state matrix of the robot at time k+1, p x,k represents the coordinate of the end of the robot in the X direction at time k, p y,k represents the Y-direction coordinate of the end of the robot at time k, α k represents the speed of the end of the robot at time k, θ k represents the deflection angle of the end of the robot at time k, ω k represents the angular velocity of the end of the robot at time k, f(x k ) represents the state transition equation during the system operation, B k ∈R 5×2 represents the observation matrix of the control input, u k represents the robot input matrix, m k represents the unknown input of the system state at time k, M k Represents unknown input m k The corresponding matrix, y k+1 =[p1 p2 p3] T represents the observation matrix of the end of the robot at time k+1, p1 represents the horizontal position of the end of the robot at time k+1, p2 represents the longitudinal position of the end of the robot at time k+1, p3 represents the vertical position of the end of the robot at time k+1, h(x k+1 ) represents the nonlinear function in the measurement process, obtained by binocular vision, n k+1 represents the unknown input observed by the system at time k+1, N k+1 Represents unknown input n k+1 The corresponding matrix, v k+1 Represents the system observation noise at time k+1, which is set to position noise.

3. The method for estimating the nonlinear motion state of a robotic arm based on a hybrid neural network according to claim 2, characterized in that: The S2 further comprises: After linearizing the observation equation, the obtained robotic arm system model is as follows: in, is the Jacobian matrix, is the Jacobian matrix, omitting x k and x k+1 The higher-order terms of , and then the intermediate variables are introduced Where μ is an adjustable parameter, we get in, Δm=m k -m k-1 , build system extended state in represents the transpose of the intermediate variable, represents the transpose of the unknown input, and the expanded state includes both the motion state of the robot and the unknown input, thereby obtaining the system expanded state z k The corresponding robot arm state estimation system model is: Among them, the intermediate variable 4. The method for estimating the nonlinear motion state of a robotic arm based on a hybrid neural network according to claim 3, characterized in that: The S3 and S7 further include: Set the initial values ​​of the parameters of the robot arm state system model at the initial time k=1: Initialize the motion state matrix A1=I(5); Control input matrix B1 = [1 1 1 0.01 0.01]T ； Unknown input matrix M1 = [0 cos(0.1) sin(0.1) 0 0 0]; Unknown input matrix M1 = [1 1 0]; System observation noise covariance matrix R = I (3) × 0.1; Intermediate variable τ1 = 0; Unknown input m1=0; Unknown input n1 = 0; System state x1 = [113 -140 34.1 39.3 -41.7] T ; Extended Status z1=[x1 τ1 n1] T =[113 -140 34.1 39.3 -41.7 0 0] T ; Adjustable variable μ = 1.5; Control input u=1; Unknown noise covariance Q1 = 0; The setting parameter in S7 is that the initial value of the parameter is the same as the above initial value.

5. The method for estimating the nonlinear motion state of a robotic arm based on a hybrid neural network according to claim 4, characterized in that: The S3 further comprises: The state estimation result data set is constructed by the adaptive extended Kalman filter method. First, the prior state estimation is calculated by the following formula: and the measured value predicted value in, represents the posterior estimate at time k, and the prior state error is Then the prior error covariance for Among them, E(·) is the expected value calculation function of the matrix, represents the state covariance matrix at time k-1, Q k represents the unknown noise covariance, considering e k-1 and Δr k-1 Independent of each other, in, is the unknown process noise covariance of the system.

6. The method for estimating the nonlinear motion state of a robotic arm based on a hybrid neural network according to claim 5, characterized in that: The S4 further comprises: The a posteriori extended state Kalman gain K k+1 and the state covariance matrix The calculation formula is as follows: Where R represents the noise covariance during the measurement process.

7. The method for estimating the nonlinear motion state of a robotic arm based on a hybrid neural network according to claim 1, characterized in that: The S5 further comprises: According to the maximum likelihood principle, the estimated process noise covariance is: in, represents the estimated error covariance at time j, represents the prior estimate at time k-1, and the process noise covariance in the Kalman filter method is updated as And k=k+1 and return to S3 to reset the parameters to initial values, and end after looping for a specified number of times to obtain a data set of the actual value and the estimated value of the state during the movement of the robot arm.

8. The method for estimating the nonlinear motion state of a robotic arm based on a hybrid neural network according to claim 7, characterized in that: The S8 further comprises: After reading the real-time state data of the current robot arm, the prior state estimate is calculated by the method in S3 and the measured value predicted value Then calculate the a posteriori extended state by the method in S4 The Kalman gain K k+1 and the state covariance matrix Then, the system process noise is updated according to the estimated process noise covariance formula in S5, and the obtained state data is input into the trained neural network to obtain the correction residual α of the state estimation. k , then add the correction residual and the state estimate, and finally output the corrected robot state value The above step S8 is repeatedly executed until the movement of the robot arm is completed.

Images