A Machine Learning Optimization Method Based on Mathematical Models

Abstract

This invention discloses a machine learning optimization method based on a mathematical model, relating to the field of machine learning technology. The method is implemented through the following steps: First, optimization points and momentum are initialized on a Lie group; then, gradient calculation, momentum update, and point update operations are repeatedly performed until the convergence condition is met; subsequently, the iterative operations are mapped to optimization layers in a neural network, and multiple optimization layers are stacked to construct a deep unfolded neural network architecture; this architecture is trained using domain-specific data to optimize network parameters; finally, the trained network is deployed to optimize mathematical models with geometrically structured data in robotics. This invention fully utilizes the geometrical characteristics of the data, improving the efficiency, accuracy, and stability of the optimization process, enhancing the model's understanding and adaptability to problems, and can be applied to multiple problems in robotics, such as posture estimation and trajectory tracking, to achieve effective optimization.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, specifically to a machine learning optimization method based on a mathematical model. Background Technology

[0002] With the rapid development of artificial intelligence technology, machine learning methods have been widely applied in many fields, such as robotics, computer vision, and natural language processing. However, in practical applications, many problems involve data with complex geometric structures, and traditional machine learning optimization methods have many limitations when processing such data.

[0003] In robotics, pose estimation and trajectory tracking are two key tasks. Pose data are usually located on special Euclidean groups such as SE(3), which have unique mathematical properties, such as non-Euclidean metrics and curvature characteristics. Traditional optimization algorithms based on Euclidean space assumptions, such as gradient descent and Newton's method, often fail to fully utilize the geometric information when processing this type of data, resulting in low optimization efficiency, easy getting trapped in local optima, and difficulty in guaranteeing the accuracy and stability of the solution.

[0004] Patent CN111738261B proposes a single-image robot unordered target grasping method based on pose estimation and correction. By training a convolutional neural network with synthetic data, it avoids overfitting to specific dataset distributions, resulting in a network robust to changes in illumination, camera angles, and background. This method boasts high reliability, robustness, and real-time performance. However, it heavily relies on large amounts of training data and a complex model training process, leaving room for improvement in real-time performance and robustness.

[0005] Patent CN111508024A proposes a method for estimating robot pose based on deep learning. It uses a deep learning network to estimate the robot's pose and learns pose information in a data-driven manner. However, this method is highly dependent on data and may not be able to fully utilize the geometric characteristics of the data when dealing with complex geometric data, resulting in limited optimization effect.

[0006] Patent CN110703783B proposes an algorithm for real-time identification of the current reference trajectory point in autonomous driving trajectory tracking. This algorithm is used for trajectory tracking control of autonomous vehicles and can identify the current reference trajectory point in real time, improving the accuracy and real-time performance of trajectory tracking. However, computational efficiency and accuracy may be affected when facing complex environments and large-scale data.

[0007] Deep unrolling technology offers a new approach to solving the aforementioned problems. By mapping each step of a traditional iterative optimization algorithm to a trainable layer of a neural network, a deeply unrolled neural network architecture can be constructed. This architecture not only retains the theoretical foundation of traditional optimization algorithms but also fully utilizes the powerful modeling capabilities of neural networks to achieve efficient optimization of the problem. However, some challenges remain in applying deep unrolling technology to optimization problems with geometrically structured data, such as how to design optimization layer structures suitable for geometrically structured data and how to effectively integrate domain knowledge into the network architecture.

[0008] In summary, existing technologies have many shortcomings in handling machine learning optimization problems with geometrically structured data. Therefore, this invention proposes a machine learning optimization method based on a mathematical model, which aims to fully utilize the geometric characteristics of the data to improve the efficiency, accuracy, and stability of the optimization process, while enhancing the model's understanding and adaptability to the problem. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides a machine learning optimization method based on mathematical models, which solves the problems mentioned in the background.

[0010] To achieve the above objectives, the present invention provides the following technical solution: a machine learning optimization method based on a mathematical model, comprising the following steps:

[0011] S1. Initialize the optimization point X0 on the Lie group manifold and initialize the momentum to the zero vector V0 = 0; wherein, the initial value of the optimization point is determined based on the geometric structural characteristics of the target neighborhood data;

[0012] S2. Repeat the following sub-steps until the convergence condition is met:

[0013] S2.1 Gradient calculation, calculate the objective function f(x) = X k At the current optimization point X k gradient at By projecting the gradient onto the Lie algebra space g using the differential mapping from the Lie group to the Lie algebra, we obtain the Lie algebraic gradient direction ξ. k :

[0014] S2.2 Momentum Update, based on Lie algebra gradient direction ξ k and the momentum V of the previous step k-1 According to the momentum update rule V k =βV k-1 +ηξ k Update the current momentum, where β is the momentum coefficient and η is the learning rate;

[0015] S2.3, Update, update the momentum V kThe corresponding Lie algebra elements are transformed onto the Lie group through the exponential mapping exp: g → G, generating a new optimization point X. k+1 =X k *exp(V k );

[0016] S3. Deep Unfolded Network Construction Steps: Map the single iteration optimization steps (S2.1-S2.3) to a trainable optimization layer of the neural network; construct a deep unfolded neural network architecture by stacking N isomorphic optimization layers, where the network depth N and the in-layer structural parameters are dynamically configured according to the complexity of the target mathematical model;

[0017] S4. Train the network end-to-end using a domain-specific dataset: Calculate the gradient of the loss function using the backpropagation algorithm; update the network parameters, including the learning rate η, momentum coefficient β, and mapping operator weights, through the optimizer;

[0018] S5. Deploy the trained deep unfolded network on the target system, input data with geometric structure, and output optimized mathematical model parameters.

[0019] Preferably, in step S1, the initial value of the optimization point is determined based on the geometric characteristics of the robotic data; the initial value of the momentum is a zero vector.

[0020] Preferably, in step S2.2, the momentum update rule in the adaptive optimization algorithm is adopted, wherein the momentum coefficient is dynamically adjusted according to the state information during the optimization process.

[0021] Preferably, in step S3, residual connections are used between adjacent optimization layers in the deep unfolded neural network architecture.

[0022] Preferably, in step S3, the specific input and output design of the optimization layer is as follows: the input layer receives data with geometric structure in robotics, including position, attitude and velocity information; the output layer outputs optimized motion planning parameters, including trajectory point position and attitude.

[0023] Preferably, in step S3, the depth of the deep unfolded neural network architecture is determined according to the complexity of the mathematical model to be optimized, and the structural parameters of the optimization layer are determined according to the complexity.

[0024] Preferably, in step S4, the parameters of the deep unfolded neural network are calculated using the backpropagation algorithm to calculate the gradient and are adjusted using an optimization algorithm.

[0025] Preferably, the method can also be applied to pose estimation and trajectory tracking problems in robotics, and can achieve effective optimization of different robotic problems by adjusting the loss function and output layer design in the deep unfolded network architecture.

[0026] This invention provides a machine learning optimization method based on a mathematical model, which has the following beneficial effects:

[0027] 1. By performing optimization operations on Lie groups, the geometric structure of the data is fully respected. Compared with traditional optimization algorithms based on Euclidean space assumptions, this method can more effectively handle information with complex geometric structures, such as pose data in robotics. It avoids problems such as low optimization efficiency and susceptibility to local optima caused by ignoring geometric structure, making the optimization process more targeted and efficient, and significantly improving the accuracy and stability of the optimization results.

[0028] 2. Each step of a traditional iterative optimization algorithm is mapped to a trainable optimization layer in a neural network, and a deep unfolded neural network architecture is constructed by stacking multiple optimization layers. This architecture not only retains the solid theoretical foundation of traditional optimization algorithms but also cleverly integrates the powerful modeling and learning capabilities of neural networks. Compared to a single optimization algorithm or a simple neural network model, the deep unfolded architecture can more deeply mine the potential patterns and complex relationships in the data with limited computing resources, achieving more accurate optimization results, and is especially suitable for complex robotics optimization tasks. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Example 1:

[0032] In the field of industrial robotics, accurate motion trajectory planning is crucial. Traditional methods often struggle to balance trajectory smoothness, obstacle avoidance requirements, and the physical limitations of joint movement when planning robot motion trajectories. This is especially true for workspaces with complex geometries, such as environments with multiple obstacles or narrow passages, where it is difficult to generate optimal motion trajectories.

[0033] Implementation steps are as follows Figure 1 As shown, it includes the following steps:

[0034] Step 1: Data preparation. Collect motion data of industrial robots in complex workspaces, including geometric data such as position, posture, and velocity information. This data reflects the robot's motion state and trajectory characteristics in different work scenarios, and also records the geometric layout of the workspace, such as obstacle positions and aisle widths.

[0035] Step 2: Model Construction. Based on the robot's kinematic model and the geometric characteristics of the workspace, a corresponding mathematical model is constructed. Using the robot's joint angles and the position and orientation of the end effector as variables, and with trajectory smoothness and obstacle avoidance distance maximization as optimization objectives, an objective function is constructed, and its expression on a Lie group is determined to suit subsequent optimization algorithms.

[0036] Step 3: Initialization settings. Initialize optimization points on the Lie group based on the robot's initial pose and the workspace geometry. These optimization points represent the initial guess of the robot's motion trajectory, and the momentum is initialized to a zero vector to prepare for the subsequent iterative optimization process.

[0037] Step 4: Optimize the training. Input the prepared motion data into the optimization model built on a deep unfolded neural network architecture. Through end-to-end training, the gradient of the loss function is calculated using the backpropagation algorithm, and then the network parameters, including the learning rate η, momentum coefficient β, and mapping operator weights, are updated by an optimizer (such as the Adam optimizer). During training, the deep unfolded neural network architecture simulates a multi-step optimization iteration process. Each step corresponds to an optimization adjustment of the robot's motion trajectory. Residual connections are used between adjacent optimization layers, which helps to alleviate the gradient vanishing problem and accelerate network convergence.

[0038] Step 5: Application of Results. After sufficient training, the optimized model is deployed into the control system of the industrial robot. During actual operation, the robot's current position, posture, speed, and workspace geometry are input into the model. The model quickly outputs optimized motion trajectory planning parameters, such as the angle change sequence of each joint, the trajectory point position and posture of the end effector, etc., enabling the robot to move smoothly and efficiently according to the optimized trajectory, effectively avoiding obstacles, while meeting the physical constraints of joint movement, thus improving production efficiency and operational accuracy.

[0039] Performance verification: Compared with traditional robot motion trajectory planning methods, such as gradient descent-based numerical optimization methods or classic kinematic planning algorithms, this method improves the trajectory smoothness index by about 30%, increases the obstacle avoidance success rate by 25%, and reduces joint motion speed fluctuation by 40%, significantly improving the robot's motion performance and verifying the effectiveness and superiority of this optimization method in industrial robot trajectory planning problems.

[0040] Example 2: Robot pose estimation optimization

[0041] In the field of robot navigation and localization, accurate attitude estimation is crucial for the autonomous movement and environmental perception of robots. For example, during the flight of a drone, its attitude (including roll, pitch, and yaw angles) needs to be estimated accurately in real time for stable flight control and precise terrain mapping. However, due to factors such as sensor noise, environmental interference, and the dynamic characteristics of the drone itself, attitude estimation obtained directly from sensor data often contains significant errors.

[0042] Implementation steps:

[0043] Data collection: Using various sensors on the drone, such as gyroscopes, accelerometers, and barometers, raw sensor data is collected during flight. This data includes information such as the drone's angular velocity, linear acceleration, and barometric altitude. At the same time, relatively accurate position and attitude reference data obtained by the drone through GPS or vision systems are also collected for subsequent optimization training and error assessment.

[0044] Mathematical Modeling: Based on the dynamic model of the UAV and the characteristics of the sensors, a mathematical model for attitude estimation is constructed. The attitude of the UAV is represented as a rotation matrix. The objective function is constructed using the rotation matrix and sensor noise parameters as variables. The accuracy of attitude estimation and robustness to sensor noise are considered comprehensively. The objective function is then transformed into an optimization problem on the SO(3) Lie group.

[0045] Initialization operation: On the SO(3) Lie group, the optimization point is initialized based on the prior estimate of the initial attitude of the UAV (such as the initial attitude obtained by GPS and magnetic compass), and the momentum is initialized to the zero vector to set the initial state for the subsequent attitude optimization iteration process.

[0046] Network Training: The deep unfolded neural network architecture was trained using acquired sensor data and baseline pose data. The error between the network's output pose estimate and the baseline pose was calculated as the loss function. The gradient was solved using backpropagation, and the network parameters were adjusted using an optimization algorithm. Each optimization layer in the deep unfolded neural network simulates one iteration of pose estimation, incorporating gradient calculation, momentum update, and point update processes on the Lie algebra space into the network structure. Residual connections between adjacent optimization layers help capture long-term dependencies in pose changes, enhancing the network's optimization capabilities.

[0047] Practical Application: The trained deep unfolded neural network model is integrated into the flight control system of a UAV. During flight, the UAV collects sensor data in real time and inputs it into the model. The model quickly outputs optimized attitude estimation results, including precise roll, pitch, and yaw angles. The flight controller adjusts the flight attitude and controls the flight trajectory based on this optimized attitude information, enabling the UAV to maintain stable flight even in complex environments and under interference conditions, thus improving flight safety and mission execution accuracy.

[0048] Performance Comparison: Compared with commonly used Kalman filter attitude estimation methods and traditional numerical optimization-based attitude estimation methods, this machine learning optimization method based on mathematical models improves attitude estimation accuracy by 20% and 28%, respectively. In terms of sensor noise immunity, it improves the suppression of accelerometer noise by 35% and the correction capability for gyroscope drift by 42%, effectively improving the attitude estimation performance of UAVs and verifying the practicality and effectiveness of this method in the field of robot attitude estimation.

[0049] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A machine learning optimization method based on a mathematical model, characterized in that, Includes the following steps: S1. Initialize the optimization point on the Lie group manifold and initialize the momentum to the zero vector V0=0; wherein, the initial value of the optimization point is determined based on the geometric characteristics of the robotic data; S2. Repeat the convergence sub-step until the convergence condition is met. The convergence sub-step is as follows: S2.1 Gradient calculation, calculating the objective function. At the current optimization point gradient at By projecting the gradient onto the Lie algebra space g using the differential mapping from the Lie group to the Lie algebra, we obtain the Lie algebra gradient direction. ; S2.2 Momentum Update, Based on Lie Algebra Gradient Direction and the momentum of the previous step Update the current momentum according to the momentum update rules. , where β is the momentum coefficient and η is the learning rate; S2.3, Update, update the momentum V k The corresponding Lie algebra elements are mapped through an exponential map. Transform to a Lie group to generate new optimization points. ; S3, Deep Unfolded Network Construction Steps: Map the single-iteration optimization steps S2.1-S2.3 to a trainable optimization layer of the neural network; construct a deep unfolded neural network architecture by stacking N isomorphic optimization layers, wherein the network depth N and the intra-layer structural parameters are dynamically configured according to the complexity of the target mathematical model. In the deep unfolded neural network architecture, adjacent optimization layers are connected by residual connections. The specific input and output design of the optimization layer is as follows: the input layer receives data with geometric structure in robotics, including position, attitude and velocity information; the output layer outputs the optimized motion planning parameters, including trajectory point position and attitude. S4. Train the network end-to-end using a domain-specific dataset, and calculate the gradient of the loss function using the backpropagation algorithm; update the network parameters, including the learning rate η, momentum coefficient β, and mapping operator weights, through the optimizer. S5. Deploy the trained deep unfolded network on the target system, input data with geometric structure, and output optimized mathematical model parameters. The method can be applied to pose estimation and trajectory tracking problems in robotics. By adjusting the loss function and output layer design in the deep unfolded network architecture, it can effectively optimize different robotic problems.

2. The machine learning optimization method based on a mathematical model according to claim 1, characterized in that: In step S2.2, the momentum update rule in the adaptive optimization algorithm is adopted, wherein the momentum coefficient is dynamically adjusted according to the state information during the optimization process.

3. The machine learning optimization method based on a mathematical model according to claim 1, characterized in that: In step S4, the depth of the deep unfolded neural network architecture is determined based on the complexity of the mathematical model to be optimized, and the structural parameters of the optimization layer are determined based on the complexity.

4. The machine learning optimization method based on a mathematical model according to claim 1, characterized in that: In step S4, the parameters of the deep unfolded neural network are calculated using the backpropagation algorithm to calculate gradients and then adjusted using an optimization algorithm.

Images