Robot motion state prediction method and device, electronic equipment, and storage medium

Abstract

The embodiment of the application provides a kind of robot motion state prediction method and device, electronic equipment, storage medium, it is related to robot technical field.The method comprises: in response to speed control instruction, the original motion state data of robot system is acquired;According to original motion state data, original motion state sequence is extracted;According to original motion state sequence, attention motion state sequence is calculated;By robot motion state prediction model, attention motion state sequence is nonlinearly transformed, and target motion state sequence is obtained;State sequence conversion is carried out to original motion state sequence and target motion state sequence, and actual promotion state is obtained;According to speed control instruction, speed control feature is extracted, and according to actual promotion state, speed control feature and the parameter of robot motion state prediction model, motion state prediction is carried out, and prediction motion state data is obtained.The embodiment of the application can improve the accuracy of predicting robot motion state.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a method and apparatus for predicting robot motion states, electronic devices, and storage media. Background Technology

[0002] To analyze and control a robot's motion, it is often necessary to predict the robot's motion state (such as position and velocity). For example, artificial intelligence models can be used to predict the robot's motion state under specific force conditions, thereby applying appropriate forces to control the robot's motion.

[0003] However, the dynamics of robot systems typically exhibit significant nonlinear characteristics. For example, the relationship between the forces acting on a robot's arm and its velocity and position changes is quite complex. Current methods for predicting robot motion states struggle to accurately predict them, failing to meet the demands of precise robot control applications for executing designated actions.

[0004] Therefore, improving the accuracy of predicting robot motion states has become an urgent technical problem to be solved. Summary of the Invention

[0005] The main objective of this application is to provide a method, device, electronic device, and storage medium for predicting robot motion states, aiming to improve the accuracy of predicting robot motion states.

[0006] To achieve the above objectives, a first aspect of this application proposes a method for predicting the motion state of a robot, the method comprising: In response to the user's speed control command for the robot system, the original motion state data of the robot system is acquired; Feature sequences are extracted from the original motion state data to obtain the original motion state sequence; Attention is calculated based on the original motion state sequence to obtain the attention motion state sequence; The attention motion state sequence is nonlinearly transformed by a pre-constructed robot motion state prediction model to obtain the target motion state sequence. The robot motion state prediction model is used to perform state sequence transformation on the original motion state sequence and the target motion state sequence to obtain the actual improved state. Feature extraction is performed based on the speed control command to obtain speed control features. Then, motion state prediction is performed based on the actual lifting state, the speed control features, and the parameters of the robot motion state prediction model to obtain the predicted motion state data of the robot system.

[0007] In some embodiments, the number of attentional motion state sequences is multiple; The step of performing a nonlinear transformation on the attention motion state sequence using a pre-built robot motion state prediction model to obtain the target motion state sequence includes: A multi-head attention motion state sequence is obtained by multiplying the spliced ​​sequence of multiple attention motion state sequences with a preset projection weight parameter; The first motion state sequence is obtained by adding the normalized value of the multi-head attention motion state sequence to the original motion state sequence; The activation values ​​of the first motion state sequence are calculated based on multiple target activation functions of the robot motion state prediction model to obtain motion state activation values. The motion state activation matrix is ​​obtained by combining multiple motion state activation values ​​into a matrix. The second motion state sequence is obtained by adding the normalized value of the motion state activation matrix and the first motion state sequence, and the second motion state sequence is determined as the target motion state sequence.

[0008] In some embodiments, the step of calculating activation values ​​for the first motion state sequence based on multiple target activation functions of the robot motion state prediction model to obtain motion state activation values ​​includes: The target basis function value is obtained by calculating the basis function value of the first motion state sequence using the target basis function. The first motion state sequence is subjected to spline basis function calculation to obtain spline basis function values. Then, the spline function values ​​are obtained by weighted summation based on preset spline weight parameters and the spline basis function values. The motion state activation value is obtained by summing the target basis function value and the spline function value.

[0009] In some embodiments, the parameters of the robot motion state prediction model include a first operator matrix and a second operator matrix; The step of predicting the motion state of the robot system based on the actual lifting state, the speed control characteristics, and the parameters of the robot motion state prediction model, to obtain the predicted motion state data of the robot system, includes: The predicted lift state features are obtained by summing the product of the first operator matrix and the actual lift state, and the product of the second operator matrix and the speed control features. The predicted improved state features are feature-mapped according to the preset feature mapping weight matrix to obtain the predicted motion state features; The predicted motion state features are decoded to obtain the predicted motion state data of the robot system.

[0010] In some embodiments, the parameters of the robot motion state prediction model include a first operator matrix; After predicting the motion state of the robot system based on the actual lifting state, the speed control characteristics, and the parameters of the robot motion state prediction model, the method further includes: The mean square error is calculated based on the predicted motion state data and the actual motion state data corresponding to the predicted motion state data to obtain the mean square error loss value. The first loss function value is obtained by weighting and summing the preset first loss weight and the mean square error loss value. The spectral regularization loss function value is calculated based on the product of the preset second loss weight and the first operator matrix to obtain the second loss function value; The first loss function value and the second loss function value are weighted and summed to obtain the total loss function value, and the parameters of the robot motion state prediction model are adjusted according to the total loss function value.

[0011] In some embodiments, each of the predicted motion state data corresponds to a time step; After predicting the motion state of the robot system based on the actual lifting state, the speed control characteristics, and the parameters of the robot motion state prediction model, the method further includes: Obtain the actual motion state data corresponding to the predicted motion state data at the time step; The difference between the predicted motion state data and the actual motion state data at the time step is calculated to obtain the motion state deviation characteristics. The optimization function value is calculated based on the motion state deviation characteristics and the speed control characteristics to obtain the total optimization function value. By updating the speed control characteristics to minimize the total optimization function value, a target speed control characteristic is obtained, and the robot system is controlled according to the target speed control characteristic.

[0012] In some embodiments, the step of calculating the optimization function value based on the motion state deviation characteristics and the speed control characteristics to obtain the total optimization function value includes: The first sub-optimization value is obtained by multiplying the transpose vector of the motion state deviation feature, the preset first weight matrix, and the motion state deviation feature. The second sub-optimization value is obtained by weighted summation of the transpose vector of the motion state deviation feature at each time step, the preset second weight matrix, and the motion state deviation feature. The third sub-optimization value is obtained by multiplying the transpose vector of the speed control feature, the preset third weight matrix, and the speed control feature. The total optimization function value is obtained by summing the first sub-optimization value, the second sub-optimization value, and the third sub-optimization value.

[0013] To achieve the above objectives, a second aspect of this application provides a robot motion state prediction device, the device comprising: The motion state data acquisition module is used to acquire the original motion state data of the robot system in response to the user's speed control command to the robot system. The feature sequence extraction module is used to extract feature sequences from the original motion state data to obtain the original motion state sequence. An attention calculation module is used to perform attention calculation based on the original motion state sequence to obtain an attention motion state sequence. The nonlinear transformation module is used to perform a nonlinear transformation on the attention motion state sequence through a pre-built robot motion state prediction model to obtain the target motion state sequence; The state sequence conversion module is used to convert the original motion state sequence and the target motion state sequence through the robot motion state prediction model to obtain the actual improved state. The motion state prediction module is used to extract features according to the speed control command to obtain speed control features, and to predict the motion state according to the actual lifting state, the speed control features, and the parameters of the robot motion state prediction model to obtain the predicted motion state data of the robot system.

[0014] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0015] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0016] The robot motion state prediction method, device, electronic device, and storage medium proposed in this application acquire the original motion state data of the robot system in response to the user's speed control commands. Based on this data, an original motion state sequence is extracted, allowing for the extraction of key temporal features to understand the temporal continuity of the robot system's motion state changes. Attention is calculated from the original motion state sequence to obtain an attention-enhanced motion state sequence. A pre-constructed robot motion state prediction model is then used to perform a nonlinear transformation on this sequence to obtain a target motion state sequence. This dynamic weighting through the attention mechanism more accurately captures the spatial dynamic dependencies of different parts of the robot system, such as the influence of the connection between the joints of the robotic arm and the end effector on its motion state, leading to a deeper understanding of the robot's nonlinear motion. Finally, the robot motion state prediction model performs a state sequence transformation on the original and target motion state sequences to obtain the actual enhanced state. This integrates information from both the original and attention-enhanced states, rather than relying on a single motion state, thereby improving the accuracy of subsequent motion state predictions. Speed ​​control features are extracted based on speed control commands, and motion state prediction is performed based on the actual lifting state, speed control features, and parameters of the robot motion state prediction model. This allows for the capture of the spatial dynamic dependencies between different parts of the robot system, and then the prediction of the robot system's dynamic characteristics based on the user's control intent and the robot system's spatial motion state (i.e., the actual lifting state), rather than directly handling complex nonlinear relationships through a simple mechanical model. This enables high-precision prediction of the robot system's motion state, improving the accuracy of motion state prediction. Attached Figure Description

[0017] Figure 1 This is a flowchart of the robot motion state prediction method provided in the embodiments of this application; Figure 2 yes Figure 1 The flowchart for step 104 in the document; Figure 3 yes Figure 2 The flowchart for step 203 in the text; Figure 4 yes Figure 1 The flowchart for step 106 in the document; Figure 5 This is a flowchart of a robot motion state prediction method provided in another embodiment of this application; Figure 6 This is a flowchart of a robot motion state prediction method provided in another embodiment of this application; Figure 7 yes Figure 6The flowchart for step 603 in the document; Figure 8 This is a schematic diagram of the robot motion state prediction device provided in the embodiments of this application; Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0021] First, let's analyze some of the terms used in this application: Artificial Intelligence (AI) is a new technical science that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can be a simulation of the information processes of human consciousness and thought. AI can also be the theory, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. Basic AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning. This application can acquire and process relevant data based on AI technology.

[0022] A robotic system is an integrated whole comprised of a robot, its work object, and its environment. It includes mechanical systems, drive systems, control systems, and perception systems. A robot is an intelligent machine capable of semi-autonomous or fully autonomous operation. Robots possess some intelligent capabilities similar to humans or other living organisms, such as perception, planning, movement, and coordination.

[0023] Dynamics is the study of the relationship between force and motion. In robotic systems, dynamics can be used to analyze how much power the robot's controller needs to send (such as how much current to the motors) to produce the desired, precise movements of the robot's components (robotic arms, legs, etc.).

[0024] Operator: Also known as a symmetric operator. An operator is typically a mapping or function that acts on elements of one space to produce elements of another space. Essentially, an operator is a rule that transforms one or more inputs (which can be numbers, functions, or other mathematical objects) into an output. For example, any arithmetic operation (such as the addition operator) can be considered an operator.

[0025] Koopman operator: It is an operator that uses a linear system to approximate a nonlinear system. Its core idea is to map the evolution of a nonlinear dynamic system onto a function space, thereby linearizing the behavior of the system in the function space.

[0026] The robot motion state prediction method, device, electronic device, and storage medium provided in this application are specifically described through the following embodiments. First, the robot motion state prediction method in this application is described.

[0027] The robot motion state prediction method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms; the software can be an application that implements the robot motion state prediction method, but is not limited to the above forms.

[0028] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0029] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0030] Figure 1 This is an optional flowchart of the robot motion state prediction method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps 101 to 106.

[0031] Step 101: In response to the user's speed control command to the robot system, acquire the original motion state data of the robot system; Step 102: Extract feature sequences from the original motion state data to obtain the original motion state sequence; Step 103: Perform attention calculation based on the original motion state sequence to obtain the attention motion state sequence; Step 104: The attention motion state sequence is nonlinearly transformed using a pre-built robot motion state prediction model to obtain the target motion state sequence. Step 105: The original motion state sequence and the target motion state sequence are transformed into a state sequence using the robot motion state prediction model to obtain the actual lifting state; Step 106: Extract features based on speed control commands to obtain speed control features, and predict motion state based on actual lifting state, speed control features, and parameters of robot motion state prediction model to obtain predicted motion state data of robot system.

[0032] The beneficial effects of this application's embodiments include, but are not limited to: acquiring the original motion state data of the robot system in response to the user's speed control commands, and extracting the original motion state sequence from the original motion state data. This allows for the extraction of key temporal features from the original data, enabling an understanding of the temporal continuity of the robot system's motion state changes. Attention calculation is performed on the original motion state sequence to obtain an attention-enhanced motion state sequence. Furthermore, a pre-built robot motion state prediction model is used to perform a nonlinear transformation on the attention-enhanced motion state sequence to obtain the target motion state sequence. This allows for more accurate capture of the spatial dynamic dependencies of different parts of the robot system through dynamic weighting via the attention mechanism, such as the influence of the connection between the joints of the robotic arm and the end effector on its motion state, thus providing a deeper understanding of the robot's nonlinear motion. The robot motion state prediction model is used to perform state sequence transformation on the original motion state sequence and the target motion state sequence to obtain the actual enhanced state. This integrates the dual information of the original state and the attention-enhanced state, rather than relying solely on a single motion state, thereby improving the accuracy of subsequent motion state predictions. Speed ​​control features are extracted from the speed control commands, and motion state prediction is performed based on the actual enhanced state, speed control features, and parameters of the robot motion state prediction model. In this way, after capturing the spatial dynamic dependencies of different parts of the robot system, the dynamic characteristics of the robot system can be predicted based on the user's control intention and the spatial motion state of the robot system (i.e., the actual lifting state), instead of directly dealing with complex nonlinear relationships through a simple mechanical model. This allows for high-precision prediction of the robot system's motion state and improves the accuracy of predicting the robot's motion state.

[0033] In step 101 of some embodiments, the speed control command refers to a command sent by the user to control the motion state (such as speed) of the robot system. For example, if the robot system includes a robotic arm, the speed control command may be a command to adjust the joint speed of the robotic arm. For example, the user can issue commands to the robot system through the robot system's human-machine interface module (such as a control handle, operation panel, etc.) to generate the speed control command. Speed ​​control commands can also be generated in other ways, and are not limited to these.

[0034] In some embodiments, raw motion state data is data used to characterize the current motion state of the robot system. For example, in the case where the robot system includes a robotic arm, the raw motion state data may include the angles of the robotic arm's joints, the speeds of the joints, and the three-dimensional coordinates of the robotic arm's end effector. It should be noted that the end effector of the robotic arm refers to a device installed at the end of the robotic arm and used to directly interact with the environment to perform a specific task, such as a gripper or a welder.

[0035] In step 102 of some embodiments, the original motion state sequence is a feature sequence extracted from the original motion state data. In some embodiments, an original motion state feature vector can be extracted from the original motion state data, and this feature vector can be expanded into a matrix form to obtain an original motion state feature matrix. Then, the original motion state feature matrices of multiple time steps can be serialized to obtain the original motion state sequence. For example, the original motion state feature matrix is ​​defined as follows: , In the formula, This represents the feature matrix of the original motion state corresponding to time step t; Represents the feature vector of the original motion state Perform matrix expansion operations; This represents the Hadamard product, which is element-wise multiplication. Represents the identity matrix The transpose of .

[0036] In some embodiments, for example, the original motion state sequence is defined as shown in the following formula: , In the formula, Represents the original motion state sequence; This represents the feature matrix of the original motion state corresponding to time step t. Similarly, Indicates time step The corresponding original motion state feature matrix.

[0037] In step 103 of some embodiments, the attention motion state sequence is a feature sequence obtained by performing attention calculations based on the original motion state sequence. Specifically, the attention motion state sequence can be generated using a multi-head attention mechanism. In some embodiments, a query projection, a key projection, and a value projection can be generated based on the original motion state sequence, and then the attention motion state sequence can be calculated based on these three projections. For example, the query projection... , key projection Value projection .in, Represents the original motion state sequence. Indicates the query weight. Indicates key weight. Indicates the value weight.

[0038] In some embodiments, the attention motion state sequence is defined as shown in the following formula: , In the formula, Represents a sequence of attentional movement states; Indicates query projection; Indicates key projection; Indicates key projection The transpose of the matrix; Represents the projection of values; This represents the softmax function; Indicates the dimension of the query projection; It should be noted that the softmax function is a normalized exponential function used to enhance the correlation of key features. This represents the scaling factor, used to stabilize the gradient.

[0039] In step 104 of some embodiments, the robot motion state prediction model can specifically be a neural network model, such as the Kolmogorov-Arnold Transformer (KAT) model. In another embodiment, the robot motion state prediction model can also be other types of artificial intelligence models, which are not limited in this application. The target motion state sequence is a matrix sequence obtained by nonlinear transformation based on the attention motion state sequence. The target motion state sequence is used to characterize the nonlinear motion state of the robot system. As for the specific calculation process of the target motion state sequence, please refer to the detailed description of step 204 below, which will not be repeated here.

[0040] In step 105 of some embodiments, the actual lifted state is a state matrix obtained by transforming the original motion state sequence and the target motion state sequence. In some embodiments, the transformation of the original motion state sequence and the target motion state sequence can be performed using the Koopman operator. In another embodiment, the state sequence transformation can also be performed in other ways, and is not limited to these. In some embodiments, the actual lifted state can be obtained by concatenating vectors based on the original motion state sequence and the product of the target motion state sequence and the parameters of the robot motion state prediction model. For example, the actual lifted state is defined as follows: , In the formula, Indicates the actual improvement status; Represents the original motion state sequence; A function that represents a state sequence transformation of the original motion state sequence; The parameters represent the KAT model (i.e., the robot motion state prediction model); This represents the target motion state sequence. In this embodiment, the original motion state sequence... As an actual improvement status The first n elements are embedded, where n represents the number of vector elements of the original motion state sequence.

[0041] In step 106 of some embodiments, the speed control feature is a feature vector extracted from the speed control command. The predicted motion state data is the motion state predicted based on the actual lifting state, the speed control features, and the parameters of the robot motion state prediction model. The predicted motion state data is used to characterize the motion state of the robot system after receiving the speed control command. For example, when the original motion state data is the motion state data of the robot system's manipulator, the predicted motion state data may include, but is not limited to, the angles of the manipulator's joints, the speed of the joints, and the three-dimensional coordinate position of the manipulator's end effector.

[0042] Please see Figure 2 In some embodiments, the number of attention motion state sequences is multiple; Step 104 may include, but is not limited to, steps 201 through 204: Step 201: Multiply the spliced ​​sequence of multiple attention motion state sequences with the preset projection weight parameters to obtain a multi-head attention motion state sequence. Step 202: Add the normalized value of the multi-head attention motion state sequence to the original motion state sequence to obtain the first motion state sequence; Step 203: Calculate the activation values ​​of the first motion state sequence according to the multiple target activation functions of the robot motion state prediction model to obtain the motion state activation values, and combine the multiple motion state activation values ​​into a matrix to obtain the motion state activation matrix. Step 204: Add the normalized value of the motion state activation matrix to the first motion state sequence to obtain the second motion state sequence, and determine the second motion state sequence as the target motion state sequence.

[0043] The advantage of this embodiment lies in that a multi-head attention motion state sequence is obtained by multiplying the concatenated sequence of multiple attention motion state sequences with preset projection weight parameters. This allows for the parallel extraction of differentiated features from the state sequences through multi-head attention, enhancing the model's understanding of the dynamic relationships (such as nonlinear complex motion relationships) of the robot system. The first motion state sequence is obtained by adding the normalized value of the multi-head attention motion state sequence to the original motion state sequence. Activation values ​​are then calculated on the first motion state sequence using multiple target activation functions and combined to form a motion state activation matrix. This allows for hierarchical fitting of the nonlinear dynamic characteristics of the robot system. Finally, the target motion state sequence is obtained by adding the normalized value of the motion state activation matrix to the first motion state sequence. This allows for more accurate capture of the spatial dynamic dependencies between different parts of the robot system (such as the nonlinear transmission effect of changes in the joint angle of the robotic arm on the position of the end effector) through dynamic weighting via the attention mechanism, leading to a deeper understanding of the robot's nonlinear motion and thus improving the accuracy of predicting the robot's motion state.

[0044] In step 201 of some embodiments, the multi-head attention motion state sequence is a matrix sequence obtained by multiplying a concatenated sequence of attention motion state sequences corresponding to multiple attention heads with a preset projection weight parameter. In some embodiments, the attention motion state sequence corresponding to each attention head is defined as shown in the following formula: , In the formula, This represents the sequence of attentional motion states corresponding to the i-th attention head. Where h represents the number of attention heads, for example ; Indicates query projection; Indicates key projection; Represents the projection of values; This represents the query weight corresponding to the i-th attention head; This represents the key weight corresponding to the i-th attention head; This represents the weight value corresponding to the i-th attention head. It should be noted that... The definition is consistent with the definition formula of the attention movement state sequence above (see the specific description of step 103), and will not be repeated here.

[0045] In some embodiments, the multi-head attention motion state sequence is defined as shown in the following formula: , In the formula, This represents a sequence of motion states related to multi-head attention. Indicates query projection; Indicates key projection; Represents the projection of values; A concatenated sequence representing the attentional motion state sequences corresponding to multiple attention heads; This represents a vector concatenation operation; This represents the sequence of attentional motion states corresponding to the first attention head. Similarly, This represents the sequence of attentional motion states corresponding to the first attention head; This represents the projection weight parameter.

[0046] In step 202 of some embodiments, the first motion state sequence is a matrix sequence obtained by adding the normalized value of the multi-head attention motion state sequence to the original motion state sequence. In some embodiments, the multi-head attention motion state sequence and the original motion state sequence can be input into the KAT model (i.e., the robot motion state prediction model) to calculate the first motion state sequence. In some embodiments, the first motion state sequence is defined as shown in the following formula: , In the formula, This represents the first motion state sequence output by the j-th layer of the KAT model (i.e., the robot motion state prediction model). , where L represents the number of KAT layers in the KAT model; Normalized value representing the sequence of motion states in multi-head attention; Indicates the layer normalization function; This represents the multi-head attention motion state sequence output by the (j-1)th layer of the KAT model; This represents the second motion state sequence output by the (j-1)th layer of the KAT model.

[0047] In step 203 of some embodiments, the motion state activation value is an activation value calculated on the first motion state sequence based on multiple target activation functions of the robot motion state prediction model. The motion state activation matrix is ​​a matrix obtained by combining multiple motion state activation values.

[0048] In some embodiments, the motion state activation matrix is ​​defined as shown in the following formula: , In the formula, The activation matrix represents the motion state. This represents the activation transformation of the (L-1)th layer of the KAT model. This represents the activation transformation of the (L-2)th layer of the KAT model. and The meaning can be deduced by analogy, and will not be elaborated here; This represents the composition of functions; This represents the first motion state sequence output by the Lth layer of the KAT model; In some embodiments, the activation transformation refers to inputting each element of a matrix into an activation function. The activation transformation is defined as follows: , In the formula, This represents the activation transformation of the Lth layer of the KAT model; Indicates the target activation function passing through the Lth layer. The activation value of the motion state is calculated by the matrix element with index (1,1); Indicates the target activation function passing through the Lth layer. The activation value of the motion state is obtained by calculating the matrix element with index (1,2); Indicates the target activation function passing through the Lth layer. For index ( The motion state activation value is obtained by calculating the matrix elements of ), where, This represents the maximum value of the row index of the matrix. This represents the maximum value of the column index of the matrix; the meaning of the motion state activation value with other subscripts in the above formula can be deduced by analogy, and will not be repeated here.

[0049] It should be noted that the definition of motion state activation value can be found in the detailed description of steps 301 to 303 below, and will not be repeated here.

[0050] In step 204 of some embodiments, the second motion state sequence is a matrix sequence obtained by adding the normalized value of the motion state activation matrix to the first motion state sequence. The target motion state sequence is the second motion state sequence output by the last layer of the KAT model. In some embodiments, the second motion state sequence is defined as shown in the following formula: , In the formula, This represents the second motion state sequence output by the j-th layer of the KAT model; This represents the motion state activation matrix of the j-th layer output of the KAT model; This represents the first motion state sequence output by the j-th layer of the KAT model; The representation layer normalization function.

[0051] Please see Figure 3 In some embodiments, step 203, which involves calculating activation values ​​for the first motion state sequence based on multiple target activation functions of the robot motion state prediction model to obtain motion state activation values, may include, but is not limited to, steps 301 to 303: Step 301: Calculate the basis function values ​​of the first motion state sequence using the target basis function to obtain the target basis function values; Step 302: Calculate the spline basis function value of the first motion state sequence using spline basis functions to obtain the spline basis function value, and perform a weighted summation calculation based on the preset spline weight parameters and spline basis function value to obtain the spline function value; Step 303: Summate the target basis function value and spline function value to obtain the motion state activation value.

[0052] The advantage of this embodiment lies in that it calculates the target basis function value by applying the target basis function to the first motion state sequence. This allows the use of parameterized basis functions to capture global nonlinear features in the robot's motion state (such as the dynamic relationship between joint forces and acceleration), enhancing the model's ability to fit the robot's motion trend. Spline basis function values ​​are then calculated for the first motion state sequence, and a weighted sum is performed based on preset spline weight parameters and these values ​​to obtain the final spline function value. This weighted summation method allows the model to focus on the robot's nonlinear dynamic local characteristics. Finally, the motion state activation value is obtained by summing the target basis function value and the spline function value, enabling a more accurate fitting of nonlinear dynamic features to linear features in subsequent iterations, thereby improving the accuracy of predicting the robot's motion state.

[0053] In step 301 of some embodiments, the target basis function value is a basis function value calculated using the target basis function on the first motion state sequence. In some embodiments, the target basis function value is defined as follows: , In the formula, Indicates the value of the objective basis function; This represents the input value of the function, which is the first motion state sequence mentioned above; Represents the SiLU function; It represents the reciprocal of the natural exponential function.

[0054] It should be noted that the SiLU (Sigmoid-Weighted Linear Unit) function, also known as the Swish function, is an activation function.

[0055] In step 302 of some embodiments, the spline basis function value is a function value calculated using spline basis functions on the first motion state sequence. The spline function value is a function value obtained by weighted summation based on preset spline weight parameters and spline basis function values. In some embodiments, the spline function value is defined as shown in the following formula: , In the formula, Represents the spline function value; Let represent the weight parameter of the i-th spline, where ; This represents the value of the i-th spline basis function; Let represent the i-th k-th B-spline basis function constructed on a grid with G intervals.

[0056] In step 303 of some embodiments, the motion state activation value is the target basis function value. spline function values sum.

[0057] Please see Figure 4 In some embodiments, the parameters of the robot motion state prediction model include a first operator matrix and a second operator matrix; Step 106 may include, but is not limited to, steps 401 to 403: Step 401: Summing the product of the first operator matrix and the actual lift state, and the product of the second operator matrix and the velocity control feature, to obtain the predicted lift state feature; Step 402: Perform feature mapping on the predicted improved state features according to the preset feature mapping weight matrix to obtain the predicted motion state features; Step 403: Decode the predicted motion state features to obtain the predicted motion state data of the robot system.

[0058] The advantage of this embodiment lies in that the product of the first operator matrix and the actual lift state can characterize the key features of motion state changes, thus quantifying the nonlinear dynamic laws in the state sequence. The product of the second operator matrix and the velocity control features can quantify the influence of control commands. Furthermore, summing the above product results yields the predicted lift state features, thereby fusing state evolution laws with control inputs and enhancing the model's ability to represent the controlled response characteristics of the robot system. By performing feature mapping on the predicted lift state features using a preset weight matrix and decoding the mapped predicted motion state features, the fused features can be transformed into quantifiable predicted motion state data (such as the robot's joint velocities and positions), thereby improving the accuracy of predicting the robot's motion state.

[0059] In step 401 of some embodiments, the predicted lift-up state feature is the sum of the product of the first operator matrix and the actual lift-up state, and the product of the second operator matrix and the velocity control feature. In some embodiments, the predicted lift-up state feature is defined as follows: , In the formula, This represents the predicted improved state feature at time step k+1; Represents the first operator matrix; This indicates the actual boost state at time step k; Represents the second operator matrix; This represents the speed control characteristic at time step k.

[0060] In step 402 of some embodiments, the predicted motion state features are features obtained by mapping the predicted enhanced state features according to a preset feature mapping weight matrix. In some embodiments, the predicted motion state features are defined as shown in the following formula: , In the formula, This represents the predicted motion state features at time step k+1; This represents the predicted improved state feature at time step k+1; This represents the feature mapping weight matrix.

[0061] In some embodiments, the feature mapping weight matrix is ​​defined as shown in the following formula: , In the formula, Represents the feature mapping weight matrix; This represents the identity matrix of order n.

[0062] In step 403 of some embodiments, the predicted motion state data is data obtained by decoding the predicted motion state features. The meaning and function of the predicted motion state data have been described in detail above (refer to the specific description of step 106), and will not be repeated here.

[0063] Please see Figure 5 In some embodiments, the parameters of the robot motion state prediction model include a first operator matrix; After step 106, the robot motion state prediction method may also include, but is not limited to, steps 501 to 504: Step 501: Calculate the mean square error based on the predicted motion state data and the actual motion state data corresponding to the predicted motion state data to obtain the mean square error loss value. Step 502: Perform a weighted summation based on the preset first loss weight and the mean squared error loss value to obtain the first loss function value; Step 503: Calculate the spectral regularization loss function value based on the product of the preset second loss weight and the first operator matrix to obtain the second loss function value; Step 504: The first loss function value and the second loss function value are weighted and summed to obtain the total loss function value, and the parameters of the robot motion state prediction model are adjusted according to the total loss function value.

[0064] The advantage of this embodiment lies in that it calculates the mean squared error loss value based on the predicted motion state data and the corresponding actual motion state data. This quantifies the error in model prediction and provides a clear indicator for model optimization. The mean squared error loss value is weighted and summed according to a preset first loss weight to obtain the first loss function value. This balances the contribution of prediction error to the total loss, preventing a single error indicator from dominating the optimization process. The spectral regularization loss function value is calculated based on the product of the second loss weight and the first operator matrix. This constrains the spectral norm of the first operator matrix, suppressing model complexity and reducing the risk of overfitting. Then, the first loss function value and the second loss function value are weighted and summed to obtain the total loss function value. The model parameters are adjusted accordingly, thereby improving the prediction accuracy of the robot motion state prediction model by training it, and ultimately improving the accuracy of predicting the robot's motion state using this model.

[0065] In step 501 of some embodiments, the mean squared error loss value is the mean squared error calculated based on the predicted motion state data and the actual motion state data corresponding to the predicted motion state data.

[0066] In step 502 of some embodiments, the first loss function value is a weighted sum of the first loss weight and the mean squared error loss value. In some embodiments, the first loss function value is defined as follows: , In the formula, This represents the value of the first loss function; This represents the weight decay hyperparameter; Indicates the weight decay hyperparameter k-1 power; This represents the mean squared error loss value; This represents the Mean-Square Error (MSE) function; This indicates the predicted improved state features; This represents the actual motion state data.

[0067] In step 503 of some embodiments, the second loss function value is a spectral regularization loss function value calculated based on the product of a preset second loss weight and a first operator matrix. In some embodiments, the second loss function value is defined as follows: , In the formula, This represents the value of the second loss function; This represents the weight of the i-th second loss. ; Let represent the first operator matrix.

[0068] In step 504 of some embodiments, the total loss function value is the sum of the first loss function value and the second loss function value. In some embodiments, the total loss function value is defined as... ,in, This represents the total loss function value. This refers to user-defined hyperparameters. This represents the value of the first loss function. This represents the value of the second loss function.

[0069] Please see Figure 6 In some embodiments, each predicted motion state data corresponds to a time step; After step 106, the robot motion state prediction method may also include, but is not limited to, steps 601 to 604: Step 601: Obtain the actual motion state data corresponding to the predicted motion state data of the time step; Step 602: Calculate the difference between the predicted motion state data and the actual motion state data based on the characteristics of the time step to obtain the motion state deviation characteristics. Step 603: Calculate the optimization function value based on the motion state deviation characteristics and speed control characteristics to obtain the total optimization function value; Step 604: Obtain the target speed control feature by updating the speed control feature to minimize the total optimization function value, and control the robot system according to the target speed control feature.

[0070] The advantage of this embodiment lies in acquiring the actual motion state data corresponding to the predicted motion state data at each time step, and calculating the motion state deviation feature by subtracting the predicted motion state data from the actual motion state data based on their characteristics. This quantifies the error between the prediction result and the actual state, providing precise feedback signals for optimizing control commands. The overall optimization function value is calculated based on the motion state deviation feature and the speed control feature. This allows the establishment of an evaluation function with prediction error and control input as joint optimization objectives to measure the control effect of the current control command on the robot's motion state. By updating the speed control feature to minimize the overall optimization function value, the target speed control feature is obtained, thereby dynamically adjusting the control command to compensate for the prediction deviation. This allows the calculation of the appropriate command using the predicted motion state of the robot system, precisely controlling the robot to execute specified actions (such as controlling the speed and trajectory of the robotic arm), thereby improving the accuracy and reliability of the robot system control.

[0071] In step 601 of some embodiments, the real motion state data is the actual motion data of the robot system at that time step, such as the actual joint speed and joint angle of the robot system's robotic arm.

[0072] In step 602 of some embodiments, the motion state deviation feature is the difference between the features of the predicted motion state data at the time step and the features of the actual motion state data.

[0073] In step 603 of some embodiments, the total optimization function value is an optimization function value calculated based on motion state deviation characteristics and speed control characteristics.

[0074] In step 604 of some embodiments, the target speed control feature is the speed control feature that minimizes the total optimization function value.

[0075] Please see Figure 7 In some embodiments, step 603 may include, but is not limited to, steps 701 to 704: Step 701: Multiply the transpose vector of the motion state deviation feature, the preset first weight matrix, and the motion state deviation feature to obtain the first sub-optimization value; Step 702: The second sub-optimization value is obtained by weighted summation of the transpose vector of the motion state deviation features of each time step, the preset second weight matrix, and the motion state deviation features. Step 703: Multiply the transpose of the speed control feature, the preset third weight matrix, and the speed control feature to obtain the third sub-optimization value; Step 704: Summate the first sub-optimal value, the second sub-optimal value, and the third sub-optimal value to obtain the total optimization function value.

[0076] The advantage of this embodiment lies in that, by multiplying the transpose vector of the motion state deviation feature, a preset first weight matrix, and the motion state deviation feature, a first sub-optimization value is obtained. This quantifies the direct impact of the predicted state deviation at the current time step on system control. A second sub-optimization value is obtained by weighted summation of the transpose vector of the motion state deviation feature at each time step, a preset second weight matrix, and the motion state deviation feature. This comprehensively evaluates the cumulative prediction deviation of historical time steps, strengthening the constraint on the continuity of motion state during the optimization process. A third sub-optimization value is obtained by multiplying the transpose vector of the speed control feature, a preset third weight matrix, and the speed control feature. This quantifies the error of the control command. The summation of the first, second, and third sub-optimization values ​​yields the total optimization function value. This establishes an evaluation function with prediction error and control input as joint optimization objectives to measure the control effect of the current control command on the robot's motion state, thereby improving the accuracy and reliability of the robot system control.

[0077] In step 701 of some embodiments, the first sub-optimization value is the product of the transpose of the motion state deviation feature, a preset first weight matrix, and the motion state deviation feature. See the detailed description of step 704 below. This represents the first sub-optimal value.

[0078] In step 702 of some embodiments, the second sub-optimization value is the transpose vector of the motion state deviation features at each time step, a preset second weight matrix, and a weighted sum of the motion state deviation features. See the detailed description of step 704 below. This represents the second sub-optimal value.

[0079] In step 703 of some embodiments, the third sub-optimization value is the product of the transpose of the speed control feature, a preset third weight matrix, and the speed control feature. See the detailed description of step 704 below. This represents the third sub-optimal value.

[0080] In step 704 of some embodiments, the total optimization function value is the sum of the first sub-optimization value, the second sub-optimization value, and the third sub-optimization value. In some embodiments, the total optimization function value is defined as shown in the following formula: , In the formula, This represents the total optimal function value; Indicates time step The feature vector of the predicted motion state data; This represents the maximum value at the time step. Indicates time step Real motion state data; Indicates time step Motion state deviation characteristics The transpose of ; This represents the first weight matrix; Represents the motion state deviation characteristics at time step k The transpose of ; This represents the second weight matrix; Represents the velocity control characteristics at time step k. The transpose of ; This represents the third weight matrix.

[0081] In some embodiments, the constraints on the total optimal function value are: , In the formula, This represents the predicted improved state feature at time step k+1, where ; Represents the first operator matrix; This represents the predicted improved state feature at time step k; Represents the second operator matrix; This represents the speed control characteristic at time step k.

[0082] Please see Figure 8 This application also provides a robot motion state prediction device, which can implement the above-described robot motion state prediction method. The device includes: The motion state data acquisition module 801 is used to acquire the original motion state data of the robot system in response to the user's speed control command to the robot system. The feature sequence extraction module 802 is used to extract feature sequences from the original motion state data to obtain the original motion state sequence. The attention calculation module 803 is used to perform attention calculation based on the original motion state sequence to obtain the attention motion state sequence. The nonlinear transformation module 804 is used to perform a nonlinear transformation on the attention motion state sequence through a pre-built robot motion state prediction model to obtain the target motion state sequence. The state sequence conversion module 805 is used to convert the original motion state sequence and the target motion state sequence through the robot motion state prediction model to obtain the actual lift state. The motion state prediction module 806 is used to extract features based on the speed control command to obtain speed control features, and to predict the motion state based on the actual lifting state, speed control features, and parameters of the robot motion state prediction model to obtain the predicted motion state data of the robot system.

[0083] In one embodiment, the parameters of the robot motion state prediction model include a first operator matrix; the robot motion state prediction device further includes a model training module, used for: The mean squared error is calculated based on the predicted motion state data and the corresponding actual motion state data to obtain the mean squared error loss value. The mean squared error loss value is then weighted and summed according to the preset first loss weight to obtain the first loss function value. The spectral regularization loss function value is calculated based on the product of the preset second loss weight and the first operator matrix to obtain the second loss function value. The first loss function value and the second loss function value are then weighted and summed to obtain the total loss function value. The parameters of the robot motion state prediction model are then adjusted according to the total loss function value.

[0084] In one embodiment, each predicted motion state data corresponds to a time step; the robot motion state prediction device further includes an optimization function calculation module, used for: Obtain the actual motion state data corresponding to the predicted motion state data at each time step; calculate the difference between the features of the predicted motion state data and the features of the actual motion state data at each time step to obtain the motion state deviation features; calculate the optimization function value based on the motion state deviation features and the speed control features to obtain the total optimization function value; obtain the target speed control features by updating the speed control features to minimize the total optimization function value, and control the robot system according to the target speed control features.

[0085] The specific implementation of this robot motion state prediction device is basically the same as the specific implementation of the robot motion state prediction method described above, and will not be repeated here.

[0086] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described robot motion state prediction method. This electronic device can include any smart terminal such as a tablet computer or an in-vehicle computer.

[0087] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the robot motion state prediction method of the embodiments of this application. The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0088] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described robot motion state prediction method.

[0089] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0090] It should be noted that the software tools or components not belonging to our company that appear in the embodiments of this application are merely examples and do not represent actual use.

[0091] The embodiments described in this application are intended to more clearly illustrate the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0092] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0093] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0094] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0095] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0096] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0097] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, or indirect coupling or communication connection between the apparatus or units, and may be electrical, mechanical, or other forms.

[0098] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0099] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0100] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for predicting the motion state of a robot, characterized in that, The method includes: In response to the user's speed control command for the robot system, the original motion state data of the robot system is acquired; Feature sequences are extracted from the original motion state data to obtain the original motion state sequence; Attention is calculated based on the original motion state sequence to obtain the attention motion state sequence; The attention motion state sequence is nonlinearly transformed by a pre-constructed robot motion state prediction model to obtain the target motion state sequence. The robot motion state prediction model is used to perform state sequence transformation on the original motion state sequence and the target motion state sequence to obtain the actual improved state. Feature extraction is performed based on the speed control command to obtain speed control features. Then, motion state prediction is performed based on the actual lifting state, the speed control features, and the parameters of the robot motion state prediction model to obtain the predicted motion state data of the robot system. The parameters of the robot motion state prediction model include a first operator matrix and a second operator matrix. The step of predicting the motion state of the robot system based on the actual lifting state, the speed control characteristics, and the parameters of the robot motion state prediction model, to obtain the predicted motion state data of the robot system, includes: The predicted lift state features are obtained by summing the product of the first operator matrix and the actual lift state, and the product of the second operator matrix and the speed control features. The predicted improved state features are feature-mapped according to the preset feature mapping weight matrix to obtain the predicted motion state features; The predicted motion state features are decoded to obtain the predicted motion state data of the robot system.

2. The method according to claim 1, characterized in that, The number of attention movement state sequences is multiple; The step of performing a nonlinear transformation on the attention motion state sequence using a pre-built robot motion state prediction model to obtain the target motion state sequence includes: A multi-head attention motion state sequence is obtained by multiplying the spliced ​​sequence of multiple attention motion state sequences with a preset projection weight parameter; The first motion state sequence is obtained by adding the normalized value of the multi-head attention motion state sequence to the original motion state sequence; The activation values ​​of the first motion state sequence are calculated based on multiple target activation functions of the robot motion state prediction model to obtain motion state activation values. The motion state activation matrix is ​​obtained by combining multiple motion state activation values ​​into a matrix. The second motion state sequence is obtained by adding the normalized value of the motion state activation matrix and the first motion state sequence, and the second motion state sequence is determined as the target motion state sequence.

3. The method according to claim 2, characterized in that, The step of calculating activation values ​​for the first motion state sequence based on multiple target activation functions of the robot motion state prediction model to obtain motion state activation values ​​includes: The target basis function value is obtained by calculating the basis function value of the first motion state sequence using the target basis function. The first motion state sequence is subjected to spline basis function calculation to obtain spline basis function values. Then, the spline function values ​​are obtained by weighted summation based on preset spline weight parameters and the spline basis function values. The motion state activation value is obtained by summing the target basis function value and the spline function value.

4. The method according to any one of claims 1 to 3, characterized in that, The parameters of the robot motion state prediction model include the first operator matrix; After predicting the motion state of the robot system based on the actual lifting state, the speed control characteristics, and the parameters of the robot motion state prediction model, the method further includes: The mean square error is calculated based on the predicted motion state data and the actual motion state data corresponding to the predicted motion state data to obtain the mean square error loss value. The first loss function value is obtained by weighting and summing the preset first loss weight and the mean square error loss value. The spectral regularization loss function value is calculated based on the product of the preset second loss weight and the first operator matrix to obtain the second loss function value; The first loss function value and the second loss function value are weighted and summed to obtain the total loss function value, and the parameters of the robot motion state prediction model are adjusted according to the total loss function value.

5. The method according to any one of claims 1 to 3, characterized in that, Each of the predicted motion state data corresponds to a time step; After predicting the motion state of the robot system based on the actual lifting state, the speed control characteristics, and the parameters of the robot motion state prediction model, the method further includes: Obtain the actual motion state data corresponding to the predicted motion state data at the time step; The difference between the predicted motion state data and the actual motion state data at the time step is calculated to obtain the motion state deviation characteristics. The optimization function value is calculated based on the motion state deviation characteristics and the speed control characteristics to obtain the total optimization function value. By updating the speed control characteristics to minimize the total optimization function value, a target speed control characteristic is obtained, and the robot system is controlled according to the target speed control characteristic.

6. The method according to claim 5, characterized in that, The step of calculating the optimization function value based on the motion state deviation characteristics and the speed control characteristics to obtain the total optimization function value includes: The first sub-optimization value is obtained by multiplying the transpose vector of the motion state deviation feature, the preset first weight matrix, and the motion state deviation feature. The second sub-optimization value is obtained by weighted summation of the transpose vector of the motion state deviation feature at each time step, the preset second weight matrix, and the motion state deviation feature. The third sub-optimization value is obtained by multiplying the transpose vector of the speed control feature, the preset third weight matrix, and the speed control feature. The total optimization function value is obtained by summing the first sub-optimization value, the second sub-optimization value, and the third sub-optimization value.

7. A robot motion state prediction device, characterized in that, The device includes: The motion state data acquisition module is used to acquire the original motion state data of the robot system in response to the user's speed control command to the robot system. The feature sequence extraction module is used to extract feature sequences from the original motion state data to obtain the original motion state sequence. An attention calculation module is used to perform attention calculation based on the original motion state sequence to obtain an attention motion state sequence. The nonlinear transformation module is used to perform a nonlinear transformation on the attention motion state sequence through a pre-built robot motion state prediction model to obtain the target motion state sequence; The state sequence conversion module is used to convert the original motion state sequence and the target motion state sequence through the robot motion state prediction model to obtain the actual improved state. The motion state prediction module is used to extract features according to the speed control command to obtain speed control features, and to predict the motion state according to the actual lifting state, the speed control features, and the parameters of the robot motion state prediction model to obtain the predicted motion state data of the robot system. The parameters of the robot motion state prediction model include a first operator matrix and a second operator matrix. The step of predicting the motion state according to the actual lifting state, the speed control features, and the parameters of the robot motion state prediction model to obtain the predicted motion state data of the robot system includes: summing the product of the first operator matrix and the actual lifting state, and the product of the second operator matrix and the speed control features to obtain the predicted lifting state features; performing feature mapping on the predicted lifting state features according to a preset feature mapping weight matrix to obtain the predicted motion state features; and decoding the predicted motion state features to obtain the predicted motion state data of the robot system.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the robot motion state prediction method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the robot motion state prediction method according to any one of claims 1 to 6.

Images