Mobile mechanical arm collision detection method based on composite Transform model

Abstract

The invention relates to a mobile mechanical arm collision detection method based on a composite Transform model. According to the system, a sensing module of a mobile mechanical arm body is used. The method specifically comprises the following steps that 1, sensor data, including torque, angle, speed, acceleration, position and the like, of different positions on the mobile mechanical arm are collected, and the data are preprocessed through the abnormal peak value recognition technology based on rules; and 2, efficient time sequence analysis is performed on the preprocessed collision data by adopting a Transform model, and accurate classification of collision types is realized. According to the method, the collision detection accuracy and efficiency of the mechanical arm can be improved in a complex dynamic environment, and meanwhile, the operation safety of the robot is guaranteed. Through the application of neural network training learning, the defects of a traditional method in the collision detection process in the dynamic environment are overcome, and the method has high adaptability and practicability and is suitable for various types of mechanical arms and complex operation environments.

Description

Technical Field

[0001] The invention relates to the field of robot technology, in particular to a mobile robot arm collision detection method based on a composite Transformer model. Background Art

[0002] With the rapid development of robotics technology, mobile robotic arms are increasingly used in industrial automation, intelligent manufacturing and other fields. Especially in collaborative operations in dynamic environments, mobile robotic arms have attracted widespread attention due to their flexibility and efficiency. However, with the increasing demand for the application of mobile robotic arms in complex dynamic environments, mobile robotic arm collision detection has become an important research direction. Collision events not only affect the work efficiency of robots, but may also cause equipment damage or personal injury. Therefore, how to efficiently and accurately identify and respond to collision events in a real-time environment has become a major challenge in current research.

[0003] Existing collision detection methods mainly rely on physical sensors (such as torque sensors, IMU sensors, etc.) to monitor the motion state of the robot arm and determine whether a collision occurs through threshold setting or rule engines. Although these traditional methods can work effectively in some static environments, they often face problems of poor adaptability and insufficient accuracy when facing dynamic and complex environments. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a mobile robotic arm collision detection method based on a composite Transformer model, which effectively improves the robot's collision detection capability in complex environments and provides a new solution for achieving efficient and safe robot operation.

[0005] To achieve the above object, the present invention adopts the following technical solution: a mobile manipulator collision detection method based on a composite Transformer model, comprising the following steps:

[0006] Step 1: Obtain collision data; during the operation of the robot, real-time data from sensors including torque sensors, IMU sensors, and speed sensors are collected as pre-training time set data packets for collision detection;

[0007] Step 2: Data preprocessing: preprocess the collected sensor data packets, including denoising and normalization operations, and convert the data into target format data packets suitable for input into the model;

[0008] Step 3: LSTM layer processing, using the LSTM layer to process the sequence data and map the input data into the hidden state space to capture the temporal dependencies in the sequence;

[0009] Step 4: Collision type classification: input the preprocessed data into the Transformer model, extract the temporal features in the data through the self-attention mechanism, and realize the classification of collision types.

[0010] In a preferred embodiment, the input data of the Transformer model is a feature of dimension [N×T×input_size], where N represents the batch size of the data, T represents the time step, and input_size represents the feature dimension of each time step;

[0011] The dimension of the input feature is [N×T×input_size]. The feature is first processed by the LSTM layer. The LSTM captures the long-term and short-term dependencies in the time series through its internal gate mechanism. The output dimension of the LSTM layer is [N×T×hidden_size], where hidden_size represents the hidden layer dimension of the LSTM.

[0012] Next, the output of LSTM is passed to the self-attention layer, which extracts global context information by calculating the similarity between different time steps in the sequence; each query Query, key Key, and value Value matrix in the self-attention mechanism is obtained through the linear transformation nn.Linear; through attention calculation and weighted summation, a weighted output with the dimension [N×T×hidden_size] is finally obtained.

[0013] In a preferred embodiment, the step 2 specifically includes the following steps:

[0014] Step 21: Sensor Signal Filtering:

[0015] The sensor signal is collected using Kalman Filter to remove noise:

[0016]

[0017] in, is the predicted value of the system state, is the estimated value at time k-1, A is the state transfer matrix of the system, B is the control input matrix, u k is the control input, w k is the process noise;

[0018] Step 22: Data Normalization:

[0019] Use Min-Max Normalization: linearly map the data to the [0,1] interval:

[0020]

[0021] Where x is a characteristic value of the original data; x min is the minimum value of the feature; x max is the maximum value of the feature; x' is the normalized value; a and b are the lower and upper bounds of the interval respectively;

[0022] Step 23: Denoising: Use a low pass filter to remove high frequency noise

[0023]

[0024] Where H(f) is the frequency response of the filter, f is the frequency, and f c is the cut-off frequency and n is the order of the filter.

[0025] In a preferred embodiment, in step 3, the solution for processing local time series features of the LSTM model includes the following steps and specific formulas:

[0026] The model receives time series data, and the input feature dimension is X∈R N×T×D , where N is the batch size, T is the sequence length, and D is the feature dimension at each time step;

[0027] Given an input x at time step t t , the LSTM calculation process includes the following steps:

[0028] Input Gate InputGate:

[0029] i t =σ(W i x t +U i h t-1 +b i )

[0030] ForgetGate:

[0031] f t =σ(W f x t +U f h t-1 +b f )

[0032] Output Gate OutputGate:

[0033] o t =σ(W o x t +U o h t-1 +b o )

[0034] CellStateUpdate:

[0035] c t =f t ·c t-1 +i t tanh(W c xt+U c h t-1 +b c )

[0036] Hidden state update HiddenStateUpdate:

[0037] h t =o t ·tanh(c t )

[0038] in:

[0039] σ is the Sigmoid activation function;

[0040] tanh is the hyperbolic tangent activation function;

[0041] W i , W f , W o , W c and U i , U i , U o , U c is the learned weight matrix;

[0042] b i , b f , b o , b c is the bias term.

[0043] In a preferred embodiment, in step 4, the scheme of obtaining the collision result by the Transformer self-attention mechanism includes the following steps and specific formulas:

[0044] Step 41: Dataset time judgment rule: Based on the IMU acceleration, angular velocity, displacement / angle change of the travel meter sensor and torque change of the joint torque sensor, define a comprehensive collision judgment index C(t):

[0045] C(t)=w 1 ||a(t)||+w 2 ||ω(t)||+w 3 ||Δτ(t)||+w 4 ||Δθ(t)||+…

[0046] Among them, ||a(t)|| is the modulus of acceleration, ||ω(t)|| is the modulus of angular velocity, ||Δτ(t)|| is the modulus of torque change, ||Δθ(t)|| is the modulus of angle change, and w 1 、w 2 、w 3 、w 4 is the weight coefficient; if C(t) exceeds a certain threshold, it is determined as a collision event;

[0047] Step 42: The self-attention mechanism calculation formula of the Transformer model is:

[0048]

[0049] Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key; through the self-attention mechanism, the Transformer model calculates the relationship between different time steps to classify the collision type;

[0050] Step 43: Position encoding. The Transformer model injects sequence information through position encoding. The formula is:

[0051]

[0052] Among them, pos is the position, i is the dimension, d is the dimension size of the encoding, PE (pos,2i) PE (pos,2i+1) They correspond to the even and odd dimensions in the position encoding vector, respectively.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. The Transformer model combined with LSTM is adopted. The LSTM layer is used to process the input sequence, capture local time dependencies, and retain short-term and long-term memories. Then, it is used as the input of the Transformer through its time series data processing capabilities. It can more accurately analyze the time series features in the collision data and achieve more accurate collision type classification.

[0055] 2. Through multi-model fusion, not only the accuracy of collision detection is improved, but also the robot can respond more flexibly in dynamic and complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Attached Figure 1 It is a schematic diagram of the collision detection system flow of the present invention;

[0057] Attached Figure 2 It is a schematic diagram of the hardware system composition of the mobile mechanical arm of the present invention;

[0058] Attached Figure 3 It is a structural diagram of the Transformer model used in the present invention. DETAILED DESCRIPTION

[0059] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0060] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0061] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or their combinations.

[0062] The present invention provides a mobile manipulator collision detection method based on a composite Transformer model, referring to Figure 1-3 The robot arm components include an end flange 1, a robot arm 2 and a mobile chassis 3.

[0063] Step 1: Obtain collision data and data preprocessing. During the operation of the robot, data from sensors including torque sensors, IMU sensors, and speed sensors are collected in real time as a pre-training time collection data packet for collision detection.

[0064] Step 2: Data preprocessing: preprocess the collected sensor data packets, including denoising, normalization and other operations, to convert the data into a target format data packet suitable for input into the model.

[0065] Step 3: LSTM layer processing, use the LSTM (Long Short-Term Memory Network) layer to process the sequence data, map the input data into the hidden state space, and thus capture the temporal dependencies in the sequence.

[0066] Step 4: Collision type classification: input the preprocessed data into the Transformer model, extract the temporal features in the data through the self-attention mechanism, and classify the collision type. This step can effectively capture the temporal dependency between robot movements and collisions, and improve the accuracy of collision detection.

[0067] In this process, the Transformer model uses the self-attention mechanism to identify and process the dependencies between data at different time steps, thereby enhancing the model's ability to perceive collision events in a dynamic environment. Figure 2 shown.

[0068] Steps 3 to 4 include a model for mobile robot collision detection classification, specifically:

[0069] The Transformer backbone network module, including the combination of LSTM layer and self-attention mechanism, is used to process time series data and realize the detection and classification of collision types. The module first processes the input time series features through the LSTM layer to capture the temporal dependency information. Then, the self-attention mechanism is used to weight the output of the LSTM, and finally the classification task is realized through the fully connected layer.

[0070] Specifically, if Figure 3 As shown in the figure, the input data of the Transformer backbone network module is a feature of dimension [N×T×input_size], where N represents the data batch size, T represents the time step, and input_size represents the feature dimension of each time step.

[0071] The dimension of the input features is [N×T×input_size], which are first processed by the LSTM layer, which captures the long-term and short-term dependencies in the time series through its internal gate mechanism. The output dimension of the LSTM layer is [N×T×hidden_size], where hidden_size represents the hidden layer dimension of the LSTM.

[0072] Next, the output of the LSTM is passed to the self-attention layer, which extracts global context information by calculating the similarity between different time steps in the sequence. Each query, key, and value matrix in the self-attention mechanism is obtained through a linear transformation (nn.Linear). Through attention calculation and weighted summation, a weighted output with a dimension of [N×T×hidden_size] is finally obtained.

[0073] Finally, the features processed by the self-attention layer are input to the fully connected layer. In order to enhance the nonlinear expression ability of the model, the present invention designs two fully connected layers. The first layer expands the dimension of the hidden layer from hidden_size to hidden_size*2 and applies the ReLU activation function; the second layer compresses the feature dimension from hidden_size*2 to the number of collision type categories (num_classes), and the output dimension is [N×num_classes].

[0074] The following is the detailed process and main formula:

[0075] Input data: time series feature data, dimension is [N×T×input_size], where N is the batch size, T is the time step, and input_size is the feature dimension of each time step.

[0076] LSTM layer: captures dependencies in time series data. The feature dimension of LSTM output is [N×T×hidden_size].

[0077] Self-attention mechanism: The self-attention layer performs a weighted sum of the LSTM outputs to calculate the relationship between time steps, and the output dimension is [N×T×hidden_size].

[0078] Fully connected layer: Through two fully connected layers, the first layer expands the dimension from hidden_size to hidden_size*2, and the second layer compresses the dimension to num_classes, and finally outputs the classification result of each sample with a dimension of [N×num_classes].

[0079] In step 2, the data pre-processing solution mainly includes the following steps and specific formulas:

[0080] 1) Sensor signal filtering:

[0081] The sensor signal is collected using Kalman Filter to remove noise:

[0082]

[0083] in, is the estimated value of the system state, A is the state transfer matrix of the system, B is the control input matrix, u k is the control input, w k is the process noise.

[0084] 2) Data normalization:

[0085] The data used contains different types of sensor data. In order to eliminate the dimension difference, normalization is required. This also improves the stability of numerical calculation, simplifies feature selection, accelerates model training, and improves the effect of the algorithm. Here, Min-Max Normalization is used: linearly map the data to the [0,1] interval:

[0086]

[0087] Where x is a characteristic value of the original data. min is the minimum value of the feature. max is the maximum value of the feature. x' is the normalized value. a and b are the lower and upper bounds of the interval respectively.

[0088] 3) Denoising: Use a low-pass filter to remove high-frequency noise

[0089]

[0090] Where H(f) is the frequency response of the filter, f is the frequency, and f c is the cut-off frequency and n is the order of the filter.

[0091] In step 3, the solution for processing local time series features of the LSTM model includes the following steps and specific formulas:

[0092] The model receives time series data, and the input feature dimension is X∈R N×T×D , where N is the batch size, T is the sequence length, and D is the feature dimension at each time step.

[0093] Given an input x at time step t t , the LSTM calculation process includes the following steps:

[0094] Input Gate:

[0095] i t =σ(W i x t +U i h t-1 +b i )

[0096] ForgetGate:

[0097] f t =σ(W f x t +U f h t-1 +b f )

[0098] Output Gate:

[0099] o t =σ(W o x t +U o h t-1 +b o )

[0100] Cell State Update:

[0101] c t =f t ·c t-1 +i t tanh(W c xt+U c h t-1 +b c )

[0102] Hidden StateUpdate:

[0103] h t =o t ·tanh(c t )

[0104] in:

[0105] σ is the Sigmoid activation function.

[0106] tanh is the hyperbolic tangent activation function.

[0107] W i , W f , W o , W c and U i , U i , U o , U c is the learned weight matrix.

[0108] b i 、b f 、b o 、b c is the bias term.

[0109] In step 4, the Transformer self-attention mechanism processes the collision result, which includes the following steps and specific formulas:

[0110] 1) Dataset time judgment rule: Based on the displacement / angle changes of IMU acceleration, angular velocity, travel meter and other sensors and the torque changes of joint torque sensors, a comprehensive collision judgment indicator is defined:

[0111] C(t)=w 1 ||a(t)||+w 2 ||ω(t)||+w 3 ||Δτ(t)||+w 4 ||Δθ(t)||+…

[0112] Among them, ||a(t)|| is the modulus of acceleration, ||ω(t)|| is the modulus of angular velocity, ||Δτ(t)|| is the modulus of torque change, ||Δθ(t)|| is the modulus of angle change, and w 1 、w 2 、w 3 、w 4 is the weight coefficient. If C(t) exceeds a certain threshold, it is considered a collision event. In addition, the parameters can be adjusted according to the type of input quantity and event type.

[0113] 2) The calculation formula of the self-attention mechanism of the Transformer model is:

[0114]

[0115] Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key. Through the self-attention mechanism, the model is able to calculate the relationship between different time steps to classify the collision type.

[0116] 3) Position encoding: The Transformer model injects sequence information through position encoding. The formula is:

[0117]

[0118] Among them, pos is the position, i is the dimension, and d is the size of the encoded dimension.

Claims

1. A mobile robot collision detection method based on a composite Transformer model, characterized in that: The following steps are involved: Step 1: Get collision data; During the operation of the robot, data from sensors including torque sensors, IMU sensors and speed sensors are collected in real time as pre-training time collection data packets for collision detection; Step 2: Data preprocessing: preprocess the collected sensor data packets, including denoising and normalization operations, and convert the data into target format data packets suitable for input into the model; Step 3: LSTM layer processing, using the LSTM layer to process the sequence data and map the input data into the hidden state space to capture the temporal dependencies in the sequence; Step 4: Collision type classification: input the preprocessed data into the Transformer model, extract the temporal features in the data through the self-attention mechanism, and realize the classification of collision types.

2. According to claim 1, a mobile robot arm collision detection method based on a composite Transformer model is characterized in that: The input data of the Transformer model is a feature of dimension [N×T×input_size], where N represents the batch size, T represents the time step, and input_size represents the feature dimension of each time step; The dimension of the input feature is [N×T×input_size]. The feature is first processed by the LSTM layer. The LSTM captures the long-term and short-term dependencies in the time series through its internal gate mechanism. The output dimension of the LSTM layer is [N×T×hidden_size], where hidden_size represents the hidden layer dimension of the LSTM. Next, the output of LSTM is passed to the self-attention layer, which extracts global context information by calculating the similarity between different time steps in the sequence; each query Query, key Key, and value Value matrix in the self-attention mechanism is obtained through the linear transformation nn.Linear; through attention calculation and weighted summation, a weighted output with the dimension [N×T×hidden_size] is finally obtained.

3. According to claim 1, a mobile robot arm collision detection method based on a composite Transformer model is characterized in that: The step 2 specifically includes the following steps: Step 21: Sensor Signal Filtering: The sensor signal is collected using Kalman Filter to remove noise: in, is the predicted value of the system state, is the estimated value at time k-1, A is the state transfer matrix of the system, B is the control input matrix, u k is the control input, w k is the process noise; Step 22: Data Normalization: Use Min-Max Normalization: linearly map the data to the [0,1] interval: Where x is a characteristic value of the original data; x min is the minimum value of the feature; x max is the maximum value of the feature; x' is the normalized value; a and b are the lower and upper bounds of the interval respectively; Step 23: Denoising: Use a low pass filter to remove high frequency noise Where H(f) is the frequency response of the filter, f is the frequency, and f c is the cut-off frequency and n is the order of the filter.

4. The mobile robot arm collision detection method based on the composite Transformer model according to claim 1 is characterized in that: In step 3, the solution for processing local time series features of the LSTM model includes the following steps and specific formulas: The model receives time series data, and the input feature dimension is X∈R N×T×D , where N is the batch size, T is the sequence length, and D is the feature dimension at each time step; Given an input x at time step t t , the LSTM calculation process includes the following steps: Input Gate: i t =σ(W i x t +U i h t-1 +b i ) Forget Gate: f t =σ(W f x t +U f h t-1 +b f ) Output Gate: the t =σ(W o x t +U o h t-1 +b o ) Cell StateUpdate: c t =f t ·c t-1 +i t ·tanh(W c xt+U c h t-1 +b c ) Hidden StateUpdate: h t =o t ·tanh(c t ) in: σ is the Sigmoid activation function; tanh is the hyperbolic tangent activation function; W i , W f , W o , W c and U i , U i , U o , U c is the learned weight matrix; b i 、b f 、b o 、b c is the bias term.

5. The mobile robot arm collision detection method based on the composite Transformer model according to claim 1 is characterized in that: In step 4, the Transformer self-attention mechanism processes the collision result, which includes the following steps and specific formulas: Step 41: Dataset time judgment rule: Based on the IMU acceleration, angular velocity, displacement / angle change of the travel meter sensor and torque change of the joint torque sensor, define a comprehensive collision judgment index C(t): C(t)=w1·||a(t)||+w2·||ω(t)||+w3·||Δτ(t)||+w4·||Δθ(t)||+… Among them, ||a(t)|| is the modulus of acceleration, ||ω(t)|| is the modulus of angular velocity, ||Δτ(t)|| is the modulus of torque change, ||Δθ(t)|| is the modulus of angle change, w1, w2, w3, w4 are weight coefficients; if C(t) exceeds a certain threshold, it is determined to be a collision event; Step 42: The self-attention mechanism calculation formula of the Transformer model is: Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key; through the self-attention mechanism, the Transformer model calculates the relationship between different time steps to classify the collision type; Step 43: Position encoding. The Transformer model injects sequence information through position encoding. The formula is: Among them, pos is the position, i is the dimension, d is the dimension size of the encoding, PE (pos,2i) PE (pos,2i+1) They correspond to the even and odd dimensions in the position encoding vector, respectively.

Images