A trans-GCN-based dead reckoning method for mobile robots

Abstract

The application claims a kind of Trans-GCN-based mobile robot dead reckoning method.It contains the following steps, S1: the wheel speed sensor and inertial sensor data of wheeled mobile robot are collected, and time series data is intercepted by sliding window and adaptive sliding step length, sensor time series data is characterized as graph structure data;S2: a Laplace position code is constructed for each node in the graph, and after linear pooling, it is aggregated into the new embedding representation of node;S3: graph feature signal and encoding information are embedded into Transformer, and the output features of the overall model are input into the fully connected layer to obtain an output vector;S4: construct training true value, train the model according to sample data, stop training and save the model when less than threshold;S5: input the data under the condition of satellite signal loss into the trained model to obtain the predicted two-dimensional position increment of mobile robot in the sampling period, to realize the dead reckoning of basic ontology perception sensor signal under the condition of no GNSS signal.

Description

Technical Field

[0001] This invention belongs to the field of dead reckoning for mobile robots, and specifically relates to a dead reckoning method based on graph convolutional neural networks and Transformer structures. Background Technology

[0002] Against the backdrop of modern technological transformation from automation to intelligence, robotics, as a prime example of the integration of mechanical design, sensing technology, electronic information, automatic control theory, and artificial intelligence, is widely applied in various fields. Mobile robots, in particular, are playing an increasingly prominent role in various production and daily life scenarios, significantly improving the efficiency and safety of tasks such as power line inspection, security search and rescue, and material delivery. However, the core prerequisite for mobile robots to successfully handle various challenging tasks is their autonomous positioning capability, which is also a prerequisite for decision-making and path planning. Wheeled mobile robots, with their flexible movement, high stability, and excellent load-bearing capacity, are the most widely used type of mobile robot. Dead reckoning systems based on wheel speed sensors and inertial sensors for wheeled mobile robots have been extensively studied. Odometers measure the number of pulses per wheel per unit time using wheel encoders to calculate wheel speed and distance traveled. However, due to factors such as uneven ground friction and unstable differential speed calculation heading, pure odometer positioning may suffer from various cumulative errors. IMUs (Inertial Measurement Units) typically include two types of sensors: accelerometers and gyroscopes. They are used to measure and output the acceleration and angular velocity information of the vehicle. IMU measurements are affected by multiple types of noise, which are gradually amplified during integration, leading to the gradual accumulation of attitude angle and displacement errors, which grow exponentially. Traditional filtering methods rely on accurate system models and noise statistics, which may be difficult to adapt to changes in the model with the dynamic environment, resulting in increased estimation errors. With the continuous advancement of big data and artificial intelligence technologies, data-driven methods, mainly based on machine learning, are being widely used to solve navigation and positioning-related problems such as sensor calibration, suppressing positioning error divergence, and multi-sensor fusion. Dead reckoning for mobile robots based on multi-sensor fusion can be viewed as a prediction problem between multiple time series and sequences, not limited to time angles. The inherent connections and interactions between sensor data also affect the accuracy of the prediction.

[0003] This invention proposes a location prediction model, Trans-GCN, which integrates GCN and Transformer. It utilizes GCN to learn complex topological structures to capture spatial dependencies between data. It proposes a node embedding method that fuses node location features with Laplacian vectors, embedding graph feature signals and encoded information into Transformer to enhance the global location awareness of nodes in the graph, further improving the expressive power of graph structures. This makes the model more globally aware during feature extraction and enables multi-scale modeling of time-series data. Summary of the Invention

[0004] This invention aims to solve the problems of the prior art mentioned above. It proposes a mobile robot dead reckoning method based on Trans-GCN. The technical solution of this invention is as follows:

[0005] A mobile robot dead reckoning method based on Trans-GCN includes the following steps:

[0006] S1: Collect data from the wheel speed sensor and inertial sensor of the wheeled mobile robot, and extract time series data through a sliding window and an adapted sliding step size. In each window of data, construct graph nodes from the sensor data of a single dimension, and construct bidirectional edge connection structure for some nodes to form a complete graph structure data.

[0007] S2: Construct a graph convolutional network (GCN) to extract the dependencies of data in the graph in non-Euclidean space, including three steps: feature transfer, aggregation, and update, to obtain node features and edge features;

[0008] S3: Decompose the adjacency matrix of the constructed graph, construct the position encoding matrix of the nodes, integrate it through a linear layer and an L2 pooling layer, and then perform a linear transformation with the features extracted in S2 before inputting it into the Transformer encoder structure.

[0009] S4: Calculate the correlation between the target node and its neighboring nodes in each layer to obtain the attention weights. Then, connect the attention features with the original feature residuals and integrate them through a feedforward neural network (FFN). Finally, input the results into a fully connected network to obtain the network's predicted values.

[0010] S5: Train the model based on the training sample data, stop training when the verification loss is less than the threshold, save the model, and then input the wheel speed sensor and inertial sensor data into the model to obtain its dead reckoning results without GNSS signal.

[0011] Furthermore, step S1 specifically includes:

[0012] The mobile robot's inertial sensors record three-axis angular velocity and three-axis acceleration time-series data, and wheel speed sensors record two-dimensional data for the left and right wheels respectively. A sliding window of a certain length and sliding step size are set, and the data is dimensionality reduced. The mean and standard deviation of the data in a single-dimensional window are taken, and the labels of the true values ​​are aligned according to the timestamp. Then, the unlabeled training data in a single dimension is modeled as nodes, and edge connections are set between nodes to generate a record adjacency matrix, thus constructing a complete graph structure training data.

[0013] Furthermore, in step S3, the location encoding matrix is ​​constructed by including three dimensions of feature data: after obtaining the feature matrix from the Laplacian matrix of the graph, the normalized feature vectors corresponding to the first m smallest feature values ​​and the standardized feature vectors are selected and constructed together with the original feature vectors to form a three-dimensional initial location encoding matrix. Then, a new embedding representation is obtained by aggregation through a linear layer, and then simplified through an L2 pooling layer to obtain the final location encoding for propagation and learning.

[0014] Furthermore, in step S4, the input data is processed through four fully connected layers to obtain four matrices: Query(Q), Key(K), Value(V), and Edge(E). The feature data is then projected into the attention space. The formula for calculating the Attention-score matrix of the h-th attention head at the k-th layer is as follows:

[0015]

[0016] Where i is the target node number, j is the neighbor node, and d is the target node number. K The dimension represents the key vector. The Softmax function calculates the value for each row of the matrix, as shown in the following formula:

[0017]

[0018] Where y a y represents the value of the a-th column in a certain row of the Attention-score matrix. b represents the value in the b-th column of a certain row of the matrix, and w represents the column number of the matrix.

[0019] Furthermore, after calculating the Attention-score, residual connections are introduced, and Layernorm standardizes the input data. The features of each node are processed by a two-layer feedforward neural network for further non-linear feature representation. The specific formula is expressed as follows:

[0020]

[0021] in It is a Layernorm layer. It is a multi-head self-attention mechanism, where W is the weight matrix and ReLU is the activation function. This represents the hidden state after LayerNorm normalization. It is the hidden state after passing through a feedforward neural network and the ReLU activation function, with the addition of nonlinear transformation, enabling the model to learn more complex features.

[0022] Furthermore, step S4 concatenates the attention features with the original feature residuals, integrates them through a feedforward neural network (FFN), and finally inputs them into a fully connected network to obtain the network's predicted value. Specifically, this includes:

[0023] After passing through a two-layer feedforward neural network, an adaptive learning rate adjustment factor was designed, which is the ratio of the current gradient norm to the weight norm. The local learning rate scaling factor is calculated as follows:

[0024]

[0025] Where η represents the global learning rate. This represents the gradient of the k-th layer in the current batch of data, where λ represents the weight decay coefficient. Finally, the node feature output after passing through Trans-GCN is:

[0026]

[0027] Then, after passing through a fully connected layer, the model's predicted output is obtained through inverse normalization.

[0028] Furthermore, in step S5, the training ground truth is derived from the latitude and longitude data output by the RTK differential positioning system. The latitude and longitude data is transformed as follows: the latitude and longitude coordinates in the geographic coordinate system are transformed into those in the local coordinate system, with r... e r represents the equatorial radius. p The radius of the Earth's poles is represented by lat0, which is the latitude of the reference point in radians. The effective radius at the reference point is expressed as:

[0029]

[0030] r ns r ew Used to describe the Earth's radius of curvature in different directions, affecting the calculation of position information:

[0031] r ns It represents the radius of curvature in the north-south direction (meridian direction) and assists in the calculation of displacement in the latitudinal direction;

[0032] r ew It represents the radius of curvature in the east-west direction (latitude direction) and assists in the calculation of displacement in the longitude direction.

[0033] This allows the transformation of latitude and longitude coordinates into displacement changes in a local plane coordinate system, using lat. rad and lon rad The values ​​in radians represent the target's latitude and longitude; N and E represent the northward and eastward displacements, respectively.

[0034]

[0035] By aligning the timestamps of the reduced-dimensional input data with the latitude and longitude timestamps, the model can be trained and features extracted.

[0036] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the mobile robot dead reckoning method based on Trans-GCN as described in any one of the claims.

[0037] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the mobile robot dead reckoning method based on Trans-GCN as described in any one of the claims.

[0038] The advantages and beneficial effects of this invention are as follows:

[0039] The Trans-GCN model of this invention is a novel, multi-scale spatial search neural network model. It overcomes the limitation of traditional dead reckoning methods that rely solely on time series modeling.

[0040] 1. Breaking through the limitations of traditional temporal modeling, achieving multi-scale spatial feature learning: This invention constructs graph structure data (claim 1, S1-S2), models sensor data at different time steps as graph nodes, and extracts their correlation information in non-Euclidean space through graph convolutional networks (GCN), thereby adding spatial structure modeling capabilities in addition to the time dimension.

[0041] 2. An innovative node position encoding method enhances feature learning capabilities and, combined with the global modeling capabilities of Transformer, improves node feature fusion performance: In traditional GCN methods, nodes can only aggregate information from their local neighborhoods, making it difficult to capture long-distance dependencies. This invention proposes a 3D position encoding method that integrates Laplacian features and normalized feature vectors (claim 3). Compared to traditional methods that rely solely on absolute position encoding, this method can more accurately represent the relationships between nodes and further optimizes embedded features in the L2 pooling layer, improving feature learning capabilities. When calculating the attention score of the graph, this invention employs the Query-Key-Value-Edge (QKVE) mechanism (claims 4-5), simultaneously considering node and edge feature calculations. Furthermore, through residual connections and LayerNorm, the model can be trained more stably, preventing the gradient vanishing problem.

[0042] 3. Introducing an adaptive learning rate adjustment factor to accelerate model convergence: Traditional deep learning models often struggle to select a suitable learning rate during training, leading to slow convergence or getting stuck in local optima. This invention designs an adaptive learning rate adjustment factor (claim 6), which dynamically adjusts the learning rate based on the ratio of the current gradient norm to the weight norm, improving the model's training stability and accelerating convergence.

[0043] This invention can provide a new perspective and new thinking for dead reckoning methods for mobile robots, and promote the development of high-precision autonomous positioning of unmanned systems from the perspective of more intelligent and diversified methods. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the overall model learning data information flow designed according to a preferred embodiment of the present invention.

[0045] Figure 2 This is a schematic diagram of a graph structure construction method according to an embodiment of the present invention.

[0046] Figure 3 This is a schematic diagram illustrating the implementation of overall dead reckoning.

[0047] Figure 4 This is a flowchart of an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. The described embodiments are merely some embodiments of the present invention.

[0049] The technical solution of the present invention to solve the above-mentioned technical problems is:

[0050] Firstly, this invention provides a mobile robot dead reckoning method based on Trans-GCN, such as... Figure 4 As shown, the process includes data acquisition and processing, inputting the data into the model, training the model, updating the parameters, and performing dead reckoning for the robot when GNSS signal loss prevents obtaining the absolute position. The specific steps for model training, verification, and testing are as follows: Figure 3 As shown, it includes the following steps:

[0051] Step 1: Using the inertial sensors and wheel encoders inside the robot to record the wheel angular velocity, record the mileage data and attitude change data of the robot's movement, and set a sliding window of a certain length to capture the data of the corresponding length and the actual displacement change corresponding to each sliding window.

[0052] Step 1.1: Sliding Window Splitting

[0053] Suppose there are L sampling time steps, the sliding window size is M, and the step size is S. Then the data can be divided into H parts, where H is also the number of graphs. H is calculated as follows:

[0054]

[0055] By configuring the size of the sliding window, time-step data of length M is modeled as a single graph, and the features of each node simultaneously contain the temporal features of M time steps.

[0056] Step 1.2: Conversion from latitude and longitude to displacement

[0057] Transform latitude and longitude coordinates from the geographic coordinate system into a local coordinate system, using r e r represents the equatorial radius. p This represents the Earth's polar radius. lat0 is the latitude of the reference point in radians, with the latitude and longitude reference point as the reference. The effective radius of the reference point is calculated as follows:

[0058]

[0059] This allows the transformation of latitude and longitude coordinates into displacement changes in a local plane coordinate system, using lat. rad and lon rad The values ​​in radians represent the target's latitude and longitude. N and E represent the northward and eastward displacements, respectively. The calculation process is as follows:

[0060]

[0061] After the above steps, the true label values ​​for training are obtained. Then, the training data is aligned with the label values ​​according to the timestamps and keeps the dimensions consistent. The training set and validation set are divided into complete training data.

[0062] Step 2: The Trans-GCN model was constructed;

[0063] like Figure 1 As shown, the Trans-GCN model includes three steps.

[0064] Step 2.1: Feature Representation and Encoding Embedding

[0065] First, we need to construct the Laplacian matrix of the graph to obtain the eigenvectors:

[0066]

[0067] Where U∈R N×N It is the eigenvector matrix of the Laplacian matrix, where N is the number of nodes, and Λ is the diagonalized eigenvalue matrix. The columns corresponding to the first m smallest eigenvalues ​​in U are represented as follows:

[0068] P (i) =(U i,1 U i,2 ,...,U i,m )∈R m

[0069] Let λ i Let φ represent the i-th smallest eigenvalue. i and Let m represent the normalized eigenvector and the standardized eigenvector, respectively. Then, let m minimum eigenvalues ​​be associated with their corresponding φ and φ. Pairing yields node v i The positional encoding matrix is ​​then processed through a linear layer to aggregate a new embedding representation of size k. Finally, an L2 pooling layer is applied to further simplify the representation, resulting in the final input to the Transformer Encoder node encoding representation.

[0070] For each graph structure, the node features of the base graph are first transformed to obtain the d-dimensional node features. Sum of edge features Embedded into the graph Transformer layer, pre-computed k-dimensional encoded information is added to the node features through a linear transformation. The specific computational representation of the entire input embedding is as follows:

[0071]

[0072] in and Represents the basic characteristics of graph nodes, where B 0 ∈R d×k , and a 0 ,b 0 ∈R d These are the parameters of the linear projection layer.

[0073] Step 2.2: Attention Calculation

[0074] Each node v i and v j This will generate Query(Q), Key(K), and Value(V) respectively, where d h It is the dimension of the vector projected into the attention space, node v i The query and node v j The key can be used to calculate the attention score, which reflects the similarity between the two vectors.

[0075]

[0076] After the dot product calculation in high-dimensional space is completed, Scaling, in order to handle numerical stability issues, resizes the dot product result according to the dimension d of the key. K To scale, use the following formula:

[0077]

[0078] The Softmax function can be used to control the attention score within a reasonable range, avoiding excessive gradient problems in subsequent calculations, thus enabling the model to be trained stably.

[0079]

[0080] After obtaining the attention weights for each neighboring node Then, using these weights, the Value of each neighboring node is summed in a weighted manner to obtain node v. i The new feature representation

[0081]

[0082] To further refine feature extraction, multiple attention heads are operated simultaneously, each corresponding to a different query, key, and value vector (where H represents the number of heads). These heads capture the relationships between nodes from multiple different subspaces. The specific process is as follows:

[0083]

[0084] Step 2.3: Feature Integration and Transfer

[0085] Residual connections help preserve original input features, prevent over-updating of information, and solve the gradient vanishing problem in deep networks. This setup also ensures the propagation of original input features in each layer. Layer normalization standardizes the mean and variance of the input for faster convergence and gradient stabilization. The features of each node pass through a two-layer FFN for further non-linear mapping of feature representation, improving the model's expressive power and enabling it to learn more complex feature patterns. The specific formula is expressed as follows:

[0086]

[0087] To prevent the model from becoming overly smooth and affecting overall prediction performance, the model is improved as follows: By calculating the scaling factor for each layer—the ratio of the current gradient norm to the weight norm—it is ensured that layers with larger gradients have smaller learning rates, while layers with smaller gradients have larger learning rates. This adaptive adjustment helps stabilize the training process and effectively guide parameter updates. The local learning rate scaling factor is calculated as follows:

[0088]

[0089] Where η represents the global learning rate, W (k) This represents the weight matrix of the k-th layer. This represents the gradient of the k-th layer on the current batch of data, where λ represents the weight decay coefficient. The final node feature output is as follows:

[0090]

[0091] The steps for edge feature updating are similar to those for node feature updating, also including residual connections and layer normalization. Through these steps, the node and edge features in Trans-GCN are continuously updated in each layer, combining graph structure and edge information to capture the complex relationships between nodes in the graph.

[0092] Step 3: Model Training and Validation

[0093] Based on the data processed in step one, continue training the model designed in step two. Set a certain data ratio and parameters for the neural network model, start training the model, use mean squared error loss as the loss function, and use the Adam optimizer. Stop training when the loss function value is lower than the set threshold, and save the model.

[0094] Step 4: When the GNSS signal is briefly interrupted or obstructed and drifts, use the trained Trans-GCN model to perform dead reckoning for the mobile robot and complete position compensation when the absolute position is lost.

[0095] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions.

[0096] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0097] The above embodiments should be understood as illustrative only and not as limiting the scope of protection of the present invention. After reading the description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A method for dead reckoning of mobile robots based on Trans-GCN, characterized in that, Includes the following steps: S1: Collect data from the wheel speed sensor and inertial sensor of the wheeled mobile robot, and extract time series data through a sliding window and an adapted sliding step size. In each window of data, construct graph nodes from the sensor data of a single dimension, and construct bidirectional edge connection structure for some nodes to form a complete graph structure data. S2: Construct a graph convolutional network (GCN) to extract the dependencies of data in the graph in non-Euclidean space, including three steps: feature transfer, aggregation, and update, to obtain node features and edge features; S3: Decompose the adjacency matrix of the constructed graph, construct the position encoding matrix of the nodes, integrate it through a linear layer and an L2 pooling layer, and then perform a linear transformation with the features extracted in S2 before inputting it into the Transformer encoder structure; S4: Calculate the correlation between the target node and its neighboring nodes in each layer to obtain the attention weights. Then, connect the attention features with the original feature residuals and integrate them through a feedforward neural network (FFN). Finally, input the results into a fully connected network to obtain the network's predicted values. S5: Train the model based on the training sample data, stop training when the loss is less than the threshold, save the model, and then input the wheel speed sensor and inertial sensor data into the model to obtain its dead reckoning results without GNSS signal. In step S4, the input data is processed through four fully connected layers to obtain four matrices: Query(Q), Key(K), Value(V), and Edge(E). The feature data is then projected into the attention space. The h-th attention head at the k-th layer is calculated using the following formula: Where i is the target node number, j is the neighbor node, and d K The dimension represents the key vector. The Softmax function calculates the value for each row of the matrix, as shown in the following formula: Where y a y represents the value of the a-th column in a certain row of the Attention-score matrix. b represents the value in the b-th column of a certain row of the matrix, and w represents the number of columns in the matrix; After calculating the Attention-score, residual connections are introduced, and Layernorm standardizes the input data. The features of each node are passed through a two-layer feedforward neural network for further non-linear feature mapping, as expressed in the following formula: in It is a Layernorm layer. It is a multi-head concatenation self-attention mechanism, where W is the weight matrix and ReLU is the activation function. This represents the hidden state after LayerNorm normalization. It is the hidden state after passing through a feedforward neural network and the ReLU activation function, with the addition of nonlinear transformation, enabling the model to learn more complex features; S4 concatenates the attention features with the original feature residuals, integrates them through a feedforward neural network (FFN), and finally inputs them into a fully connected network to obtain the network's predicted value. Specifically, this includes: After passing through a two-layer feedforward neural network, an adaptive learning rate adjustment factor was designed, which is the ratio of the current gradient norm to the weight norm. The local learning rate scaling factor is calculated as follows: Where η represents the global learning rate. This represents the gradient of the k-th layer in the current batch of data, where λ represents the weight decay coefficient. Finally, the node feature output after passing through Trans-GCN is: Then, after passing through a fully connected layer, the model's predicted output is obtained through inverse normalization.

2. The mobile robot dead reckoning method based on Trans-GCN according to claim 1, characterized in that, Step S1 specifically includes: The mobile robot's inertial sensors record three-axis angular velocity and three-axis acceleration time-series data, and wheel speed sensors record two-dimensional data for the left and right wheels respectively. A sliding window of a certain length and sliding step size are set, and the data is dimensionality reduced. The mean and standard deviation of the data in a single-dimensional window are taken, and the labels of the true values ​​are aligned according to the timestamp. Then, the unlabeled training data in a single dimension is modeled as nodes, and edge connections are set between nodes to generate a record adjacency matrix, thus constructing a complete graph structure training data.

3. The mobile robot dead reckoning method based on Trans-GCN according to claim 1, characterized in that, In step S3, the location encoding matrix is ​​constructed, which includes feature data in three dimensions: after obtaining the feature matrix from the Laplacian matrix of the graph, the normalized feature vectors corresponding to the first m smallest feature values ​​and the standardized feature vectors are selected and constructed together with the original feature vectors to form a three-dimensional initial location encoding matrix. Then, a new embedding representation is obtained by aggregation through a linear layer, and then simplified through an L2 pooling layer to obtain the final location encoding for propagation and learning.

4. The mobile robot dead reckoning method based on Trans-GCN according to claim 1, characterized in that, In step S5, the training ground truth is derived from the latitude and longitude data output by the RTK differential positioning system. The latitude and longitude data is transformed as follows: the latitude and longitude coordinates in the geographic coordinate system are transformed into the local coordinate system, with r... e r represents the equatorial radius. p The radius of the Earth's poles is represented by lat0, which is the latitude of the reference point in radians. The effective radius at the reference point is expressed as: r ns r ew Used to describe the Earth's radius of curvature in different directions, affecting the calculation of position information: r ns It represents the radius of curvature in the north-south direction, i.e., the meridian direction, and assists in the calculation of displacement in the latitudinal direction; r ew It represents the radius of curvature in the east-west direction, i.e., the direction of latitude, and assists in the calculation of displacement in the longitude direction; This allows the transformation of latitude and longitude coordinates into displacement changes in a local plane coordinate system, using lat. rad and lon rad The values ​​in radians represent the target's latitude and longitude. N and E represent the northward and eastward displacements, respectively, and are calculated as follows: By aligning the timestamps of the reduced-dimensional input data with the latitude and longitude timestamps, the model can be trained and features extracted.

5. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the mobile robot dead reckoning method based on any one of claims 1 to 4.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the mobile robot dead reckoning method based on Trans-GCN as described in any one of claims 1 to 4.

Images