Online dead reckoning error prediction method based on robot motion and ground perception

Abstract

The application provides an online dead reckoning error prediction method based on robot motion and ground surface perception. The method comprises the following steps: constructing an online dead reckoning error neural network predictor model of the robot, including a body convolutional neural network (CNN), a visual CNN and a prediction MLP; training parameters of the online dead reckoning error neural network predictor to obtain a trained online dead reckoning error neural network predictor model; and using the body CNN, the visual CNN and the prediction MLP in the trained online dead reckoning error neural network predictor model to perform online dead reckoning error prediction on the robot. The method can incrementally optimize neural network parameters online using robot data, has a continuous learning capability, thereby improving the adaptability of the robot positioning system to complex motion and various ground surface conditions, has strong application value, and fills the blank in the field of robot dead reckoning error prediction.

Description

Technical Field

[0001] This invention relates to the field of dead reckoning technology, and in particular to an online dead reckoning error prediction method based on robot motion and ground perception. Background Technology

[0002] Dead Reckoning is a method that uses sensor inputs such as wheel speed encoders and IMUs (Inertial Measurement Units) to estimate the relative pose changes of a robot between frames (relative positioning) based on a kinematic model, and then calculates the robot's global pose (global positioning). Positioning is crucial for mobile robots to achieve autonomous navigation. In the absence of satellite positioning, current mainstream mobile robot positioning systems estimate relative pose changes (relative positioning) by fusing dead reckoning and environmental perception, and then calculate the position and attitude in the global coordinate system (global positioning) through methods such as estimation, loop closure detection, and optimization. Dead Reckoning (DR) is a method that uses sensor inputs such as wheel speed encoders and IMUs, based on a kinematic model, to estimate the relative pose changes of a robot between frames, and then calculates the robot's global pose.

[0003] Environmental perception-based localization refers to achieving relative localization (such as visual odometry, VO, and laser odometry, LO) or global localization (such as Simultaneous Localization and Mapping (SLAM)) of a robot by matching data from environmental perception sensors such as cameras and LiDAR. Dead reckoning and environmental perception are different modalities of localization, using different sensors and algorithms, each with its own advantages and disadvantages. Multimodal fusion localization aims to leverage the strengths of each method and make them complementary. Its key lies in accurately assessing the localization errors of each modality in different scenarios, allowing the modality with smaller errors and higher accuracy to play a greater role.

[0004] Methods for evaluating environmental perception positioning errors have been extensively studied. Currently, one existing method involves a visual / laser odometry error prediction model based on environmental perception positioning. This model maps environmental data collected by cameras or lidar into an odometry error information matrix, thereby improving the adaptability of robot fusion positioning in different scenarios. On the other hand, although dead reckoning is widely used and serves as a fundamental model for mobile robot positioning, its positioning errors have rarely been studied in depth. In practical applications, simplified error models are often constructed, and pre-calibrated error coefficients are used to estimate the dead reckoning positioning error.

[0005] The drawbacks of the existing online dead reckoning error prediction methods include: because they utilize pre-calibrated error coefficients, they cannot adapt to changes in robot motion and environment online, making it difficult to accurately estimate errors. Different robot movements lead to different dead reckoning errors. In particular, measurement errors from wheel speed encoders and IMUs are not only related to robot motion but also significantly affected by surface conditions. For example, on slippery surfaces, although the wheel speed encoder readings may be large, the actual robot travel may be very short due to wheel slippage, resulting in a large range error. Similarly, wheel speed encoders also exhibit significant range errors in muddy, sandy, carpeted, and grassy environments. Summary of the Invention

[0006] The embodiments of the present invention provide an online dead reckoning error prediction method based on robot motion and ground perception, so as to effectively improve the robot's positioning capability.

[0007] To achieve the above objectives, the present invention adopts the following technical solution.

[0008] An online dead reckoning error prediction method based on robot motion and surface perception includes:

[0009] Construct a neural network predictor model for online dead reckoning error of a robot, including a body convolutional neural network (CNN), a visual CNN, and a predictive MLP;

[0010] The parameters of the online dead reckoning error neural network predictor are trained to obtain a trained online dead reckoning error neural network predictor model;

[0011] The robot's online dead reckoning error is predicted using the body CNN, visual CNN, and prediction MLP in the trained online dead reckoning error neural network predictor model.

[0012] Preferably, the online dead reckoning error neural network predictor model for the robot includes an aircraft CNN, a visual CNN, and a prediction MLP, comprising:

[0013] An online dead reckoning error neural network predictor model is constructed, comprising an airframe CNN, a visual CNN, and a prediction MLP. The visual CNN uses a MobileNet v2 model pre-trained on ImageNet, taking a 224×224 image patch as input and outputting a 1×512 visual feature vector. The airframe CNN uses a one-dimensional convolutional neural network, taking a 1×7 vector as input, which includes the velocity obtained from the wheel speed encoder, three dimensions of linear acceleration obtained from the IMU, and three dimensions of angular velocity. After upsampling, it outputs a 1×256 airframe feature vector. The prediction MLP uses a multilayer perceptron, taking a 1×768 vector concatenated with the airframe feature vector as input and outputting a 1×4 error coefficient vector.

[0014] Preferably, training the parameters of the online dead reckoning error neural network predictor to obtain a trained online dead reckoning error neural network predictor model includes:

[0015] Let dead reckoning and visual / laser odometry respectively obtain the robot's relative pose estimate u. t and z t Let μ t To fuse the relative positioning results, assume u t ,z t ,μ t The errors all follow a Gaussian distribution, and their covariance matrices are R0 and R1 respectively. t Q t ,Σ t ;

[0016] z was obtained using visual / laser odometry. t Q is calculated using the matching error of corresponding point pairs. t Using dead reckoning formula 3, u is obtained t Using an online dead reckoning error neural network predictor π Θ Get R t The information filter method is used to perform fusion positioning calculations and solve for μ. t ,Σ t Let the information matrix be... The information vector is ξ t =Ω t μ t Let the fusion positioning algorithm be denoted as This process can be summarized as follows:

[0017]

[0018] Detailed calculation steps

[0019] Based on dead reckoning prediction information matrix

[0020] Based on dead reckoning prediction information vector

[0021] Update the information matrix using environmental perception

[0022] Update information vectors using environmental perception

[0023] Calculate the covariance matrix of the fused localization

[0024] Calculate the fusion localization result μ t =Σ t ξ t

[0025] Taking two-dimensional positioning as an example, the relative pose estimation μ at time t t =(μ x,t ,μ y,t ,μ θ,t ) T The displacement and rotation of the robot relative to time t-1 in the x and y coordinate components are represented in the form of a coordinate transformation matrix:

[0026]

[0027] Given the robot's initial pose in the global coordinate system And relative positioning estimation μ1, μ2, ... μ across multiple consecutive frames. t The robot's global pose is calculated as follows:

[0028]

[0029] set up The coordinate transformation function described above can be simplified as follows:

[0030]

[0031] Let I t Let π be the sensor data collected at time t by the wheel speed encoder, IMU, and camera at the current moment. Θ This is an online dead reckoning error neural network predictor, where Θ represents the parameters of the neural network, and A... t =(α1,…,α4) t R represents the error coefficient of the dead reckoning model. t For A t Based on the dead reckoning error covariance matrix estimated by Equation 4, the forward inference process of robot fusion localization is as follows:

[0032] The covariance matrix R of the predicted dead reckoning error t =πΘ (I t )

[0033] Relative positioning by integrating dead reckoning and environmental perception

[0034] Global positioning calculation

[0035] Suppose the robot starts from an initial point where the true value of the position is known. After starting and passing through T frames, the true location value in the global coordinate system was obtained. The fusion positioning calculation result is Let T > τ, where τ is the minimum number of iteration frames, and define the loss as follows:

[0036]

[0037] Where λ is a hyperparameter balancing positional and orientation accuracy, and Θ represents the parameters of the body CNN, visual CNN, and prediction MLP neural networks, the learning objective is to optimize Θ to minimize the loss function J:

[0038]

[0039] During the training process, for each training trajectory, the loss J is calculated according to Formula 10, and the neural network parameters are updated according to Formula 12. This process is repeated until the network parameters converge.

[0040] A trained online dead reckoning error neural network predictor model was obtained.

[0041] Preferably, the method of using the body CNN, visual CNN, and prediction MLP in the trained online dead reckoning error neural network predictor model to predict the robot's online dead reckoning error includes:

[0042] The robot body data, including stroke, acceleration, and angular velocity, is collected by a wheel speed encoder and an IMU. This body data is then input into a trained online dead reckoning error neural network predictor model. A 224×224 image block is cut from the bottom of the original 1024×768 image in the body data and input into a visual CNN. The visual CNN outputs a 1×512 visual feature vector. A 1×7 vector, including three dimensions of linear acceleration from the velocity sensor and three dimensions of angular velocity from the IMU, is input into the body CNN. The body CNN outputs a 1×256 body feature vector. The visual feature vector and the body feature vector are concatenated to obtain a 1×768 vector. This 1×768 vector is input into the prediction MLP, which outputs a 1×4 error coefficient vector α1,…,α4. Finally, the dead reckoning result u of the robot to be identified is obtained using the following formulas (1)-(5). t The covariance matrix R of the error t ;

[0043] Let Δs be the robot's travel distance measured by the wheel speed encoder from time t-1 to time t, Δθ be the change in the robot's heading angle measured by the IMU, and Δt be the time taken for this process. Then, the measured values ​​of the robot's velocity v and angular velocity ω are calculated as follows:

[0044]

[0045] Where ∈ t and ∈ ω Let v be the measurement error of velocity v and angular velocity ω, with zero mean and variance α1Δs. 2 +α2Δθ 2 ,α3Δs 2 +α4Δθ 2 The Gaussian distribution is α1,…,α4, which are the error coefficients output by the aforementioned online dead reckoning error prediction model.

[0046] Dead reckoning using a robot motion model, relative pose u at time t. t =(u x,t ,u y,t ,u θ,t ) T The estimate is as follows:

[0047]

[0048] in, Let be the robot's orientation angle in the global coordinate system at time t-1; ∈ t For u t The error follows a zero-mean Gaussian distribution, R0 t The covariance matrix of the error is calculated as follows:

[0049]

[0050] in

[0051]

[0052] Substituting the error coefficients α1,…,α4 output by the aforementioned online dead reckoning error prediction model into formula (4), the dead reckoning result u of the robot to be identified is obtained. t The covariance matrix R of the error t .

[0053] As can be seen from the technical solutions provided by the embodiments of the present invention described above, the method of the present invention does not rely on manual annotation, can incrementally optimize neural network parameters online using robot data, and has continuous learning capabilities, thereby improving the adaptability of the robot positioning system to complex movements and diverse surface conditions. It has strong application value and fills the gap in robot dead reckoning error prediction.

[0054] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0055] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 A flowchart illustrating the online dead reckoning error prediction method for robot motion and ground perception provided in an embodiment of the present invention;

[0057] Figure 2 This is a schematic diagram illustrating the implementation principle of an online dead reckoning error neural network predictor provided in an embodiment of the present invention;

[0058] Figure 3 This is a schematic diagram illustrating the implementation principle of a parameter training method for a dead reckoning error neural network predictor provided in an embodiment of the present invention. Detailed Implementation

[0059] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0060] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0061] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0062] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0063] This invention proposes an online dead reckoning error prediction method based on robot motion and ground perception. First, a dead reckoning positioning and error model is derived using a robot motion model and error propagation theory. Second, a neural network predictor is designed, taking robot motion and ground sensor data as input, to predict error coefficients online and calculate the covariance matrix of the dead reckoning error. Then, within a fusion positioning framework that includes dead reckoning and environmental perception, the dead reckoning error prediction network parameters are trained unsupervised using online acquired robot data, with the goal of minimizing the fusion positioning error.

[0064] The following section first presents the dead reckoning and error model, and then explains the online dead reckoning error neural network predictor method of the present invention.

[0065] Using robot motion models and error propagation theory, a positioning and error model is derived. Let Δs be the robot's travel distance measured by the wheel speed encoder from time t-1 to time t, Δθ be the change in the robot's heading angle measured by the IMU (Inertial Measurement Unit), and Δt be the time taken for this process. Then, the measured values ​​of the robot's velocity v and angular velocity ω are calculated as follows:

[0066]

[0067] Where ∈ t and ∈ ω Let v be the measurement error of velocity v and angular velocity ω, with zero mean and variance α1Δs. 2 +α2Δθ 2 ,α3Δs 2 +α4Δθ 2 The Gaussian distribution is used. Intuitively, the longer the robot's travel and the larger the turning angle, the greater the potential measurement error, and the greater the uncertainty in estimating the robot's speed and angular velocity, with these effects being correlated. α1,…,α4 are error coefficients, predicted online using a trained online dead reckoning error neural network predictor.

[0068] Dead reckoning using a robot motion model, relative pose u at time t. t =(u x,t ,u y,t ,u θ,t ) T The estimate is as follows:

[0069]

[0070] in, Let be the robot's orientation angle in the global coordinate system at time t-1; ∈ t For u t The error follows a zero-mean Gaussian distribution, R0 t The covariance matrix of the error is calculated as follows:

[0071]

[0072] in

[0073]

[0074] Substituting the measured values ​​of velocity v and angular velocity ω, along with the error coefficients α1,…,α4 predicted online using a trained online dead reckoning error neural network predictor, into the above equation, the covariance matrix R can be estimated. t .

[0075] The specific implementation steps of the dead reckoning method at time t are as follows:

[0076] (1) Input the measured values ​​of the wheel speed encoder and IMU from time t-1 to time t.

[0077] (2) Calculate the robot's relative pose change u using Formula 3. t

[0078] (3) Using the trained online dead reckoning error neural network predictor, the online prediction error coefficients α1,…,α4 are calculated.

[0079] (4) Substitute the measured values ​​of velocity v and angular velocity ω, and the error coefficients α1,…,α4 into Formula 4 to calculate the covariance matrix R of the error. t .

[0080] The processing flow of an online dead reckoning error prediction method based on robot motion and ground perception provided in this embodiment of the invention is as follows: Figure 1 As shown, the processing steps include the following:

[0081] Step S10: Construct a neural network predictor model for the robot's online dead reckoning error.

[0082] The online dead reckoning error neural network predictor model includes an aircraft CNN (Convolutional Neural Networks), a visual CNN, and a prediction MLP. The specific structure and parameter information are as follows:

[0083] 1. A visual CNN: using a MobileNet v2 model pre-trained on ImageNet, taking a 224×224 image patch as input and outputting a 1×512 visual feature vector;

[0084] 2. A machine CNN: Using a one-dimensional convolutional neural network, the input is a 1×7 vector (velocity obtained from the wheel speed encoder, linear acceleration in three dimensions and angular velocity in three dimensions obtained from the IMU), which is upsampled and outputs a 1×256 machine feature vector;

[0085] 3. A predictive MLP: using a multilayer perceptron, the input is a 1×768 vector of visual features and a vector of machine features concatenated together, and the output is a 1×4 vector of error coefficients.

[0086] Step S20: Train the parameters of the online dead reckoning error neural network predictor to obtain the trained online dead reckoning error neural network predictor model.

[0087] Step S30: Use the trained online dead reckoning error neural network predictor model to predict the online dead reckoning error of the robot.

[0088] The input data for the trained online dead reckoning error neural network predictor model are as follows: a 224×224 image block is cut from the bottom of the original 1024×768 image of the robot to be identified and input into the visual CNN; the velocity, the three-dimensional linear acceleration and the three-dimensional angular velocity obtained from the IMU, and the 1×7 vector are input into the body CNN. The velocity is calculated by the wheel speed encoder as follows: let the wheel speed encoder count between two adjacent frames be A, the pre-calibrated travel-encoder conversion coefficient be B (e.g., 1Count≈0.003846154meter), and the inter-frame time be C, then the velocity = A×B / C.

[0089] Output data: The output is a 1×4 error coefficient vector, and then the covariance matrix R of the error is obtained using Equation 4. t .

[0090] Processing procedure: A 224×224 image patch is input into a visual CNN to obtain a 1×512 visual feature vector. A 1×7 vector is input into a machine CNN to obtain a 1×256 machine feature vector. The two feature vectors are concatenated to obtain a 1×768 vector, which is then input into a prediction MLP. The output is a 1×4 error coefficient vector α1,…,α4. Finally, using Equation 4, the covariance matrix R of the error of the robot to be identified is obtained. t .

[0091] A schematic diagram illustrating the implementation principle of an online dead reckoning error neural network predictor provided in this embodiment of the invention is shown below. Figure 2 As shown, the online dead reckoning error neural network predictor utilizes robot motion models and error propagation theory to derive a dead reckoning error model containing multiple coefficients. Using sensor data such as the robot's motion at the current moment and surface images as input, it predicts the dead reckoning error coefficients online, thereby calculating the covariance matrix of the dead reckoning error.

[0092] like Figure 2 As shown, the robot's motion, acceleration, and angular velocity are collected via a wheel speed encoder and an IMU. This motion data is then input into a CNN network (called the motion CNN), which extracts motion features from the motion data. Conversely, a camera captures images of the road ahead, which are used as visual data. This visual data is then input into another convolutional neural network (called the visual CNN), which extracts visual features from the visual data. The motion features and visual features are then fused and input into a multilayer perceptron (MLP) (called the prediction MLP), with prediction error coefficients α1,…,α4.

[0093] Let I tLet π be the sensor data collected at time t by wheel speed encoder, IMU, camera, etc., representing the current time. Θ This is an online dead reckoning error neural network predictor, where Θ represents the parameters of the neural network, and A... t =(α1,…,α4) t R represents the error coefficient of the dead reckoning model. t For A t The calculation process of the online dead reckoning error neural network predictor, based on the dead reckoning error covariance matrix obtained from Formula 4, can be expressed as R t =π Θ (I t ).

[0094] The present invention designs a parameter training method for an online dead reckoning error neural network predictor model. The implementation principle is as follows: Figure 3 As shown in the figure. In the parameter training method, a fusion localization framework based on dead reckoning and environmental perception is used. After multiple consecutive frame iterations, the fusion localization results are compared with the ground truth localization values ​​to calculate the localization loss and backpropagate to optimize the parameters of the neural network predictor.

[0095] The following section first presents a fusion positioning framework for dead reckoning and environmental awareness, and then explains the neural network parameter training method for the dead reckoning error predictor of this invention.

[0096] A fusion positioning framework combining dead reckoning and environmental perception: such as Figure 3 The forward inference process involves dead reckoning and visual / laser odometry (VO / LO) to obtain the robot's relative pose estimate, u. t and z t Let μ t To fuse the relative positioning results. Assume u t ,z t ,μ t The errors all follow a Gaussian distribution, and their covariance matrices are R0 and R1 respectively. t Q t ,Σ t .

[0097] This invention uses visual / laser odometry to obtain z t Q is calculated using the matching error of corresponding point pairs. t Using dead reckoning formula 3, u is obtained t Using an online dead reckoning error neural network predictor π Θ Get R t The information filter method is used to perform fusion positioning calculations and solve for μ. t ,Σ t Let the information matrix be... The information vector is ξt =Ω t μ t Let the fusion positioning algorithm be denoted as This process can be simplified as follows:

[0098]

[0099] Detailed calculation steps

[0100] Based on dead reckoning prediction information matrix

[0101] Based on dead reckoning prediction information vector

[0102] Update the information matrix using environmental perception

[0103] Update information vectors using environmental perception

[0104] Calculate the covariance matrix of the fused localization

[0105] Calculate the fusion localization result μ t =Σ t ξ t

[0106] Taking two-dimensional positioning as an example, the relative pose estimation μ at time t t =(μ x,t ,μ y,t ,μ θ,t ) T The displacement and rotation of the robot relative to time t-1 in the x and y coordinate components can be represented by a coordinate transformation matrix:

[0107]

[0108] Given the robot's initial pose in the global coordinate system And relative positioning estimation μ1, μ2, ... μ across multiple consecutive frames. t The robot's global pose can be calculated as follows:

[0109]

[0110] set up The coordinate transformation function described above can be simplified as follows:

[0111]

[0112] Let I t Let π be the sensor data collected at time t by wheel speed encoder, IMU, camera, etc., representing the current time.Θ This is an online dead reckoning error neural network predictor, where Θ represents the parameters of the neural network, and A... t =(α1,…,α4) t R represents the error coefficient of the dead reckoning model. t For A t Based on the dead reckoning error covariance matrix estimated by Formula 4, the forward inference process of robot fusion localization is as follows.

[0113] Detailed calculation steps

[0114] The covariance matrix R of the predicted dead reckoning error t =π Θ (I t )

[0115] Relative positioning by integrating dead reckoning and environmental perception

[0116] Global positioning calculation

[0117] The parameter training process of the online dead reckoning error neural network predictor provided in this embodiment of the invention includes:

[0118] Suppose the robot starts from an initial point where the true value of the position is known. Starting from point T, the true positioning value in the global coordinate system was obtained through methods such as GPS or loop closure detection. The fusion positioning calculation result is To improve the stability of the learning algorithm, it is usually assumed that T > τ, where τ is the minimum number of iteration frames, to ensure that there is significant error accumulation in the localization calculation. Loss is defined as follows:

[0119]

[0120] Where λ is a hyperparameter balancing positional and orientation accuracy. Let Θ be the parameters of the organism CNN, visual CNN, and prediction MLP neural networks, then the learning objective is to optimize Θ to minimize the loss function J:

[0121]

[0122] During training, for each training trajectory, the loss J is calculated according to Formula 10, and the neural network parameters are updated according to Formula 12. This process is repeated until the network parameters converge.

[0123]

[0124] A trained online dead reckoning error neural network predictor model was obtained.

[0125] In summary, the mobile robots and unmanned ground platform technologies of this invention have broad application prospects in industry and defense, with positioning technology being the foundation for performing various tasks. In the absence of satellite positioning, dead reckoning positioning using inputs from sensors such as wheel speed encoders and inertial measurement units (IMUs), and environmental perception positioning (such as VO / LO, VSLAM / LSLAM) using data matching from environmental perception sensors such as cameras and lidar, are two main modes. Multimodal fusion positioning is crucial for achieving accurate positioning of mobile robots, and a reliable error model is the basis for effective fusion. Existing research has largely focused on environmental perception positioning, while in-depth research on dead reckoning error models is lacking. In practical applications, simplified error models are often constructed, using pre-calibrated error coefficients to estimate dead reckoning positioning errors, which cannot adapt to changes in robot movement and environment online.

[0126] This invention proposes an online dead reckoning error prediction method that considers robot motion and surface conditions. This method can incrementally optimize the model parameters of the prediction neural network online using robot data, aiming to improve the accuracy of fused positioning. This method does not rely on manual annotation, possesses continuous learning capabilities, and can improve the adaptability of the robot positioning system to complex motion and diverse surface conditions.

[0127] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0128] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0129] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0130] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for predicting online dead reckoning errors based on robot motion and surface perception, characterized in that, include: Construct a neural network predictor model for online dead reckoning error of a robot, including a body convolutional neural network (CNN), a visual CNN, and a predictive MLP; The parameters of the online dead reckoning error neural network predictor are trained to obtain a trained online dead reckoning error neural network predictor model; The robot's online dead reckoning error is predicted using the body CNN, visual CNN, and prediction MLP in a pre-trained online dead reckoning error neural network predictor model. The online dead reckoning error prediction includes: The robot's body data, including stroke, acceleration, and angular velocity, is acquired through wheel speed encoders and an inertial measurement unit (IMU). This body data is then input into a trained online dead reckoning error neural network predictor model. A 224×224 image patch is cropped from the bottom of the original 1024×768 image acquired by the camera and input into the visual CNN. The visual CNN outputs a 1×512 visual feature vector. A 1×7 vector, including the velocity from the wheel speed encoder and the three-dimensional linear acceleration and three-dimensional angular velocity from the IMU, is input into the body CNN. The body CNN outputs a 1×256 body feature vector. The visual feature vector and the body feature vector are concatenated to obtain a 1×768 vector. This 1×768 vector is then input into the prediction MLP, which outputs a 1×4 error coefficient vector. and the error coefficient vector Substituting the following robot kinematic error propagation formulas (1)-(5), the dead reckoning result of the robot to be identified is obtained. The covariance matrix of the error ; Let from Time's up During the time interval, the robot's travel distance measured by the wheel speed encoder is The change in the robot's heading angle measured by the IMU is The process takes time to complete. Then the robot's speed With angular velocity The measured values ​​are calculated as follows: in and For speed With angular velocity The measurement error follows a zero mean and variances of . Gaussian distribution, These are the error coefficients output by the aforementioned online dead reckoning error prediction model; Dead reckoning using robot motion models Relative pose at time The estimate is as follows: in, for The robot's orientation angle in the global coordinate system at any given moment; for The error follows a zero-mean Gaussian distribution. The covariance matrix of the error is calculated as follows: in The error coefficients output by the aforementioned online dead reckoning error prediction model Substituting into formula (4), the dead reckoning result of the robot to be identified is obtained. The covariance matrix of the error It is used for the fusion positioning of robot dead reckoning and environmental perception.

2. The method according to claim 1, characterized in that, The aforementioned online dead reckoning error neural network predictor model for the robot includes an aircraft CNN, a visual CNN, and a predictive MLP, comprising: An online dead reckoning error neural network predictor model is constructed, comprising an airframe CNN, a visual CNN, and a prediction MLP. The visual CNN uses a MobileNet v2 model pre-trained on ImageNet, taking a 224×224 image patch as input and outputting a 1×512 visual feature vector. The airframe CNN uses a one-dimensional convolutional neural network, taking a 1×7 vector as input, which includes the velocity obtained from the wheel speed encoder, three dimensions of linear acceleration obtained from the IMU, and three dimensions of angular velocity. After upsampling, it outputs a 1×256 airframe feature vector. The prediction MLP uses a multilayer perceptron, taking a 1×768 vector concatenated with the airframe feature vector as input and outputting a 1×4 error coefficient vector.

3. The method according to claim 2, characterized in that, The process of training the parameters of the online dead reckoning error neural network predictor to obtain a trained online dead reckoning error neural network predictor model includes: Let dead reckoning and visual / laser odometry be used to obtain the robot's relative pose estimation, respectively. and ,set up To integrate the relative positioning results, we assume The errors all follow a Gaussian distribution, and their covariance matrices are respectively... ; Obtained using visual / laser odometry Calculate using the matching error of corresponding point pairs Using dead reckoning formula (3), we obtain Using an online dead reckoning error neural network predictor get The information filter method is used to perform fusion positioning calculations and solve the problem. Let the information matrix be... The information vector is Let the fusion positioning algorithm be denoted as The process can be abbreviated as: Detailed calculation steps Based on dead reckoning prediction information matrix Based on dead reckoning prediction information vector Update the information matrix using environmental perception Update information vectors using environmental perception Calculate the covariance matrix of the fused localization Calculate the fusion localization results Taking two-dimensional positioning as an example, Relative pose estimation at time step Includes robots relative to Always Displacement and rotation in the coordinate components are represented by coordinate transformation matrices: Given the robot's initial pose in the global coordinate system And relative positioning estimation across multiple consecutive frames The robot's global pose is calculated as follows: set up The coordinate transformation function described above can be simplified as follows: set up for The sensor data at the current moment is collected by the wheel speed encoder, IMU, and camera. This is an online dead reckoning error neural network predictor, in which... The parameters of the neural network, The error coefficients for the dead reckoning model. Based on The forward inference process of robot fusion localization is as follows, based on the dead reckoning error covariance matrix estimated by formula (4): Covariance matrix of predicted dead reckoning error ; Relative positioning by integrating dead reckoning and environmental perception Global positioning calculation Suppose the robot starts from an initial point where the true value of the position is known. Departure, via After the frame, the ground truth of the positioning in the global coordinate system was obtained. The fusion positioning calculation result is ,set up To minimize the number of iteration frames, define the loss as follows: (10) in, To balance the hyperparameters of position accuracy and orientation accuracy, let... For the parameters of the organismal CNN, visual CNN, and predictive MLP neural network, the learning objective is to optimize... This makes the loss function Minimize: During the training process, for each training trajectory, the loss J is calculated according to formula (10), and the neural network parameters are updated according to formula (12). This process is repeated until the network parameters converge. A trained online dead reckoning error neural network predictor model was obtained.

Images