Method and apparatus for estimating trajectory of mobile robot

Abstract

The application discloses a kind of estimation method and device of mobile robot motion trajectory.Therein, the method includes: obtaining the environmental information around the robot, and constructing image sequence based on the environmental information;Using the feature extractor constructed based on convolutional neural network in advance, extract multidimensional image features from the image sequence;Based on the multidimensional image features, using the time series model constructed based on bidirectional LSTM network in advance, to estimate the relative pose change of the robot, and calculate the absolute pose of the robot based on the relative pose change;Based on the absolute pose, to determine the motion trajectory of the robot.The application solves the technical problem of poor pose estimation accuracy in existing mobile robot visual navigation technology.

Description

Technical Field

[0001] This invention relates to the field of navigation, and more specifically, to a method and apparatus for estimating the trajectory of a mobile robot. Background Technology

[0002] In researching autonomous localization of mobile robots, the robot's pose is one of the most crucial pieces of information. Currently, the main localization methods used by mobile robots include satellite navigation, inertial navigation, laser navigation, and visual navigation. In the working environment of mobile robots, due to satellite signal loss, the large size and low accuracy of inertial devices, and the high cost of high-precision inertial devices and laser equipment, visual navigation is often employed for autonomous localization. Currently, there are various pose estimation methods for mobile robots based on visual navigation, but they also have certain limitations.

[0003] In mobile robot pose estimation methods, since visual information is used as navigation information, the accuracy and speed of image feature extraction have a significant impact on pose estimation accuracy. Feature point methods are sensitive to illumination and have poor positioning accuracy in weak texture areas. Direct methods have large cumulative errors when the robot moves too fast. Deep learning-based methods have a wide range of applications, but they only use convolutional neural networks as feature extraction modules. However, since they do not consider the temporal relationship between images, the algorithm accuracy is not high and there is a large scale drift.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a method and apparatus for estimating the motion trajectory of a mobile robot, thereby at least solving the technical problem of poor pose estimation accuracy in existing mobile robot visual navigation technologies.

[0006] According to one aspect of the present invention, a method for estimating the motion trajectory of a mobile robot is provided, comprising: acquiring environmental information surrounding the robot and constructing an image sequence based on the environmental information; extracting multidimensional image features from the image sequence using a feature extractor pre-constructed based on a convolutional neural network; estimating the relative pose change of the robot based on the multidimensional image features using a temporal model pre-constructed based on a bidirectional LSTM network, and calculating the absolute pose of the robot based on the relative pose change; and determining the motion trajectory of the robot based on the absolute pose.

[0007] According to another aspect of the present invention, an apparatus for estimating the motion trajectory of a mobile robot is also provided, comprising: a sequence construction module configured to acquire environmental information surrounding the robot and construct an image sequence based on the environmental information; an extraction module configured to extract multidimensional image features from the image sequence using a feature extractor pre-constructed based on a convolutional neural network; an estimation module configured to estimate the relative pose change of the robot based on the multidimensional image features using a temporal model pre-constructed based on a bidirectional LSTM network, and calculate the absolute pose of the robot based on the relative pose change; and a trajectory determination module configured to determine the motion trajectory of the robot based on the absolute pose.

[0008] In this embodiment of the invention, a feature extractor pre-built based on a convolutional neural network is used to extract multidimensional image features from the image sequence; based on the multidimensional image features, a temporal model pre-built based on a bidirectional LSTM network is used to estimate the relative pose change of the robot, thus solving the technical problem of poor pose estimation accuracy in existing mobile robot visual navigation technologies. Attached Figure Description

[0009] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0010] Figure 1 This is a flowchart of a method for estimating the motion trajectory of a mobile robot according to an embodiment of the present invention;

[0011] Figure 2 This is a flowchart of another method for estimating the motion trajectory of a mobile robot according to an embodiment of the present invention;

[0012] Figure 3 This is a network system structure diagram of a mobile robot pose estimation method based on vision navigation according to an embodiment of the present invention;

[0013] Figure 4 This is a diagram of the forward propagation network structure of a bidirectional LSTM network according to an embodiment of the present invention;

[0014] Figure 5 This is a schematic diagram of a mobile robot equipped with a vision sensor and the vision sensor coordinate system according to an embodiment of the present invention;

[0015] Figure 6 This is a structural diagram of a bidirectional LSTM network according to an embodiment of the present invention;

[0016] Figure 7 This is a structural diagram of an adaptive storage network according to an embodiment of the present invention;

[0017] Figure 8 This is a schematic diagram of the true pose of the visual sensor and the resulting robot motion trajectory according to an embodiment of the present invention.

[0018] Figure 9 This is a structural diagram of a mobile robot pose estimation method device based on vision navigation according to an embodiment of the present invention; Detailed Implementation

[0019] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0021] Example 1

[0022] According to embodiments of the present invention, a method for estimating the motion trajectory of a mobile robot is provided, such as... Figure 1 As shown, the method includes:

[0023] Step S102: Obtain environmental information around the robot and construct an image sequence based on the environmental information.

[0024] In this embodiment, the mobile robot is equipped with a visual sensor for capturing information about its surrounding environment. Before constructing the image sequence, the image information acquired by the visual sensor should be preprocessed, including setting the image size to 640*192 and normalizing and standardizing all images.

[0025] Then, the robot's surrounding environment information is acquired using a visual sensor, and an image sequence is constructed. For example, all images in the environmental information are normalized and standardized; the image sequence is constructed based on the normalized and standardized environmental information.

[0026] Step S104: Extract multidimensional image features from the image sequence using a feature extractor pre-built based on a convolutional neural network.

[0027] A feature extractor based on a convolutional neural network (CNN) is designed. The image sequence is used as input to the feature extractor to obtain high-dimensional image features. After the CNN extracts the features, they are input into an adaptive storage network.

[0028] For example, the feature extractor can be constructed based on the following: setting multiple convolutional layers with gradient-descending kernels for the convolutional neural network, wherein the kernel of the first convolutional layer is larger than the kernels of the second to fourth convolutional layers, and the kernels of the second to fourth convolutional layers are larger than the kernels of the remaining convolutional layers; the feature extractor is constructed based on the multiple convolutional layers with gradient-descending kernels.

[0029] Step S106: Based on the multidimensional image features, the relative pose change of the robot is estimated using a time series model pre-built based on a bidirectional LSTM network, and the absolute pose of the robot is calculated based on the relative pose change.

[0030] The design constructs a temporal model based on a bidirectional LSTM network and an adaptive storage network. The high-dimensional image features are used as input to the temporal model to estimate the relative pose change of the visual sensor and calculate the absolute pose of the visual sensor.

[0031] For example, the time series model can be constructed based on the following: the bidirectional LSTM network is constructed based on the hidden state variables, state inputs, memory variables, nonlinear activation functions, weights, and bias vectors of the bidirectional LSTM network propagating forward and backward at time t; and the time series model is constructed based on the bidirectional LSTM network.

[0032] After estimating the robot's relative pose change, an adaptive storage network can be used to correct the relative pose change. For example, based on a preset threshold, some hidden layers of the bidirectional LSTM network are selected and stored in the adaptive storage network; for example, based on a preset rotation angle threshold and translation distance threshold, some hidden layers of the bidirectional LSTM network are selected; an influence factor is set for each hidden layer in the partial hidden layers, and based on the influence factor, the partial hidden layers are stored in the adaptive storage network. Then, the partial hidden layers in the adaptive storage network are used to correct the relative pose change.

[0033] After storing the portion of the hidden layer into the adaptive storage network, the current hidden layer state h of the bidirectional LSTM network can also be stored. t The cosine similarity between the current hidden layer state and the historical hidden layer state stored in the adaptive storage network is set as a weight factor for the current hidden layer state.

[0034] Step S108: Determine the robot's motion trajectory based on the absolute pose.

[0035] The method and apparatus for estimating the motion trajectory of a mobile robot provided in this invention solve the technical problem of poor pose estimation accuracy in existing mobile robot visual navigation technologies.

[0036] Example 2

[0037] The vision-based mobile robot pose estimation technology provided in this application involves capturing raw images of the robot's surrounding environment using a vision sensor; obtaining high-dimensional image features through a convolutional neural network after image preprocessing; outputting the relative pose change of the vision sensor through a bidirectional LSTM network and an adaptive storage network; and calculating the absolute pose of the vision sensor, i.e., the robot.

[0038] Figure 2 This is a method for estimating the motion trajectory of a mobile robot according to embodiments of this application, such as... Figure 2 As shown, the method includes:

[0039] Step S202: Use a visual sensor to acquire environmental information around the robot and construct an image sequence.

[0040] In an exemplary embodiment, before constructing the image sequence, the image information acquired by the visual sensor should be preprocessed. For example, the image size should be uniformly set to 640*192, and all images should be normalized and standardized. The normalization and standardization processing is performed based on the following formula, so that the image mean becomes 0 and the variance becomes 1:

[0041]

[0042] Where a is the original value of the image, a norm For the standardized and normalized images, mean(a) and std(a) are the mean and variance of all images, respectively.

[0043] Step S204: Design a feature extractor based on a convolutional neural network, using the image sequence as input to the feature extractor to obtain high-dimensional image features.

[0044] Figure 3 This is a network system architecture diagram of a vision-based mobile robot pose estimation method according to an embodiment of the present invention, such as... Figure 3 As shown, a feature extractor based on a convolutional neural network is constructed. Specifically, a convolutional neural network consisting of 10 convolutional layers is used, where the first convolutional kernel is 7*7, the second to fourth convolutional kernels are 5*5, and the remaining convolutional kernels are 3*3. The image sequence is used as the input to the feature extractor to extract high-dimensional image features.

[0045] Image features can be extracted based on the following formula:

[0046] x t，t-1 =f(I t-1 I t )

[0047] Where f represents the process by which the feature extractor acquires image features, I t-1 and I t For the images in frame t-1 and frame t, x t，t-1 The high-dimensional image features extracted at time t are: X = {x 1，0 x 2，1 , ..., x t，t-1}

[0048] Step S206: Design a time series model based on a bidirectional LSTM network and an adaptive storage network, use the high-dimensional image features as input to the time series model, estimate the relative pose change of the visual sensor and calculate the absolute pose of the visual sensor.

[0049] In an exemplary embodiment, constructing a temporal model, using the high-dimensional image features as input to the temporal model, includes: constructing a temporal model based on a bidirectional LSTM network, estimating the relative pose change of the visual sensor and calculating the absolute pose of the visual sensor, and using the absolute pose to constitute the robot's motion trajectory.

[0050] In an exemplary embodiment, assuming the current time is t, a bidirectional LSTM network is constructed based on the following formula, where the network forward propagation process is as follows: Figure 4 As shown:

[0051]

[0052] Where, x t For system input, h t-1 and h t+1 h represents the hidden state variables at times t-1 and t+1. t and h′ t S represents the hidden state variables of the Bi-LSTM network during forward and backward propagation at time t. t and S′ t Let o be the memory variable for the Bi-LSTM network during forward and backward propagation at time t. t For system output, f t i t g t and f′ t 、i′ t g′ t These are the nonlinear activation functions of the Bi-LSTM network during forward and backward propagation at time t, respectively, where σ is the tanh or sigmoid function, and W... fx W ix W gx W hx W h′x For x t The corresponding network weight, W fh W ih W gh W hh for h t-1 The corresponding network weight, W fh′ W ih′ W gh′ W hh′ for h t+1 The corresponding network weight, b f b i b g b h b o Let be the corresponding deviation vector in each equation. and These represent matrix addition and multiplication operations, respectively. h W h’ Weights corresponding to hidden state variables in the output variables

[0053] In this embodiment, the hidden state variables h of the bidirectional LSTM network during forward and backward propagation at time t are calculated. t and h′ tThen, the outputs of forward propagation and backward propagation are calculated separately, and the outputs of forward propagation and backward propagation are summed to obtain the output of the bidirectional LSTM network, so as to construct the bidirectional LSTM network; then, a time series model is constructed based on the bidirectional LSTM network.

[0054] Specifically, in the bidirectional LSTM network, for forward and backward propagation, to coordinate with the subsequent adaptive storage network, the hidden layer information h during forward and backward propagation is first calculated respectively. t and h′ t Then, in the output function, the outputs of forward propagation and backward propagation are calculated separately and summed, instead of calculating the weighted h first as in the prior art. t Then h′ t This method applies to the system output. For example, if the current time is t, existing methods only consider information from time t-1, while this method considers all information from time 1 to time t-1. This allows all historical information to be considered in the current system output, rather than just information from the previous time. Furthermore, information from the bidirectional LSTM network at time t+1 can also be incorporated simultaneously.

[0055] In one exemplary embodiment, the relative pose p of the visual sensor can be obtained from the high-dimensional image features X. 1，0 p 2，1 , ..., p t，t-1 The relative pose is composed of a rotation matrix R and a translation vector T, as shown in the following equation:

[0056]

[0057] Figure 5 For a mobile robot equipped with a vision sensor and a vision sensor coordinate system according to an embodiment of the present invention, in one exemplary embodiment, as follows: Figure 5 As shown, the direction the vision sensor is facing is the positive Z-axis, the right side of the vision sensor is the positive X-axis, and directly below the vision sensor is the positive Y-axis. Assuming the current time is t, the relative pose describes the relative change in the robot's position and orientation from time t-1 to time t. The position change is determined by T. t，t-1 = [Δx, Δy, Δz] represents the positional changes of the robot when moving straight, turning, and climbing, and the attitude is determined by the rotation matrix R. t，t-1 This indicates that the robot's posture changes are described.

[0058] Rotating the visual sensor coordinate system around the X-axis yields the pitch angle ψ, which is used to describe the robot's attitude change during hill climbing:

[0059]

[0060] Rotating the visual sensor coordinate system around the Y-axis yields the yaw angle γ, which is used to describe the robot's attitude changes when turning left or right.

[0061]

[0062] Rotating the visual sensor coordinate system around the Z-axis yields the roll angle φ, which is used to describe the robot's attitude change when moving on a non-horizontal surface:

[0063]

[0064] The rotation matrix R is obtained by multiplying the above three matrices in sequence:

[0065]

[0066] The relative pose calculation method is as follows:

[0067]

[0068] Where Δx, Δy, and Δz represent the coordinate transformations of the X-axis, Y-axis, and Z-axis, respectively.

[0069] In this embodiment, the method for calculating relative pose is improved. The relative pose is calculated based on Δx, Δy, Δz, pitch angle, roll angle, and yaw angle, making the calculated relative pose more accurate.

[0070] The absolute pose p of the vision sensor is calculated based on the following formula. t :

[0071]

[0072] In another example, relative pose can be represented using quaternions. A quaternion is a hypercomplex number containing four real elements, one real and three imaginary, with the basic form: q = (q... r q x q y q z Euler angles (or quaternions) describe the motion of rotating around an axis by a certain angle. Compared to rotation matrices, quaternions are faster to calculate and save computational resources. Converting Euler angles to quaternions is as follows:

[0073]

[0074]

[0075]

[0076]

[0077] In this example, unit quaternions are used to calculate pose rotation:

[0078]

[0079] In this example, the relative pose p of the vision sensor t，t-1 Represented as:

[0080] p t，t-1 =[Δx, Δy, Δz, q] r q x q y q z ] T

[0081] The visual sensor may have accumulated errors in relative pose, so in some examples, an adaptive storage network can also be used to correct these errors. Figure 6 This is a structural diagram of a bidirectional LSTM network according to an embodiment of the present invention, as shown below. Figure 6 As shown, for example, the hidden layer outputs of the bidirectional LSTM network are selectively stored in the adaptive storage network to reduce the impact of the accumulated error of relative pose on the accuracy of the temporal model. A threshold is set to select some hidden layer outputs for storage, and a weighting factor is set for the hidden layer outputs based on cosine similarity to emphasize the contribution of the hidden layer outputs at different times to the pose estimation at the current time.

[0082] Figure 7 This is a structural diagram of an adaptive storage network according to an embodiment of the present invention. In one exemplary embodiment, as shown... Figure 7 As shown, a threshold is set to select a portion of the hidden layer output for storage. The selection of hidden layers can be based on the following formula, where θ and d are the threshold values ​​for the rotation angle and translation distance of the vision sensor. For this example, θ = 0.006 (rad) and d = 0.8 (m). Of course, different values ​​can be used in other examples.

[0083]

[0084] Among them, R t，t-1 and T t，t-1 The relative pose p at time t t，t-1 The corresponding rotation matrix and translation vector, R t-1，t-2 and T t-1，t-2 The relative pose p at time t-1 t-1，t-2 The corresponding rotation matrix and translation vector.

[0085] In an exemplary embodiment, assume that n hidden layer states are selected and placed into a storage unit, denoted as M = {m1, m2, ..., m}. nSince each historical hidden layer state contributes differently to the current output, an influence factor c is set for each hidden layer state and normalized. Then, at time t, the historical information recorded in the storage unit is as follows:

[0086] M t =c1m1+c2m2+…+c n m n

[0087] In one exemplary embodiment, based on the current hidden layer state h t and in the storage unit m i The cosine similarity is used to assign a weight factor to the hidden layer states, emphasizing the contribution of the hidden layer states at different times to the pose estimation at the current time. For example, the weight factor can be designed and normalized based on the following formula:

[0088]

[0089] Among them, c i Let α represent the weight factor of the i-th hidden layer. i Let i be the weight factor of the i-th hidden layer after normalization. Therefore, the hidden layer information in the storage unit is updated as follows:

[0090]

[0091] In this embodiment, not only were weight factors designed, but the weight factors were also normalized to make the values ​​of the weight factors more suitable, thereby improving the accuracy of the neural network model.

[0092] Step S208: The robot's motion trajectory is composed of the absolute pose of the visual sensor.

[0093] Figure 8 This is a schematic diagram of the actual pose of the visual sensor and the resulting robot motion trajectory according to an embodiment of the present invention. Figure 8 As shown, the robot's motion trajectory is composed of the absolute pose of the visual sensor.

[0094] In this embodiment, the cumulative error of the relative pose is reduced, and the final pose estimation accuracy is improved. The motion trajectory of the mobile robot is composed of the absolute pose points of the visual sensor.

[0095] This embodiment provides a visual navigation-based method for estimating the pose of a mobile robot, which overcomes the shortcomings of existing technologies and enables accurate tracking of the robot's motion trajectory.

[0096] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0098] Example 3

[0099] Figure 9 This is a schematic diagram of the structure of a mobile robot trajectory estimation device according to an embodiment of this application, as shown below. Figure 9 As shown, the device includes:

[0100] The sequence construction module 72 is configured to acquire environmental information around the robot and construct an image sequence based on the environmental information;

[0101] Extraction module 74 is configured to extract multidimensional image features from the image sequence using a feature extractor pre-built based on a convolutional neural network;

[0102] The estimation module 76 is configured to estimate the relative pose change of the robot based on the multidimensional image features and using a time series model pre-built based on a bidirectional LSTM network, and to calculate the absolute pose of the robot based on the relative pose change.

[0103] The trajectory determination module 78 is configured to determine the robot's motion trajectory based on the absolute pose.

[0104] In this embodiment of the invention, the original image of the environment surrounding the mobile robot is acquired, the original image is normalized and standardized, and an image sequence is constructed; the sequence is extracted into image features using a convolutional neural network; the features are temporally modeled based on a bidirectional LSTM network and an adaptive storage network, thereby realizing robot pose estimation, reducing the impact of relative pose error accumulation on the final pose output, and solving the technical problem of poor pose estimation accuracy in existing mobile robot visual navigation technologies.

[0105] The motion trajectory estimation device for the mobile robot in this embodiment can realize the motion trajectory estimation method of the mobile robot in Embodiments 1 and 2, therefore, it will not be described again here.

[0106] Example 4

[0107] Embodiments of the present invention also provide a storage medium. The storage medium is configured to store program code for performing the motion trajectory estimation method of the mobile robot described in the above embodiments.

[0108] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0109] Optionally, specific examples in this embodiment can refer to the examples described in Embodiments 1 and 2 above, and will not be repeated here.

[0110] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0111] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0112] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0113] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0114] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0115] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0116] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for estimating the trajectory of a mobile robot, characterized in that, include: Acquire environmental information surrounding the robot and construct an image sequence based on the environmental information; Multidimensional image features are extracted from the image sequence using a feature extractor pre-built based on a convolutional neural network. Based on the multidimensional image features, the relative pose change of the robot is estimated using a time series model pre-built based on a bidirectional LSTM network, and the absolute pose of the robot is calculated based on the relative pose change. The robot's motion trajectory is determined based on the absolute pose; After estimating the relative pose change of the robot, the method further includes: selecting a portion of the hidden layers of the bidirectional LSTM network based on a preset threshold, and storing the portion of the hidden layers in an adaptive storage network; and using the portion of the hidden layers in the adaptive storage network to correct the relative pose change. The process of selecting a portion of the hidden layers of the bidirectional LSTM network based on a pre-set threshold and storing the portion of the hidden layers in the adaptive storage network includes: selecting a portion of the hidden layers of the bidirectional LSTM network based on a pre-set rotation angle threshold and translation distance threshold; setting an influence factor for each hidden layer in the portion of the hidden layers; and storing the portion of the hidden layers in the adaptive storage network based on the influence factor. After storing the portion of the hidden layer into the adaptive storage network, the method further includes: storing the current hidden layer state of the bidirectional LSTM network. The cosine similarity between the current hidden layer state and the historical hidden layer state stored in the adaptive storage network is set as a weight factor for the current hidden layer state. The timing model is constructed based on the following: calculating the bidirectional LSTM network in... Hidden state variables propagating forward and backward at every moment and Then, the outputs of forward propagation and backward propagation are calculated separately, and the outputs of forward propagation and backward propagation are summed to obtain the output of the bidirectional LSTM network, so as to construct the bidirectional LSTM network; then, a time series model is constructed based on the bidirectional LSTM network. The relative pose is calculated based on the coordinate transformations Δx, Δy, and Δz of the X, Y, and Z axes, as well as the pitch, roll, and yaw angles.

2. The method according to claim 1, characterized in that, Constructing an image sequence based on the environmental information includes: All images in the environmental information are normalized and standardized. The image sequence is constructed based on the environmental information after normalization and standardization.

3. The method according to claim 1, characterized in that, The feature extractor is constructed based on the following: The convolutional neural network is configured with multiple convolutional layers whose kernels descend in a gradient manner. The kernel of the first convolutional layer is larger than the kernels of the second to fourth convolutional layers, and the kernels of the second to fourth convolutional layers are larger than the kernels of the remaining convolutional layers. The feature extractor is constructed based on the multiple convolutional layers with gradient-decreasing kernels.

4. The method according to claim 1, characterized in that, After estimating the relative pose change of the robot, the method further includes: using an adaptive storage network to correct the relative pose change.

5. A device for estimating the motion trajectory of a mobile robot, characterized in that, include: A sequence construction module is configured to acquire environmental information around the robot and construct an image sequence based on the environmental information; The extraction module is configured to extract multidimensional image features from the image sequence using a feature extractor pre-built based on a convolutional neural network. The estimation module is configured to estimate the relative pose change of the robot based on the multidimensional image features and using a time series model pre-built based on a bidirectional LSTM network, and to calculate the absolute pose of the robot based on the relative pose change. The trajectory determination module is configured to determine the robot's motion trajectory based on the absolute pose; After estimating the robot's relative pose change, the device is further configured to: select a portion of the hidden layers of the bidirectional LSTM network based on a preset threshold, and store the portion of the hidden layers in an adaptive storage network; The relative pose change is corrected using the partial hidden layer in the adaptive storage network; The process of selecting a portion of the hidden layers of the bidirectional LSTM network based on a pre-set threshold and storing the portion of the hidden layers in the adaptive storage network includes: selecting a portion of the hidden layers of the bidirectional LSTM network based on a pre-set rotation angle threshold and translation distance threshold; setting an influence factor for each hidden layer in the portion of the hidden layers; and storing the portion of the hidden layers in the adaptive storage network based on the influence factor. After storing the portion of the hidden layer into the adaptive storage network, the device is further configured to: store the current hidden layer state of the bidirectional LSTM network. The cosine similarity between the current hidden layer state and the historical hidden layer state stored in the adaptive storage network is set as a weight factor for the current hidden layer state. The timing model is constructed based on the following: calculating the bidirectional LSTM network in... Hidden state variables propagating forward and backward at every moment and Then, the outputs of forward propagation and backward propagation are calculated separately, and the outputs of forward propagation and backward propagation are summed to obtain the output of the bidirectional LSTM network, so as to construct the bidirectional LSTM network; then, a time series model is constructed based on the bidirectional LSTM network. The relative pose is calculated based on the coordinate transformations Δx, Δy, and Δz of the X, Y, and Z axes, as well as the pitch, roll, and yaw angles.

6. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed, the computer performs the method as claimed in any one of claims 1 to 4.

Images