An unsupervised monocular visual odometry pose estimation method based on sequence learning

Abstract

The application discloses a kind of based on sequence learning's unsupervised monocular visual odometry pose estimation method, belong to pose estimation field, this method includes obtaining monocular image sequence data, and obtaining the per-pixel depth image sequence of monocular image sequence data;According to monocular image sequence data, segmentation is carried out using fixed step sliding window, each adjacent two frame image pair is spliced, and a plurality of image pair sequence data are obtained;According to each image pair sequence data, the relative camera pose after multiple pose transformation accumulation between image pairs is obtained using variable step convolution long short-term memory network;According to the per-pixel depth image sequence of monocular image sequence data, monocular image sequence data and the relative camera pose after multiple pose transformation accumulation between image pairs, based on the self-supervised learning framework for pose estimation, pose estimation result is obtained.The application solves the problem that the existing unsupervised monocular visual odometry technology pose estimation accuracy and robustness is poor.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of pose estimation, and particularly relates to an unsupervised monocular visual odometry pose estimation method based on sequence learning. BACKGROUND

[0002] With the development of electronic information technology and artificial intelligence technology, the demand for various robots with automation and intelligence is increasing. Robots with autonomous positioning and navigation systems are widely used in logistics, factories, rescue and other scenarios, greatly improving production efficiency and safety. At the same time, high-precision positioning and navigation technology is also a key technology in the fields of automatic driving and unmanned delivery.

[0003] Currently, global navigation satellite systems (GNSS) such as global positioning system (GPS) and Beidou satellite navigation system (BDS) are widely used in various fields. However, this positioning technology is affected by the environment. In complex outdoor scenes (such as forests and densely built-up areas) or indoor scenes with certain shielding and interference of satellite signals, the positioning accuracy and stability of global navigation satellite systems (GNSS) will be greatly affected. The positioning technology of visual odometry (VO) using visual sensors is one of the effective ways to solve the above problems. This positioning technology has the advantages of wide application scenarios, low cost, rich input, etc., and can perform reliable positioning with high precision in the absence of satellite signals.

[0004] Monocular visual odometry mainly uses images collected by a visual sensor at adjacent times as input to predict the pose transformation of the sensor carrier at adjacent times, and accumulates to obtain the motion trajectory of the entire sensor. Traditional monocular visual odometry mostly uses feature point-based methods, such as ORB-SLAM2. However, feature point-based methods cannot avoid excessive dependence on camera internal parameters and external environmental parameters, which is not conducive to platform transplantation. Compared with traditional monocular visual odometry methods, deep learning methods based on data-driven characteristics have greatly improved robustness in camera internal parameters and complex environmental conditions. Deep learning methods can be divided into supervised and unsupervised methods. Supervised methods use a large amount of high-quality, structured, and labeled data to train the model to obtain high-precision trajectory estimation results, but real depth and pose information, which is difficult to collect in some cases, so supervised methods cannot benefit from large amounts of unlabeled image data. Unsupervised methods can eliminate the dependence on a large amount of training data and can use unlabeled sensor data to train the model, which will save a lot of manpower and resources, and the generalization ability of the model in new scenes that have not been trained is also very good. It is also one of the important directions of the next step of development of artificial intelligence. However, the accuracy and robustness of unsupervised methods are still a challenge. SUMMARY

[0005] In view of the above problems in the prior art, the sequence learning-based unsupervised monocular visual odometry pose estimation method provided by the present application solves the problems of poor pose estimation accuracy and robustness of the existing unsupervised monocular visual odometry technology.

[0006] To achieve the above-mentioned purposes, the technical scheme adopted by the present application is as follows: a sequence learning-based unsupervised monocular visual odometry pose estimation method, comprising the following steps:

[0007] S1, acquiring monocular image sequence data and acquiring a pixel-by-pixel depth image sequence of the monocular image sequence data;

[0008] S2, according to the monocular image sequence data, using a fixed step sliding window to segment, splicing each adjacent two frames of image pairs to obtain a plurality of image pair sequence data;

[0009] S3, according to each image pair sequence data, using a variable step convolutional long short-term memory network to obtain the relative camera pose after multiple pose transformations between image pairs are accumulated;

[0010] S4, according to the pixel-by-pixel depth image sequence of the monocular image sequence data, the monocular image sequence data and the relative camera pose after multiple pose transformations between image pairs are accumulated, based on a self-supervised learning framework for pose estimation, obtaining a pose estimation result.

[0011] Further, the step S2 is specifically:

[0012] S201, using a sliding window with a length of l and a fixed step of 1 to segment the monocular image sequence data to obtain input sequence data with a length of l;

[0013] S202, according to the input sequence data, splicing each adjacent two frames of image pairs along the color channel dimension to obtain a plurality of image pair sequence data.

[0014] ​Further, in step S3, the variable stride convolutional short-term memory network includes a first variable stride convolutional short-term memory network layer, a second variable stride convolutional short-term memory network layer, a third variable stride convolutional short-term memory network layer, a fourth variable stride convolutional short-term memory network layer, a fifth variable stride convolutional short-term memory network layer, a sixth variable stride convolutional short-term memory network layer, a seventh variable stride convolutional short-term memory network layer, and a convolutional network layer connected in sequence; the first variable stride convolutional short-term memory network layer, the second variable stride convolutional short-term memory network layer, the third variable stride convolutional short-term memory network layer, the fourth variable stride convolutional short-term memory network layer, the fifth variable stride convolutional short-term memory network layer, the sixth variable stride convolutional short-term memory network layer, the seventh variable stride convolutional short-term memory network layer, and a convolutional network layer; The number of hidden layers in the second, third, fourth, fifth, sixth, and seventh variable-stride convolutional short-term memory (SSTMM) layers are 16, 32, 64, 16, 16, 16, and 16, respectively; the kernel sizes are 7, 5, 3, 3, 3, 3, and 1, respectively; and the stride is 2. Each of these variable-stride convolutional SSTMM layers includes... A series of variable stride convolutional short-term memory encoder units are connected in sequence; each variable stride convolutional short-term memory encoder unit is connected to the variable stride convolutional short-term memory encoder unit at the same time in the adjacent variable stride convolutional short-term memory network layer. This represents the number of image pair sequence data.

[0015] Furthermore, the variable stride convolutional long short-term memory encoder unit includes cross-correlated motion pattern branches and geometric structure branches.

[0016] Furthermore, the motion pattern branch is used to generate a feature map based on the output of the variable stride convolutional long short-term memory encoder unit at the same time in the previous variable stride convolutional long short-term memory network layer. The state of the motor pattern branch neurons in the previous moment and the short-term memory in the previous moment are controlled by the state of the motor pattern branch neurons, thus obtaining the short-term memory in the current moment. and the current state of the motor pattern branch neurons ;

[0017] The geometric structure branch is used to generate feature maps based on the output of the variable stride convolutional long short-term memory encoder unit at the same time in the previous variable stride convolutional long short-term memory network layer. The state of the geometric branch neurons in the previous time step and the short-term memory in the previous time step are controlled by the state of the geometric branch neurons. Geometric structure of variable stride convolutional short-term memory encoder unit, branches, hidden unit states and the current geometric structure of the branch neuron state .

[0018] Further, the geometric structure branch hidden unit state of the time-varying step-length convolutional long short-term memory encoder unit and the short-term memory of the current time The expressions are respectively:

[0019]

[0020]

[0021]

[0022]

[0023]

[0024]

[0025]

[0026]

[0027]

[0028]

[0029] wherein, is geometric structure branch hidden unit state of the time-varying step-length convolutional long short-term memory encoder unit; is the time sequence relationship of the motion of the geometric structure branch; is the Hadamard product; is the hyperbolic tangent function; is the geometric structure branch neuron state of the current time; is the sigmoid activation function; is the weight of the output gate state of the geometric structure branch; is the weight of the output gate state corresponding to the geometric structure branch neuron state; is the bias corresponding to the output gate state of the geometric structure branch; is the forget gate state of the geometric structure branch; is the geometric structure branch neuron state of the previous time; is the input gate state of the geometric structure branch of the current time; is the weight of the input data candidate information of the geometric structure branch; is the bias corresponding to the input data candidate information of the geometric structure branch; is the weight of the forget gate state of the geometric structure branch; a weight corresponding to a state of a forget gate of the geometric branch for a state of a neuron of the geometric branch; a bias corresponding to a state of a forget gate of the geometric branch; a weight of input data of the geometric branch; a weight corresponding to a state of a neuron of the geometric branch for input data; a bias corresponding to input data of the geometric branch; a time sequence relationship of a motion of the motion pattern branch; a state of a neuron of the motion pattern branch at a current moment; a weight of an output gate state of the motion pattern branch; a weight corresponding to a state of a neuron of the motion pattern branch for an output gate state; a bias corresponding to an output gate state of the motion pattern branch; a state of a forget gate of the motion pattern branch; a state of a neuron of the motion pattern branch at a previous moment; an input gate state of the motion pattern branch at a current moment; a weight of input data candidate information of the motion pattern branch; a bias corresponding to input data candidate information of the motion pattern branch; a weight of a forget gate state of the motion pattern branch; a weight corresponding to a state of a neuron of the motion pattern branch for a forget gate state; a bias corresponding to a forget gate state of the motion pattern branch; a weight of input data of the motion pattern branch; a weight corresponding to a state of a neuron of the motion pattern branch for input data; a bias corresponding to input data of the motion pattern branch; a feature map output by a variable step length convolutional long short-term memory encoder unit at the same moment including a variable step length convolutional long short-term memory layer of a previous layer and input data of a short-term memory at a previous moment.

[0030] Further, the step S3 is specifically:

[0031] S301, according to each image pair sequence data, obtaining high-dimensional motion features with tight coupling space-time information of each image pair sequence data by each variable step length convolutional long short-term memory network layer of the variable step length convolutional long short-term memory network;

[0032] S302, according to the high-dimensional motion features with tight coupling space-time information of each image pair sequence data, completing information interaction between feature channels by using a convolutional network layer of the variable step length convolutional long short-term memory network, and obtaining a 6-DOF pose result of each image pair sequence data.

[0033] S303, respectively convert the 6-DOF pose results of each image pair sequence data into pose transformation matrixes to obtain the pose transformation matrixes of each image pair;

[0034] S304, according to the pose transformation matrixes of each image pair, calculate the relative camera poses after multiple pose transformations between image pairs are accumulated:

[0035]

[0036] wherein, is the relative camera pose between the image collected by the monocular camera at time t and the image collected by the monocular camera at time t-1, is the relative camera pose between the image collected by the monocular camera at time t and the image collected by the monocular camera at time t-1, is the first frame of the sliding window ; is the number of time points; is the relative pose of the first pair of continuous images in the sliding window ; is the relative pose of the third pair of continuous images in the sliding window ; is the relative pose of the first pair of continuous images in the sliding window ; is the relative pose of the first pair of continuous images in the sliding window ; is the relative pose of the first pair of continuous images in the sliding window ; is the relative pose of the first pair of continuous images in the sliding window ; is the relative pose of the first pair of continuous images in the sliding window ; is the relative pose of the first pair of continuous images in the sliding window .

[0037] ​The beneficial effects of the present application are: the present application mines the sequence correlation of continuous local motion based on a variable step length convolutional long short-term memory network composed of a series of mutually correlated sequence artificial neural network basic hidden layer memory units without needing a training data set depth and pose true value label, visual sensor camera parameters and scene point landmarks, and constructs a self-supervised learning framework to jointly calculate a re-projection image using an original RGB image and an estimated pixel-by-pixel depth image, and finally, a high-dimensional pose feature of tightly coupled spatio-temporal information can be accurately and robustly obtained to realize the recovery of a relative scale motion trajectory completely autonomously depending on an unsupervised framework and monocular image data. The high-dimensional spatial structure feature of motion and the local timing relationship of continuous motion calculated by the sequence artificial neural network basic hidden layer memory unit are input into the next encoder layer at the same time and the next time memory unit of the same layer encoder respectively to guide the network to learn the local context information of continuous motion and the timing relationship of motion, the image spatial structure feature extracted by the previous layer and the previous time short-term memory representing the timing relationship are fused into each encoder unit to model the time dependence information in the continuous motion geometry information of the sensor data, and finally, the pose change estimation value of the camera carrier between adjacent camera sampling points is obtained through optimization iteration to achieve a high-precision pose estimation model target. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 The method flowchart of the present application.

[0039] Figure 2 The variable step length convolutional long short-term memory network structure block diagram of the multi-layer network structure of the present application.

[0040] Figure 3 The variable step length convolutional long short-term memory encoder unit structure block diagram of the present application. DETAILED DESCRIPTION

[0041] The specific embodiments of the present application are described below to facilitate the understanding of the present application by those skilled in the art, but it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the present application defined and determined by the appended claims, and all the inventions utilizing the concept of the present application are within the scope of protection.

[0042] As shown in Figure 1 In one embodiment of the present application, a sequence learning based unsupervised monocular visual odometry pose estimation method comprises the following steps:

[0043] S1, acquiring monocular image sequence data and acquiring a pixel-by-pixel depth image sequence of the monocular image sequence data;

[0044] S2, according to monocular image sequence data, segmentation is performed by using a fixed step sliding window, and each adjacent two image pairs are spliced to obtain a plurality of image pair sequence data;

[0045] S3, according to each image pair sequence data, a variable step convolution long short-term memory network is used to obtain a relative camera pose accumulated after multiple pose transformations between image pairs;

[0046] S4, according to the pixel-by-pixel depth image sequence of the monocular image sequence data, the monocular image sequence data and the relative camera pose accumulated after multiple pose transformations between image pairs, a self-supervised learning framework for pose estimation is used to obtain a pose estimation result.

[0047] In the step S1, a corresponding pixel-by-pixel depth map is calculated for each monocular image sampled by the monocular visual sensor through a depth estimation network.

[0048] The step S2 is specifically:

[0049] S201, a sliding window with a length of l and a fixed step of 1 is used to segment the monocular image sequence data to obtain input sequence data with a length of l;

[0050] S202, according to the input sequence data, each adjacent two images are spliced along the color channel dimension to obtain a plurality of image pair sequence data.

[0051] In the embodiment, the specific method of the step S2 is:

[0052] S2-1, a sliding window with a length of 3 and a fixed step of 1 is used to segment the image sequence sampled by the monocular visual sensor in the step S1 to obtain input sequence data with a length of 3;

[0053] S2-2, the segmented input sequence data is spliced along the color channel dimension according to each adjacent two images. The representation form of the image pair sequence data as the model input is specifically a four-dimensional tensor form, wherein, t, c, h, w is the number of image pairs, that is, the total number of images in the input sequence segmented in the step S2-1 minus 1, t is the number of image channels obtained by splicing the three-channel color RGB images of the adjacent two images sampled in the step S1; c and h are the pixel numbers of the height and width of the image, respectively. w

[0054] ​​The single-vision sensor has a sampling frequency of 20 Hz; the data dimension of the three-channel color RGB image is (3, 128, 416); in step S1, each three-channel color RGB image frame corresponds to a calculated pixel-by-pixel depth map, and the data dimension of the pixel-by-pixel depth map is (1, 128, 416); and the data dimension of the model input obtained in step S2 is (2, 6, 128, 416).

[0055] As shown in Figure 2 , the variable step length convolutional long short-term memory network in step S3 includes a first variable step length convolutional long short-term memory network layer, a second variable step length convolutional long short-term memory network layer, a third variable step length convolutional long short-term memory network layer, a fourth variable step length convolutional long short-term memory network layer, a fifth variable step length convolutional long short-term memory network layer, a sixth variable step length convolutional long short-term memory network layer, a seventh variable step length convolutional long short-term memory network layer, and a convolutional network layer connected in sequence; the number of hidden layers of the first variable step length convolutional long short-term memory network layer, the second variable step length convolutional long short-term memory network layer, the third variable step length convolutional long short-term memory network layer, the fourth variable step length convolutional long short-term memory network layer, the fifth variable step length convolutional long short-term memory network layer, the sixth variable step length convolutional long short-term memory network layer, and the seventh variable step length convolutional long short-term memory network layer are 16, 32, 64, 16, 16, 16, and 16, respectively; the convolution kernel sizes are 7, 5, 3, 3, 3, 3, and 1, respectively; and the steps are all 2; each variable step length convolutional long short-term memory network layer includes a variable step length convolutional long short-term memory encoder unit connected in sequence; each variable step length convolutional long short-term memory encoder unit is connected to a variable step length convolutional long short-term memory encoder unit at the same time of an adjacent variable step length convolutional long short-term memory network layer; The number of image pairs in the sequence data.

[0056] In this embodiment, the convolutional network layer is a convolutional layer with a channel number of 16 and a convolution kernel size of 1x1. The hidden unit state of the last variable step length convolutional long short-term memory network layer stores motion features containing tightly coupled space-time information, and the dimension is (1, 6); and the output of the convolutional neural network is a 6-DOF pose result estimated by the pose network. The representation form of the 6-DOF pose transformation is specifically six-dimensional x , y , z , α , β , γ , wherein x , y , z is the translation of the monocular vision sensor in the three-dimensional space between the shooting of two image frames.x spatial transformation of an axis, y spatial transformation of an axis and z spatial transformation of an axis. α , β , γ represents Euler angles, respectively, a pitch angle transformation, a yaw angle transformation and a roll angle transformation of the sensor in a three-dimensional space.

[0057] As shown in Figure 3 , the variable step length convolutional long short-term memory encoder unit comprises a motion mode branch and a geometric structure branch which are correlated.

[0058] The motion mode branch is configured to obtain, according to a feature map output by a variable step length convolutional long short-term memory encoder unit at a same time of a previous layer of variable step length convolutional long short-term memory network layers, a motion mode branch neuron state at a previous time and a short-term memory at the previous time, a short-term memory at a current time and a motion mode branch neuron state at the current time controlled by the motion mode branch neuron state.

[0059] The geometric structure branch is configured to obtain, according to a feature map output by a variable step length convolutional long short-term memory encoder unit at a same time of a previous layer of variable step length convolutional long short-term memory network layers, a geometric structure branch neuron state at a previous time and a short-term memory at the previous time, a geometric structure branch hidden unit state of the variable step length convolutional long short-term memory encoder unit at the current time and a geometric structure branch neuron state at the current time controlled by the geometric structure branch neuron state.

[0060] In this embodiment, as shown in Figure 3 , the structure designed by the present application extracts time and space structures and fully fuses information to guide feature extraction at a next layer and a next time. Figure 3 is a feature map output by a variable step length convolutional long short-term memory encoder unit at a same time of a previous layer of variable step length convolutional long short-term memory network layers; is a geometric structure branch hidden unit state of the variable step length convolutional long short-term memory encoder unit at the current time; is a short-term memory at a previous time; is a motion mode branch neuron state at the previous time; is a geometric structure branch neuron state at the previous time; is a short-term memory at the current time;​​​​ is the state of the motion branch neuron at the current time step; is the state of the geometry branch neuron at the current time step.

[0061] The is the state of the geometry branch hidden unit of the variable step length convolutional long short term memory encoder unit at the current time step is the short term memory at the current time step The expressions of the above are respectively:

[0062]

[0063]

[0064]

[0065]

[0066]

[0067]

[0068]

[0069]

[0070]

[0071]

[0072] wherein, is the state of the geometry branch hidden unit of the variable step length convolutional long short term memory encoder unit at the current time step is the state of the geometry branch hidden unit of the variable step length convolutional long short term memory encoder unit at the current time step is the temporal relationship of the motion of the geometry branch; is the Hadamard product; is the hyperbolic tangent function; is the state of the geometry branch neuron at the current time step; is the sigmoid activation function; is the weight of the output gate state of the geometry branch; is the weight of the output gate state corresponding to the state of the geometry branch neuron; is the bias corresponding to the output gate state of the geometry branch; is the forget gate state of the geometry branch; is the state of the geometry branch neuron at the previous time step; is the input gate state of the geometry branch at the current time step; is the weight of the input data candidate information of the geometry branch; The bias corresponding to the candidate information of the input data for the geometric structure branch; The weights of the forget gate states for the geometric structure branches; The weights of the neuron states in the geometric branch corresponding to the forget gate states; The bias corresponding to the forget gate state of the geometric structure branch; Weights for the input data of the geometric structure branches; The weights are the states of neurons in the geometric branches corresponding to the input data; The bias corresponding to the input data of the geometric structure branch; The temporal relationship of motion branches for motion patterns; The state of the neuron in the current motion pattern branch; The weights of the output gate states for the motion mode branch; The weights of the neuron states in the motion mode branch corresponding to the output gate state; The bias corresponding to the output gate state of the motion mode branch; Forget gate state for movement pattern branch; This represents the state of the neuron branching the motion pattern at the previous moment; Input gate state for the current motion mode branch; The weights of the candidate information for the input data of the motion pattern branch; The bias corresponding to the input data candidate information for the motion pattern branch; The weights of the forget gate states for the motion pattern branches; The weights of the neuron states in the motor pattern branch corresponding to the forget gate states; The bias corresponding to the forget gate state of the motion mode branch; Weights for the input data of the motion pattern branch; The weights of the neuron states in the corresponding motion pattern branches of the input data; The bias corresponding to the input data of the motion mode branch; The feature map is the output of the variable stride convolutional long short-term memory encoder unit at the same time as the previous variable stride convolutional long short-term memory network layer. Input data from the previous short-term memory.

[0073] Step S3 specifically involves:

[0074] S301. Based on the sequence data of each image pair, the high-dimensional motion features containing tightly coupled spatiotemporal information of each image pair sequence data are obtained through each variable stride convolutional long short-term memory network layer of the variable stride convolutional long short-term memory network.

[0075] S302, according to the high-dimensional motion features of the image pair sequence data containing tight coupling space-time information, the information interaction between feature channels is completed by using the convolution network layer of the variable step length convolution long short-term memory network, and the 6-DOF pose results of the image pair sequence data are obtained;

[0076] S303, the 6-DOF pose results of the image pair sequence data are converted into pose transformation matrices respectively, and the pose transformation matrices of the image pairs are obtained;

[0077] S304, according to the pose transformation matrices of the image pairs, the relative camera poses after multiple pose transformations between the image pairs are calculated:

[0078]

[0079] Wherein, is the relative camera pose between the image collected by the monocular camera at time t and the image collected by the monocular camera at time t-1, is the relative camera pose between the image collected by the monocular camera at time t and the image collected by the monocular camera at time t-1, is the first frame of the sliding window ; is the number of time points; is the relative pose of the first pair of continuous images in the sliding window ; is the relative pose of the third pair of continuous images in the sliding window ; is the relative pose of the first pair of continuous images in the sliding window ; is the relative pose of the first pair of continuous images in the sliding window ; is the relative pose of the first pair of continuous images in the sliding window ; is the relative pose of the first pair of continuous images in the sliding window ; is the relative pose of the first pair of continuous images in the sliding window ; is the relative pose of the first pair of continuous images in the sliding window .

[0080] ​In this embodiment, the application mines the sequence correlation of continuous local motion based on a variable step length convolutional long short-term memory network composed of a series of interrelated sequence artificial neural network basic hidden layer memory units without requiring a training data set depth and pose truth label, visual sensor camera parameters and scene point landmarks, and constructs a self-supervised learning framework to jointly calculate a re-projection image using an original RGB image and an estimated pixel-by-pixel depth image, and finally accurately and robustly obtains high-dimensional pose features of tightly coupled spatio-temporal information, thereby achieving a relative scale motion trajectory that is completely autonomously recovered only by relying on an unsupervised framework and monocular image data.

[0081] Based on the high-dimensional spatial structure features of motion and the local timing relationship of continuous motion calculated by the basic hidden layer memory units of the sequence artificial neural network, the next encoder layer at the same time and the next time memory unit of the same layer encoder are input to guide the network to learn the local context information of continuous motion and the timing relationship of motion, the image spatial structure features extracted by the previous layer and the previous time short-term memory representing the timing relationship are fused into each encoder unit to model the time dependence information in the continuous motion geometry information in the sensor data, and finally the pose change estimation value of the camera carrier between adjacent camera sampling points is obtained through optimization iteration, thereby achieving a high-precision pose estimation model target.

Claims

1. A sequence learning based unsupervised monocular visual odometry pose estimation method, characterized in that, The method comprises the following steps: S1, acquiring monocular image sequence data and acquiring a pixel-by-pixel depth image sequence of the monocular image sequence data; S2, according to the monocular image sequence data, performing segmentation by using a fixed step sliding window, splicing each adjacent two frames of image pairs to obtain a plurality of image pair sequence data; S3, according to the image pair sequence data, using a variable step convolutional long short-term memory network to obtain a relative camera pose accumulated after multiple pose transformations between image pairs; S4, according to the pixel-by-pixel depth image sequence of the monocular image sequence data, the monocular image sequence data and the relative camera pose accumulated after multiple pose transformations between image pairs, obtaining a pose estimation result based on a self-supervised learning framework for pose estimation.

2. The unsupervised monocular visual odometry pose estimation method based on sequence learning according to claim 1, characterized in that, The step S2 is specifically: S201、adopt length as And fixed step length is 1 sliding window monocular image sequence data segmentation, get length l input sequence data; S202, according to the input sequence data, splicing each adjacent two frames of images along the color channel dimension to obtain a plurality of image pair sequence data.

3. The unsupervised monocular visual odometry pose estimation method based on sequence learning according to claim 1, characterized in that, The variable step convolutional long short-term memory network in the step S3 comprises a first variable step convolutional long short-term memory network layer, a second variable step convolutional long short-term memory network layer, a third variable step convolutional long short-term memory network layer, a fourth variable step convolutional long short-term memory network layer, a fifth variable step convolutional long short-term memory network layer, a sixth variable step convolutional long short-term memory network layer, a seventh variable step convolutional long short-term memory network layer and a convolutional network layer connected in sequence; The first variable step length convolutional long short-term memory network layer, the second variable step length convolutional long short-term memory network layer, the third variable step length convolutional long short-term memory network layer, the fourth variable step length convolutional long short-term memory network layer, the fifth variable step length convolutional long short-term memory network layer, the sixth variable step length convolutional long short-term memory network layer, and the seventh variable step length convolutional long short-term memory network layer have hidden layer numbers of 16, 32, 64, 16, 16, 16, and 16 respectively, and convolution kernel sizes of 7, 5, 3, 3, 3, 3, and 1 respectively, and all have a step length of 2; each variable step length convolutional long short-term memory network layer includes one variable step length convolutional long short-term memory encoder unit connected in sequence; each variable step length convolutional long short-term memory encoder unit is connected with a variable step length convolutional long short-term memory encoder unit of the same time of an adjacent variable step length convolutional long short-term memory network layer; is the number of image pairs.

4. The unsupervised monocular visual odometry pose estimation method based on sequence learning according to claim 3, characterized in that, The variable step convolutional long short-term memory encoder unit comprises a motion mode branch and a geometric structure branch which are correlated with each other.

5. The unsupervised monocular visual odometry pose estimation method based on sequence learning according to claim 4, characterized in that, The motion mode branch is used for obtaining the feature map output by the variable step length convolution long short-term memory encoder unit at the same time of the last variable step length convolution long short-term memory network layer according to the feature map output by the variable step length convolution long short-term memory encoder unit at the same time of the last variable step length convolution long short-term memory network layer , the motion mode branch neuron state at the previous moment and the short-term memory at the previous moment, wherein the short-term memory at the current moment is obtained by controlling the motion mode branch neuron state and the motion mode branch neuron state at the current moment ​ The geometry branch is used for outputting a feature map according to a variable step length convolution long short-term memory encoder unit at the same time of a variable step length convolution long short-term memory network layer of a previous layer , a geometry branch neuron state at a previous moment and short-term memory at the previous moment, controlled by the geometry branch neuron state, to obtain A geometry branch hidden cell state of the variable step length convolution long short-term memory encoder unit at the current moment And a geometry branch neuron state at the current moment .

6. The unsupervised monocular visual odometry pose estimation method based on sequence learning according to claim 5, characterized in that, The Geometric structure branch hidden unit state of time-varying step length convolutional long short-term memory encoder unit Short-term memory of the current time The expressions are respectively: wherein, is the geometric structure branch hidden unit state of the time-varying step convolutional long short-term memory encoder unit at the current time; is the time sequence relationship of the motion of the geometric structure branch; is the Hadamard product; is the hyperbolic tangent function; is the geometric structure branch neuron state at the current time; is the sigmoid activation function; is the weight of the output gate state of the geometric structure branch; is the weight of the output gate state corresponding to the geometric structure branch neuron state; is the bias corresponding to the output gate state of the geometric structure branch; is the forget gate state of the geometric structure branch; is the geometric structure branch neuron state at the previous time; is the input gate state of the geometric structure branch at the current time; is the weight of the input data candidate information of the geometric structure branch; is the bias corresponding to the input data candidate information of the geometric structure branch; is the weight of the forget gate state of the geometric structure branch; is the weight of the forget gate state corresponding to the geometric structure branch neuron state; is the bias corresponding to the forget gate state of the geometric structure branch; is the weight of the input data of the geometric structure branch; is the weight of the input data corresponding to the geometric structure branch neuron state; is the bias corresponding to the input data of the geometric structure branch; is the time sequence relationship of the motion of the motion pattern branch; is the motion pattern branch neuron state at the current time; is the weight of the output gate state of the motion pattern branch; is the weight of the output gate state corresponding to the motion pattern branch neuron state; is the bias corresponding to the output gate state of the motion pattern branch; is the forget gate state of the motion pattern branch; is the motion pattern branch neuron state at the previous time; is the input gate state of the motion pattern branch at the current time; is the weight of the input data candidate information of the motion pattern branch; is the bias corresponding to the input data candidate information of the motion pattern branch; is the weight of the forget gate state of the motion pattern branch; a weight corresponding to a state of a motion pattern branch neuron for a forget gate state; a bias corresponding to a motion pattern branch for a forget gate state; a weight for input data of a motion pattern branch; a weight corresponding to a state of a motion pattern branch neuron for input data; a bias corresponding to input data of a motion pattern branch; a feature map output by a variable step length convolutional long short term memory encoder unit at the same time including a variable step length convolutional long short term memory network layer of a previous layer and input data of a short term memory at a previous time.

7. The method of claim 1, wherein, The step S3 is specifically: S301, according to the image pair sequence data, passing through each variable step convolutional long short-term memory network layer of the variable step convolutional long short-term memory network to obtain high-dimensional motion features of each image pair sequence data containing tight coupling space-time information; S302, according to the high-dimensional motion features of each image pair sequence data containing tight coupling space-time information, using the convolutional network layer of the variable step convolutional long short-term memory network to complete information interaction between feature channels to obtain a 6-DOF pose result of each image pair sequence data; S303, converting the 6-DOF pose result of each image pair sequence data into a pose transformation matrix respectively to obtain a pose transformation matrix of each image pair; S304, according to the pose transformation matrix of each image pair, calculating a relative camera pose accumulated after multiple pose transformations between image pairs: in, for Images captured by a monocular camera at all times and The relative camera pose between images captured by a monocular camera at any given time. The images acquired by the monocular camera at any given time are in a sliding window. The first frame; Number of moments; In the sliding window The Middle Relative pose of continuous images; The length of the sliding window; In the sliding window The relative pose of the third pair of consecutive images in the middle; In the sliding window The relative pose of the first pair of consecutive images in the middle; In the sliding window The relative pose of the first pair of consecutive images in the middle; In the sliding window The relative pose of the first pair of consecutive images in the middle; In the sliding window The relative pose of the first pair of consecutive images in the middle; In the sliding window The relative pose of the first pair of consecutive images in the image.

Images