Indoor image localization method based on similarity measurement of siamese network and improved eightpoint-ransac algorithm

Abstract

The application discloses an indoor image positioning method based on similarity measurement of Siamese Network and improved EightPoint-RANSAC algorithm. Firstly, multiple reference points are arranged in a target area, and 90% and 10% are used to divide a training set and a test set; then, a Siamese Network framework based on a convolutional neural network is constructed to extract deep features of images and train similarity measurement thereof; finally, a basic matrix is calculated and optimized according to the matched images by using the improved EightPoint-RANSAC algorithm, camera pose is solved, and a user position is estimated. The application discloses an indoor image positioning method based on similarity measurement of Siamese Network and improved EightPoint-RANSAC algorithm, which uses the similarity between images taken at reference points and images in an offline database, reduces system overhead, and ensures positioning accuracy.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of indoor positioning, and particularly relates to an indoor image positioning method based on similarity measurement of Siamese Network and improved EightPoint-RANSAC algorithm. BACKGROUND

[0002] With the development of mobile communication technology and wireless network, location-based services (LBS) play an important role in modern people's life. Global Positioning System (GPS) and positioning technology based on cellular base stations (GSM) basically meet the positioning service needs of users in outdoor scenarios. However, in indoor scenarios, GPS signals will rapidly attenuate or even completely fail to reach due to the obstruction of walls and buildings. Therefore, in indoor positioning scenarios, traditional GPS and GSM cannot meet the positioning needs of users. In order to solve this problem, people began to research and develop indoor positioning methods based on other technologies. Among them, indoor positioning technologies using Wi-Fi, Infrare, Radio Frequency Identification (RFID), Bluetooth, and Ultra Wide Band (UWB) technologies gradually emerged. Although these technologies have certain applications, in order to achieve higher accuracy, additional external infrastructure is often needed to assist, which may be expensive. With the popularization of smart phones and the progress of technology, indoor positioning technology based on visual images has attracted widespread attention due to its low cost and high accuracy, and has gradually been applied to various fields.

[0003] 2D-2D image positioning generally includes two stages of offline and online stages. The task of the offline stage is to establish an image feature database, which is a prerequisite for feature matching and position estimation. The feature information of each image is stored in the offline database; in the online stage, the test image is taken by the camera and the feature points are extracted, and the feature matching is performed with the database image, the matching image is obtained, and the real position of the image is estimated according to the feature point pair relationship of the matching image, and the user positioning is realized. The current feature retrieval and matching method mainly depends on the manually designed feature extraction algorithm, which needs to manually select the feature extractor and the matching algorithm, and then adjust the parameters to adapt to different scenes and applications. Some complex or irregular scenes may not be able to be well extracted and matched by traditional manual methods. With the expansion of the scene, the number of feature points in the image will increase significantly, which makes the feature retrieval and matching process more complex and time-consuming. At the same time, the changes of light, image quality and scene change and other factors also bring challenges to feature matching. After image matching, the user position needs to be estimated. At present, the indoor positioning based on image is usually through calculating the Euclidean distance between the feature points of the database image and the feature points of the online image, and selecting the corresponding reference point position of the feature point with the highest matching degree as the final positioning result. However, in practical applications, if the user's position is directly estimated as the position of the reference point, it will undoubtedly bring great error, so a certain algorithm or model is needed to obtain a more accurate user position. Therefore, improving the efficiency of image retrieval and matching and estimating the user position through the matching image are the key and difficulty of 2D-2D indoor image positioning technology.

[0004] Therefore, in terms of image retrieval and matching, the application proposes a SiameseNetwork framework, which extracts higher-level and more representative features by simultaneously inputting two training images to be matched into two neural networks with the same weight; by dividing the positive and negative sample pairs and marking them as positive and negative, learning the similarity measurement model between images, using the training loss function to measure the difference between the model prediction result and the true label, and judging the matching degree of the two images. In terms of image-based position estimation, the application proposes a position estimation method based on an improved EightPoint-RANSAC algorithm, which uses the epipolar geometry principle to grid the image according to the feature weight to filter the feature points, and fuses the RANSAC algorithm to remove noise and mismatched points, solves the optimal fundamental matrix to solve the camera pose change, and estimates the user's position. SUMMARY

[0005] The application aims to provide a visual image positioning method based on Siamese Network similarity measurement and improved EightPoint-RANSAC algorithm. In the offline stage, an image sample database is constructed, and a Siamese Network comparison learning framework based on a convolutional neural network is built to learn the similarity measurement between features; in the online stage, the trained model is used to realize image matching and find the most matched image with the test image, and the optimal fundamental matrix is calculated by using the improved EightPoint-RANSAC algorithm, the camera pose change is solved, and the user position is obtained, so that the positioning accuracy is ensured while the system overhead is balanced.

[0006] The visual image positioning method based on Siamese Network similarity measurement and improved EightPoint-RANSAC algorithm comprises the following steps:

[0007] Step one, 31 position known reference points (Access Point, AP) are deployed in the corridor and the room, and are recorded as AP1,..., AP m , the jth AP is recorded as AP j (j≤m, j is an integer). The lower right corner of the corridor is taken as the coordinate origin of the world coordinate system, the direction of the corridor is taken as the x-axis and y-axis of the world coordinate system, and the real two-dimensional coordinate position of each reference point is recorded.

[0008] Step two, a handheld camera is used to shoot a certain number of images at each reference point, including three directions of 45°, 90° and 135°. These images are used to construct an image database, wherein 90% of the images are used as a training set, and 10% of the images are used as a test set. All the training images and test images have a resolution of 3024*4032 pixels.

[0009] Step three, a Siamese Network comparison learning framework based on a convolutional neural network is constructed to learn effective image feature representation and similarity measurement method. Specifically, the following steps are included:

[0010] Step three (one), through the data preprocessing module, create positive and negative sample pairs of image sample library. Specifically, that is, to the image pair in the data set with similar or dissimilar label. Positive sample pair represents the high similarity between two images, while negative sample pair represents the low similarity between two images. Specifically, a pair of images is selected from the data set, which is regarded as a sample pair. Different images from the same reference point are regarded as positive sample pairs, and different images from different reference points are regarded as negative sample pairs. Y is used to represent the similarity of positive and negative sample pairs. If images X1, X2 are positive sample pairs, Y = 1 is considered, and the label of the sample pair is set to (X1, X2, 1), otherwise (X1, X2, 0). The positive and negative division of sample pairs not only simplifies the model, reduces the data demand, but also improves the performance and generalization ability of the model, which can ensure that the model learns useful features and distinguishes similar and dissimilar images.

[0011] Step three (two), process image data through convolutional neural network and extract deep features of image from input image. One of the main features of Siamese Network is that it contains two identical convolutional neural networks, which share the same network structure, weights and bias. Convolutional neural network usually includes convolutional layer, pooling layer and fully connected layer, and the extraction of image features is realized through forward propagation process. Convolutional neural network can be regarded as a nonlinear function, that is:

[0012] h i =f(X i ),i=1,2 (1)

[0013] Where h is the extracted feature vector.

[0014] Table I network parameter setting

[0015]

[0016] Step three (three), after completing feature extraction, similarity measurement training is carried out in contrast loss module. Define contrast loss function, which can effectively process the relationship between paired samples in Siamese neural network. The expression is as follows:

[0017]

[0018] Where (h1, h2, Y) i is the i-th sample pair, h1, h2 represents the feature vector of a pair of images, and Y represents the similarity; D(E w (h1, h2)) = ||h1-h2||2 represents the Euclidean distance of sample features, D G is the partial loss function of positive sample pair, D Iis the partial loss function of negative sample pair, and N is the number of samples. The training of the loss function is measured by the Euclidean distance of the feature vectors in the feature space. Specifically, when the samples are similar, Y = 1, and the loss function is only That is, if the Euclidean distance in the feature space is large, it means that the current model is not good, and the loss needs to be increased to narrow the distance of the samples. Conversely, when the samples are not similar, Y = 0, and the loss function is That is, if the Euclidean distance in the feature space is small, the loss value becomes large.

[0019] Step three (four), input the test image into the trained model, match with the database image, and output the most matched image.

[0020] Step four, use the improved EightPoint-RANSAC algorithm to calculate and optimize the fundamental matrix, to further solve the camera pose change and estimate the user's position. The specific steps are as follows:

[0021] Step four (one), initial matching point query. In order to prevent the distribution of extracted pixel points from being too concentrated, the image plane is divided into several grids using grid segmentation technology. The weight of each network is defined according to the number of feature points distributed in each network, so as to determine the probability of extracting pixel points in each grid. Such processing ensures that the selected matching feature points come from different areas. Then, the Euclidean distance of the feature point pairs is calculated, and the first eight pairs of feature points with the smallest Euclidean distance are selected as the initial value of the calculation of the fundamental matrix according to the grid weight.

[0022] Step four (two), calculation and optimization of the fundamental matrix. The expression of the fundamental matrix is obtained by using the principle of epipolar geometry:

[0023] (u i ′ ,j ) T Fu i,j =0 (2)

[0024] Where, F = (K -1 ) T t^RK -1 is the fundamental matrix, and E = t^R is the essential matrix. In order to calculate the error of all matching points in the model, the method of minimizing the projection error is used, and the projection square of u i,j is expressed as:

[0025]

[0026] Where, d represents the transfer error of the offline image, d' represents the transfer error of the query image, and obeys a m-degree of freedom ​where m is equal to the residual dimension of the model. The random variable has a cumulative chi 2 -squared distribution with k 2 degrees of freedom . If the pixel distance of the matched points is greater than a set threshold, it is considered that the pixel coordinates of the matched feature points are distorted, and the inner points with distortion are removed. If it is less than the pixel threshold, the pixel drift is within an acceptable range, and such matched feature points are recorded as a new set of inner points. When the number of inner points reaches a certain threshold, the iteration is stopped, and the model with the most inner points is retained as the final model. If there are not enough inner points, the fundamental matrix is recalculated.

[0027] Step four (three), motion and structure recovery. Singular value decomposition is performed on the essential matrix to obtain:

[0028] E ~ U diag(1,1,0) V T (5)

[0029] where diag(1,1,0) is a diagonal matrix with elements (1,1,0) on the diagonal. Since the rank of the essential matrix is 2, the SVD decomposition has only two non-zero singular values, corresponding to the two main directions of the essential matrix. When calculating the camera pose, since the last column of the decomposed V matrix corresponds to the right singular matrix vector corresponding to the smallest singular value of the essential matrix, it represents the third main direction of the essential matrix, which can be used to represent the rotation matrix R of the camera. The translation matrix t can be calculated through the third column of the essential matrix, as follows:

[0030] t ~ t u = [u 13 ,u 23 ,u 33 ] T (6)

[0031] R a = UD V T or R b = UD T V T (7)

[0032] where det(U)>0,det(V)>0, D is a coefficient matrix. Four possible motions can be obtained:

[0033]

[0034] In order to ensure that the image feature points are in front of the camera, a unique rotation matrix and translation matrix can be obtained from the above assumptions to solve the camera pose.

[0035] Step four (four), estimate the user position according to the rotation matrix and translation matrix of the camera.

[0036] Advantages

[0037] The present application is based on image matching and image-based positioning. Firstly, multiple reference points are deployed within the target area, and the training set and test set are divided by 90% and 10%. Secondly, a Siamese Network framework based on convolutional neural network is constructed to extract deep features of images and train similarity measurement. Finally, the improved EightPoint-RANSAC algorithm is used to calculate and optimize the fundamental matrix according to the matched images, solve the camera pose, and estimate the user position. The present application reduces the overhead while effectively improving the performance of the positioning system. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 The flowchart of the present application;

[0039] Figure 2 The system framework diagram of the present application;

[0040] Figure 3 The Siamese Network framework diagram;

[0041] Figure 4 The convolutional neural network structure diagram; DETAILED DESCRIPTION

[0042] The present application will be further described below in conjunction with the accompanying drawings:

[0043] The visual image positioning method based on Siamese Network similarity measurement and improved EightPoint-RANSAC algorithm according to the present application comprises the following steps:

[0044] Step one, 31 position known reference points (Access Point, AP) are deployed in the corridor and room areas, denoted as AP1,…,AP m , the jth AP is denoted as AP j (j≤m, j is an integer). The lower right corner of the corridor is taken as the coordinate origin of the world coordinate system, and the direction of the corridor is taken as the x and y axes of the world coordinate system. The real two-dimensional coordinate positions of each reference point are recorded.

[0045] Step two, a handheld camera is used to take a certain number of images at each reference point, including 45°, 90° and 135° directions. These images are used to construct an image database, in which 90% of the images are used as the training set and 10% of the images are used as the test set. All the training images and test images have a resolution of 3024*4032 pixels.

[0046] Step three, build a Siamese Network based on convolutional neural network contrast learning framework, learn effective image feature representation and similarity measurement method. Specifically includes the following steps:

[0047] Step three (one), through the data preprocessing module, create positive and negative sample pairs of image sample library. Specifically, that is, the image pairs in the data set are labeled as similar or dissimilar. Positive sample pair represents high similarity between two images, while negative sample pair represents low similarity between two images. Specifically, a pair of images is selected from the data set and regarded as a sample pair. Different images from the same reference point are regarded as positive sample pairs, and different images from different reference points are regarded as negative sample pairs. Y is used to represent the similarity of positive and negative sample pairs. If images X1 and X2 are positive sample pairs, Y = 1 is considered, and the label of the sample pair is set to (X1, X2, 1), otherwise (X1, X2, 0). The positive and negative division of sample pairs not only simplifies the model, reduces the data demand, but also improves the performance and generalization ability of the model, which can ensure that the model learns useful features and distinguishes similar and dissimilar images.

[0048] Step three (two), process image data through convolutional neural network and extract deep features of image from input image. One of the main features of Siamese Network is that it contains two identical convolutional neural networks, which share the same network structure, weights and bias. Convolutional neural network usually includes convolutional layer, pooling layer and fully connected layer, and realizes the extraction of image features through forward propagation process. Convolutional neural network can be regarded as a nonlinear function, that is:

[0049] h i =f(X i ),i=1,2 (1)

[0050] Where h is the extracted feature vector.

[0051] Table I network parameter setting

[0052]

[0053] Step three (three), after completing feature extraction, train the similarity measurement in the contrast loss module. Define the contrast loss function, which can effectively handle the relationship between paired samples in Siamese neural network. The expression is as follows:

[0054]

[0055] Where (h1, h2, Y) iis the i-th sample pair, h1, h2 represent the feature vectors of a pair of images, Y represents the similarity degree; D(E w (h1, h2)) = ||h1 - h2||2 represents the Euclidean distance of sample features, D G is the partial loss function of positive sample pairs, D I is the partial loss function of negative sample pairs, N is the number of samples. The training of the loss function is measured by the Euclidean distance of the feature vectors in the feature space. Specifically, when the samples are similar, Y = 1, and the loss function is only That is, if the Euclidean distance in the feature space is large, it means that the current model is not good, and the loss needs to be increased to narrow the distance of the samples. Conversely, when the samples are not similar, Y = 0, and the loss function is That is, if the Euclidean distance in the feature space is small, the loss value becomes large.

[0056] Step three (four), input the test image into the trained model, match with the database image, and output the most matched image.

[0057] Step four, use the improved EightPoint-RANSAC algorithm to calculate and optimize the fundamental matrix, and further solve the camera pose change to estimate the user's position. The specific steps are as follows:

[0058] Step four (one), initial matching point query. In order to prevent the distribution of extracted pixel points from being too concentrated, the image plane is divided into several grids using grid segmentation technology. The weight of each network is defined according to the number of feature points distributed in each network, so as to determine the probability of extracting pixel points in each grid. Such processing ensures that the selected matching feature points come from different areas. Then, the Euclidean distance of the feature point pairs is calculated, and the first eight pairs of feature points with the smallest Euclidean distance are selected as the initial value of the calculation of the fundamental matrix according to the grid weight.

[0059] Step four (two), calculation and optimization of the fundamental matrix. The expression of the fundamental matrix is solved using the principle of epipolar geometry:

[0060] (u′ i,j ) T Fu i,j = 0 (2)

[0061] Where, F = (K -1 ) T t^RK -1 is the fundamental matrix, and E = t^R is the essential matrix. In order to calculate the error of all matching points in the model, the method of minimizing the projection error is used, and the projection square of u i,j is expressed as: ​

[0062]

[0063] Where d represents the transfer error of the offline image, and d′ represents the transfer error of the query image, both following a system of m degrees of freedom. The distribution, where m equals the codimensionality of the model. Random variable. The value is less than k 2 The probability is determined by the cumulative χ² 2 distributed The algorithm is as follows: If the pixel distance between matching points is greater than a set threshold, the pixel coordinates of the matching feature points are considered distorted, and the distorted inliers are removed. If the distance is less than the pixel threshold, the pixel drift is within an acceptable range, and such matching feature points are recorded as a new set of inliers. When the number of inliers reaches a certain threshold, the iteration stops, and the model with the most inliers is retained as the final model. If there are not enough inliers, the fundamental matrix is ​​recalculated.

[0064] Step 4 (3), Motion and Structure Restoration. Singular value decomposition of the essential matrix yields:

[0065] E~Udiag(1,1,0)V T (5)

[0066] Here, diag(1,1,0) is a diagonal matrix with elements (1,1,0) on the diagonal. Since the rank of the essential matrix is ​​2, the SVD decomposition has only two non-zero singular values, corresponding to the two principal directions of the essential matrix. When calculating the camera pose, the last column of the decomposed V matrix corresponds to the right singular matrix vector corresponding to the smallest singular value of the essential matrix, representing the third principal direction of the essential matrix. This can be used to represent the camera's rotation matrix R, and the translation matrix t can be calculated using the third column of the essential matrix, as follows:

[0067] t~t u =[u 13 ,u 23 ,u 33 ] T (6)

[0068] R a =UDV T or R b =UD T V T (7)

[0069] Where det(U)>0, det(V)>0, and D is the coefficient matrix. Four possible motions can be obtained:

[0070]

[0071] To ensure that the image feature points are in front of the camera, a unique rotation matrix and translation matrix can be obtained from the above assumptions, and the pose of the camera is solved.

[0072] Step four (four), estimate the user position according to the rotation matrix and translation matrix of the camera.

Claims

1. A visual image localization method based on SiameseNetwork similarity measurement and an improved EightPoint-RANSAC algorithm, characterized in that... Includes the following steps: Step 1: Deploy 31 known access points (APs) in the two areas of the corridor and the room, denoted as AP1, ..., AP2. m Let the j-th AP be denoted as AP. j (j≤m, j is an integer), with the bottom right corner of the corridor as the origin of the world coordinate system, and the direction of the corridor as the x-axis and y-axis of the world coordinate system, record the true two-dimensional coordinate position of each reference point. Step 2: Using a handheld camera, a certain number of images were captured at each reference point, including from three angles: 45°, 90°, and 135°. These images were used to construct an image database, with 90% used as the training set and 10% as the test set. All training and test images had a resolution of 3024*4032 pixels. Step 3: Construct a SiameseNetwork contrastive learning framework based on convolutional neural networks to learn effective image feature representations and similarity measurement methods. Step 4: Calculate and optimize the fundamental matrix using the improved EightPoint-RANSAC algorithm to further solve the camera's attitude changes and estimate the user's position.

2. The visual image localization method based on SiameseNetwork similarity measurement and improved EightPoint-RANSAC algorithm according to claim 1, characterized in that... Step three includes the following steps: Step 3: Construct a SiameseNetwork contrastive learning framework based on convolutional neural networks to learn effective image feature representations and similarity measurement methods. This includes the following steps: Step 3 (a) Using the data preprocessing module, create positive and negative sample pairs for the image sample library. Specifically, this involves labeling image pairs in the dataset as similar or dissimilar. Positive sample pairs represent high similarity between two images, while negative sample pairs represent low similarity. Specifically, select a pair of images from the dataset and treat them as a single sample pair. Different images from the same reference point are considered positive sample pairs, while different images from different reference points are considered negative sample pairs. Let Y represent the similarity between positive and negative sample pairs. If images X1 and X2 are a positive sample pair, then Y = 1, and the label for this sample pair is set to (X1, X2, 1); otherwise, it is (X1, X2, 0). This positive / negative classification of sample pairs not only simplifies the model and reduces data requirements but also improves the model's performance and generalization ability, ensuring that the model learns useful features and distinguishes between similar and dissimilar images. Step 3 (II): Process image data using a convolutional neural network (CNN) and extract deep features from the input image. A key feature of the Siamese Network is that it contains two identical CNNs that share the same network structure, weights, and biases. A CNN typically includes convolutional layers, pooling layers, and fully connected layers. Image feature extraction is achieved through a forward propagation process. A CNN can be viewed as a non-linear function, i.e.: h i =f(X i ),i=1,2 (1) Where h is the extracted feature vector. Table I Network Parameter Settings Step 3 (III): After feature extraction, similarity measurement training is performed in the contrastive loss module. A contrastive loss function is defined, which can effectively handle the relationship between paired samples in the Siamese neural network. The expression is as follows: Where (h1,h2,Y) i It is the i-th sample pair, where h1 and h2 represent the feature vectors of a pair of images, and Y represents the similarity; D(E) w (h1,h2))=||h1-h2||2 represents the Euclidean distance of the sample features, D G It is a partial loss function for positive sample pairs, D I It is a partial loss function for negative sample pairs, where N is the number of samples. The loss function is trained by measuring the Euclidean distance between feature vectors in the feature space. Specifically, when samples are similar, Y = 1, and the loss function only has [missing information]. In other words, if similar samples have a large Euclidean distance in the feature space, it indicates that the current model is inadequate and the loss function needs to be increased to narrow the sample distance. Conversely, when samples are dissimilar, Y = 0, and the loss function is... In other words, if dissimilar samples have a small Euclidean distance in the feature space, the loss value will be larger. Step 3 (4): Input the test image into the trained model, match it with the images in the database, and output the most matching image.

3. The visual image localization method based on Siamese Network similarity measurement and improved EightPoint-RANSAC algorithm according to claim 1, characterized in that... Step four includes the following steps: Step 4: Calculate and optimize the fundamental matrix using the improved EightPoint-RANSAC algorithm to further solve for camera pose changes and estimate the user's position. The specific steps are as follows: Step 4 (a) Initial Matching Point Query: To prevent the extracted pixels from being too concentrated, a grid segmentation technique is used to divide the image plane into several grids. The weights of each grid are defined based on the number of feature points distributed in it, thus determining the probability of extracting pixels from each grid. This process ensures that the selected matching feature points come from different regions. Next, the Euclidean distance between feature point pairs is calculated. Based on the grid weights, the eight pairs of feature points with the smallest Euclidean distance are selected as the initial values ​​for calculating the fundamental matrix. Step 4 (II): Calculation and optimization of the fundamental matrix. The expression for the fundamental matrix is ​​obtained by using the principle of epipolar geometry: (u′ i,j ) T Fu i,j =0 (2) Where, F=(K -1 ) T t∧RK -1 Given the base matrix E = t∧R as the essential matrix, to calculate the error of all matching points in this model, the method of minimizing the projection error is adopted, using u i,j exist Represented by the square of the projection: Where d represents the transfer error of the offline image, and d′ represents the transfer error of the query image, both following a system of m degrees of freedom. Distribution, where m equals the codimensionality of the model, and the random variable. The value is less than k 2 The probability is determined by the cumulative χ² 2 distributed Given that if the pixel distance between matching points is greater than a set threshold, the pixel coordinates of the matching feature points are considered distorted, and the distorted inliers are removed; if the distance is less than the pixel threshold, the pixel drift is within an acceptable range, and such matching feature points are recorded as a new set of inliers. When the number of inliers reaches a certain threshold, the iteration stops, and the model with the most inliers is retained as the final model. If there are not enough inliers, the fundamental matrix is ​​recalculated. Step 4 (3), Motion and Structure Recovery: Singular value decomposition is performed on the essential matrix to obtain: E~Udiag(1,1,0)V T (5) Here, diag(1,1,0) is a diagonal matrix with elements (1,1,0) on the diagonal. Since the rank of the essential matrix is ​​2, the SVD decomposition has only two non-zero singular values, corresponding to the two principal directions of the essential matrix. When calculating the camera pose, since the last column of the decomposed V matrix corresponds to the right singular matrix vector corresponding to the smallest singular value of the essential matrix, it represents the third principal direction of the essential matrix and can be used to represent the camera's rotation matrix R. The translation matrix t can be calculated through the third column of the essential matrix, as follows: t~t u =[u 13 ,u 23 ,u 33 ] T (6) R a =UDV T orR b =UD T IN T (7) Where det(U)>0, det(V)>0, and D is the coefficient matrix, four possible motions can be obtained: To ensure that image feature points are directly in front of the camera, a unique rotation and translation matrix can be obtained from the above assumptions, and the camera pose can be calculated. Step 4 (iv): Estimate the user's position based on the camera's rotation and translation matrices.

Images