Indoor static object positioning method and VR device

Abstract

The invention provides an indoor static object positioning method and VR equipment, which are used for improving the positioning accuracy. Comprising the steps of obtaining a category set according to each 2D feature point and a feature point category in a current frame indoor image; for any category set, determining the category probability of each 2D feature point based on the prediction category of each 2D feature point in the category set; the category probability of any 2D feature point is the probability that the actual category of the 2D feature point is the prediction category; according to the category probability of each 2D feature point and the difference degree between any two 2D feature points, the confidence degree of the category set is obtained, and the difference degree represents the probability that the actual categories of any two 2D feature points are different feature point categories; determining an actual category of each 2D feature point according to a target category set obtained according to the confidence of each category set; and determining each 2D static feature point by using the actual category of each 2D feature point to obtain the position of the indoor static object.

Description

Background Art

[0002] VR (Virtual Reality) provides users with an immersive gaming experience. The translation and rotation of the user's head in three-dimensional space is an important part of interaction in the VR field. With the introduction of the concept of the metaverse and the improvement of mobile computing capabilities in recent years, the market's technical requirements for users to fully interact in the field of AR / VR technology are gradually increasing. Therefore, key performance parameters such as the 6DoF (Six Degrees Of Freedom) tracking accuracy, positioning technology, and tracking stability of the user's head on a mobile platform are direct technical indicators that directly affect the user's actual experience. If the head tracking and positioning technology is not accurate and robust enough, it will be difficult to cope with the test of special scenes such as low texture, which will inevitably cause certain troubles in the user experience, such as: the virtual scene cannot be positioned, drifting, etc.

[0003] In the task of positioning and mapping, it is crucial to continuously and stably output high-accuracy feature point matching pairs and build accurate and robust maps. Currently, most VR devices are used in indoor scenes. However, in the process of interaction in indoor scenes, there are inevitably some scenes with dynamic objects. For example, people passing in front of VR devices, running pets, interacting hands and handles, etc. If there are dynamic feature points among the extracted feature points, and mapping is performed based on the feature points, it will lead to inaccurate mapping, reducing the accuracy of positioning indoor static objects. Summary of the invention

[0004] The present application provides a method for positioning an indoor stationary object and a VR device, which are used to improve the positioning accuracy of the indoor stationary object.

[0005] In a first aspect, an embodiment of the present application provides a method for positioning a stationary object in an indoor environment, which is applied to a VR device. The method includes:

[0006] According to each 2D feature point in the current frame indoor image and the preset feature point category, a plurality of category sets are obtained, wherein the feature point category includes a static category and a dynamic category, and any one of the category sets includes the predicted category of each 2D feature point, and the predicted category is any one of the feature point categories;

[0007] For any one of the category sets, based on the predicted category of each of the 2D feature points in the category set, determining the category probability of each of the 2D feature points; wherein the category probability of any one of the 2D feature points is the probability that the actual category of the 2D feature point is the same as the predicted category; and,

[0008] Obtaining the confidence of the category set through the category probabilities of the respective 2D feature points and the difference between any two 2D feature points in the category set, wherein the difference between the any two 2D feature points is used to indicate the probability that the actual categories of the any two 2D feature points are different feature point categories, and the difference between the any two 2D feature points is obtained based on the image position coordinates of the respective two 2D feature points;

[0009] According to the confidence of each category set, a target category set is obtained, and the actual category of each 2D feature point is obtained through the target category set;

[0010] The actual categories of the 2D feature points are used to obtain the 2D static feature points corresponding to the static categories, and the positions of the indoor stationary objects are obtained based on the 2D static feature points.

[0011] A second aspect of the present application provides a VR device, including a processor and a memory, wherein the processor and the memory are connected via a bus;

[0012] The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program:

[0013] According to each 2D feature point in the current frame indoor image and the preset feature point category, a plurality of category sets are obtained, wherein the feature point category includes a static category and a dynamic category, and any one of the category sets includes the predicted category of each 2D feature point, and the predicted category is any one of the feature point categories;

[0014] For any one of the category sets, based on the predicted category of each of the 2D feature points in the category set, determining the category probability of each of the 2D feature points; wherein the category probability of any one of the 2D feature points is the probability that the actual category of the 2D feature point is the same as the predicted category; and,

[0015] Obtaining the confidence of the category set through the category probabilities of the respective 2D feature points and the difference between any two 2D feature points in the category set, wherein the difference between the any two 2D feature points is used to indicate the probability that the actual categories of the any two 2D feature points are different feature point categories, and the difference between the any two 2D feature points is obtained based on the image position coordinates of the respective two 2D feature points;

[0016] According to the confidence of each category set, a target category set is obtained, and the actual category of each 2D feature point is obtained through the target category set;

[0017] The actual categories of the 2D feature points are used to obtain the 2D static feature points corresponding to the static categories, and the positions of the indoor stationary objects are obtained based on the 2D static feature points.

[0018] According to a third aspect provided by an embodiment of the present invention, a computer storage medium is provided, wherein the computer storage medium stores a computer program, and the computer program is used to execute the method as described in the first aspect.

[0019] In the above-mentioned embodiment of the present application, multiple category sets are obtained according to each 2D feature point in the current frame indoor image and the preset feature point category, and then the confidence of each category set is obtained through the category probability of each 2D feature point in each category set and the difference between any two 2D feature points in each category set, and then the target category set is obtained through the confidence of the category set, and finally the actual category of each 2D feature point is obtained through the target category set to obtain each 2D static feature point corresponding to the static category, and the position of the indoor stationary object is obtained based on the 2D static feature points. Therefore, in the embodiment of the present application, the dynamic feature points and static feature points in the feature points can be identified, the dynamic points can be eliminated, and then the static objects are located through the static feature points, which improves the accuracy of the static object positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 The schematic diagram of the process of positioning a stationary object indoors provided by an embodiment of the present application is exemplarily shown;

[0022] Figure 2 One of the flow charts of the method for positioning an indoor stationary object provided by an embodiment of the present application is exemplarily shown;

[0023] Figure 3 The flowchart of the method for determining the static probability value of a 2D feature point provided in an embodiment of the present application is exemplarily shown;

[0024] Figure 4 The schematic diagram of the epipolar constraint provided by the embodiment of the present application is exemplarily shown;

[0025] Figure 5 A schematic diagram showing a projection error provided by an embodiment of the present application is exemplified;

[0026] Figure 6 The schematic diagram of the process of calculating the difference between 2D feature points provided in the embodiment of the present application is exemplified;

[0027] Figure 7 The actual category diagram of each feature point provided in the embodiment of the present application is exemplarily shown;

[0028] Figure 8 The second flowchart of the method for positioning an indoor stationary object provided by an embodiment of the present application is exemplarily shown;

[0029] Fig. 9 A schematic diagram of a positioning device for indoor stationary objects provided by an embodiment of the present application is exemplarily shown;

[0030] Fig.10 The hardware structure diagram of the VR device provided in the embodiment of the present application is exemplified. DETAILED DESCRIPTION

[0031] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0032] Based on the exemplary embodiments described in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the claims attached to this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of the disclosure can also constitute a complete implementation method separately.

[0033] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and common meanings.

[0034] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a series of components is not necessarily limited to those components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0035] The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0036] The idea of ​​the embodiments of the present application is summarized below.

[0037] At present, most of the usage scenarios of VR devices are in indoor scenes. However, in the process of interaction in indoor scenes, there are inevitably some scenes with dynamic objects, such as Figure 1 As shown in the figure, there are moving people, interactive gestures, and interactive handles. Therefore, if there are dynamic feature points among the extracted feature points, and mapping is performed based on the feature points, it will lead to inaccurate mapping and low accuracy in positioning indoor static objects.

[0038] Based on the problem of low accuracy in positioning indoor stationary objects in the prior art, an embodiment of the present application provides a method for positioning indoor stationary objects, by obtaining multiple category sets based on each 2D feature point in the current frame indoor image and the preset feature point category, and then obtaining the confidence of each category set through the category probability of each 2D feature point in each category set and the difference between any two 2D feature points in each category set, and then obtaining the target category set through the confidence of the category set, and finally obtaining the actual category of each 2D feature point through the target category set, so as to obtain each 2D static feature point corresponding to the static category, and obtain the position of the indoor stationary object based on the 2D static feature point. Therefore, in the embodiment of the present application, the dynamic feature points and static feature points in the feature points can be identified, and then the static object is positioned through the static feature points, thereby improving the accuracy of positioning the static object. Below, the indoor stationary object positioning method in the present application is described in detail in conjunction with the accompanying drawings.

[0039] like Figure 2 FIG. 1 is a flow chart of a method for positioning a stationary object indoors, which may include the following steps:

[0040] Step 201: obtaining a plurality of category sets according to each 2D feature point in the current frame indoor image and a preset feature point category, wherein the feature point category includes a static category and a dynamic category, and any one of the category sets includes a predicted category of each of the 2D feature points, and the predicted category is any one of the feature point categories;

[0041] In the embodiment of the present application, the prediction category of each 2D feature point is set to a static category and a dynamic category, and then arranged and combined to obtain multiple category sets. Each category set corresponds to a category combination method of each 2D feature point, and multiple category sets in the embodiment of the present application correspond to the combination methods of all feature point categories. For example, the current frame indoor image includes 2D feature point a, 2D feature point b and 2D feature point c. The corresponding category sets are (0, 0, 0), (0, 1, 1,), (1, 1, 1,), (0, 0, 1,), (0, 1, 0,), (1, 1, 0,), (1, 0, 1,), (1, 0, 0,)o, wherein the values ​​from left to right in any category set represent the prediction category of 2D feature point a, the prediction category of 2D feature point b and the prediction category of 2D feature point c respectively. 0 in the category set in the embodiment of the present application represents a dynamic category, and 1 represents a static category. Taking the category set (0, 0, 0) as an example, the category set indicates that the predicted category of the 2D feature point a is the dynamic category, the predicted category of the 2D feature point b is the dynamic category, and the category of the 2D feature point c is the dynamic category.

[0042] However, the embodiment of the present application does not limit the way in which the categories are represented, and the specific way in which the categories are represented can be set according to actual conditions.

[0043] Step 202: for any category set, based on the predicted category of each 2D feature point in the category set, determine the category probability of each 2D feature point; wherein the category probability of any 2D feature point is the probability that the actual category of the 2D feature point is the same as the predicted category;

[0044] In one embodiment, step 202 can be specifically implemented as follows: for any 2D feature point, the category probability of the 2D feature point is obtained according to the probability function corresponding to the predicted category of the 2D feature point and the static probability value of the 2D feature point, wherein the static probability value of the 2D feature point is used to represent the probability that the actual category of the 2D feature point is a static category.

[0045] Next, we first explain how to determine the static probability value of any 2D feature point. Figure 3 As shown, it is a schematic diagram of the process of determining the static probability value, which includes the following steps:

[0046] Step 301: for any 2D feature point, obtain a first intermediate probability of the 2D feature point according to the image position coordinates of the 2D feature point in the current frame indoor image and the image position coordinates of the 2D feature point in the previous frame indoor image;

[0047] In the embodiment of the present application, the image position coordinates of the 2D feature point in the previous frame of the indoor image are determined by feature point matching. The feature point matching method can be set according to actual conditions. The embodiment of the present application does not limit the feature point matching method.

[0048] In one embodiment, step 301 may be specifically implemented as follows: based on the image position coordinates of the 2D feature points in the current frame indoor image and the image position coordinates of the 2D feature points in the previous frame indoor image, obtaining two sets of epipolar line distances of the 2D feature points, and obtaining a first intermediate probability of the 2D feature points according to the two sets of epipolar line distances of the 2D feature points;

[0049] Among them, any set of epipolar line distances includes a first epipolar line distance and a second epipolar line distance, the first epipolar line distance is the distance between the 2D feature point in the current frame indoor image and the epipolar line in the previous frame indoor image, and the epipolar line in the previous frame indoor image is the intersection line between the previous frame indoor image and the epipolar plane, and the epipolar plane is a plane composed of the 3D feature point corresponding to the 2D feature point in the current frame indoor image, the camera center point of the current frame indoor image and the camera center point of the previous frame indoor image; the second epipolar line distance is the distance between the 2D feature point in the previous frame indoor image and the epipolar line in the current frame indoor image, and the epipolar line in the current frame indoor image is the intersection line between the current frame indoor image and the epipolar plane.

[0050] like Figure 4 As shown, it is a schematic diagram of the polar line constraint. It can be seen from the figure that the camera center corresponding to the previous frame indoor image I1 is O1. The camera center corresponding to the current frame indoor image I2 is O2. For the 3D feature point P in the space, it corresponds to the 2D feature point p2 in the current frame indoor image I2, and corresponds to the 2D feature point p1 in the previous frame indoor image I1. At this time, the line O1p1 and the line O2p2 will intersect at the 3D feature point P in the space in three-dimensional space. Therefore, in the embodiment of the present application, it is considered that the plane composed of the three points O1, O2, and P is the polar plane, and the intersection line l1 of this polar plane and the previous frame indoor image I1 is the polar line in the previous frame indoor image. The intersection line l2 of the polar plane and the current frame indoor image I2 is the polar line in the current frame indoor image. Below, the method of determining the distance between two sets of polar lines corresponding to any 2D feature point is introduced:

[0051] 1. The first set of epipolar distances:

[0052] First epipolar distance: For any 2D feature point, the first epipolar distance is obtained according to the matrix of the image position coordinates of the 2D feature point in the current frame indoor image, the matrix of the image position coordinates of the 2D feature point in the previous frame indoor image, the first basic matrix and the slope of the epipolar line in the previous frame indoor image. The first epipolar distance can be obtained by formula (1):

[0053]

[0054] Among them, d1 is the first polar line distance, p2 is the matrix of the image position coordinates of the 2D feature points in the current frame indoor image, p1 is the matrix of the image position coordinates of the 2D feature points in the previous frame indoor image, F1 is the first basic matrix, and k1 is the slope of the polar line in the previous frame indoor image.

[0055] Second epipolar distance: The second epipolar distance is obtained according to the matrix of the image position coordinates of the 2D feature points in the current frame indoor image, the matrix of the image position coordinates of the 2D feature points in the previous frame indoor image, the first basic matrix and the slope of the epipolar line in the current frame indoor image; wherein the second epipolar distance can be obtained by formula (2):

[0056]

[0057] Wherein, d2 is the second epipolar line distance, and k2 is the slope of the epipolar line in the current frame indoor image.

[0058] It should be noted that: the first basic matrix in the embodiment of the present application is a 3*3 matrix, and the matrix of the image position coordinates is a 1*3 matrix, the form of the matrix is ​​[x, y, 1], wherein x is the horizontal coordinate of the image position, and y is the vertical coordinate of the image position. The method for determining the slope of the polar line in the previous frame of the indoor image in the first polar line distance in the embodiment of the present application is: multiply the first basic matrix with the matrix corresponding to the image position coordinates of any 2D feature point in the previous frame of the indoor image to obtain a first intermediate matrix, and based on the parameters in the first row and first column of the first intermediate matrix and the parameters in the first row and second column, obtain the slope of the polar line in the previous frame of the indoor image. Among them, the first intermediate matrix can be obtained by formula (3):

[0059] H1=F1p1......(3);

[0060] Among them, H1 is the first intermediate matrix, F is the first basic matrix, and p1 is the matrix corresponding to the image position coordinates of any 2D feature point in the previous frame of indoor image.

[0061] The first intermediate matrix obtained in the embodiment of the present application is a 1*3 matrix. For example, if the first intermediate matrix is ​​[a, b, c], then the slope of the polar line in the previous frame of the indoor image is

[0062] Furthermore, in the embodiment of the present application, the slope of the polar line in the current frame indoor image in the second polar line distance is determined by multiplying the transposed matrix of the matrix corresponding to the image position coordinates of any 2D feature point in the current frame indoor image by the first basic matrix to obtain a second intermediate matrix, and based on the parameters in the first row and first column and the parameters in the first row and second column of the second intermediate matrix, the slope of the polar line in the current frame indoor image is obtained. The second intermediate matrix can be obtained by formula (4):

[0063]

[0064] Among them, H2 is the second intermediate matrix, and P2 is the matrix corresponding to the image position coordinates of any 2D feature point in the current frame indoor image.

[0065] The second intermediate matrix obtained in the embodiment of the present application is also a 1*3 matrix, which is the same as the first epipolar distance. For example, the second intermediate matrix is ​​[d, e, f], and the slope of the epipolar line in the current frame indoor image is

[0066] The first basic matrix in the embodiment of the present application is determined by randomly selecting a random 2D feature point from each 2D feature point, and obtaining the first basic matrix according to the matrix corresponding to the image position coordinates of the random 2D feature point in the current frame indoor image and the matrix corresponding to the image position coordinates of the random 2D feature point in the previous frame indoor image. The first basic matrix can be obtained by formula (5):

[0067]

[0068] Among them, p n is the matrix corresponding to the image position coordinates of the random 2D feature point in the previous frame of the indoor image, p m is the matrix corresponding to the image position coordinates of the random 2D feature point in the current frame indoor image, and F1 is the first basic matrix.

[0069] 2. The second set of epipolar distances:

[0070] First epipolar distance: For any 2D feature point, the first epipolar distance is obtained according to the matrix of the image position coordinates of the 2D feature point in the current frame indoor image, the matrix of the image position coordinates of the 2D feature point in the previous frame indoor image, the second basic matrix and the slope of the epipolar line in the previous frame indoor image. The first epipolar distance can be obtained by formula (6):

[0071]

[0072] Wherein, d1 is the first epipolar distance, and F2 is the second fundamental matrix.

[0073] Second epipolar line distance: The second epipolar line distance is obtained according to the matrix of the image position coordinates of the 2D feature points in the current frame indoor image, the matrix of the image position coordinates of the 2D feature points in the previous frame indoor image, the second basic matrix and the slope of the epipolar line in the current frame indoor image; wherein the second epipolar line distance can be obtained by formula (7):

[0074]

[0075] Wherein, d2 is the second epipolar distance.

[0076] It should be noted that: the second basic matrix in the embodiment of the present application is a 3*3 matrix, and the matrix corresponding to the image position coordinates is a 1*3 matrix, and the form of the matrix is ​​[x, y, 1], where x is the horizontal coordinate of the image position and y is the vertical coordinate of the image position. The slope of the polar line in the previous frame of the indoor image in the first polar line distance in the embodiment of the present application is obtained by multiplying the second basic matrix with the matrix corresponding to the image position coordinates of any 2D feature point in the previous frame of the indoor image to obtain a third intermediate matrix, and based on the parameters in the first row and first column of the third intermediate matrix and the parameters in the first row and second column, the slope of the polar line in the previous frame of the indoor image is obtained. Among them, the third intermediate matrix can be obtained by formula (8):

[0077] H3=F2p1……(8);

[0078] Among them, H3 is the third intermediate matrix, F2 is the second basic matrix, and p1 is the matrix corresponding to the image position coordinates of any 2D feature point in the previous frame of indoor image.

[0079] The third intermediate matrix obtained in the embodiment of the present application is a 1*3 matrix. For example, the third intermediate matrix is ​​[h, i, j], then the slope of the polar line in the previous frame of the indoor image is

[0080] The slope of the second polar line in the embodiment of the present application is determined by multiplying the transposed matrix of the matrix corresponding to the image position coordinates of any 2D feature point in the current frame indoor image by the second basic matrix to obtain a fourth intermediate matrix, and based on the parameters in the first row and first column of the fourth intermediate matrix and the parameters in the first row and second column, the slope of the polar line in the current frame indoor image is obtained. The fourth intermediate matrix can be obtained by formula (9):

[0081]

[0082] Wherein, H4 is the fourth intermediate matrix. For example, the fourth intermediate matrix is ​​[r, i, j], then the slope of the polar line in the current frame indoor image is

[0083] In the embodiment of the present application, the second basic matrix is ​​determined as follows: using the Imu sensor to obtain the position and posture of each 2D feature point in the current frame indoor image and the position and posture of each 2D feature point in the previous frame indoor image; for any 2D feature point, according to the position and posture of the 2D feature point in the current frame indoor image and the position and posture of the 2D feature point in the previous frame image, the rotation and translation matrix of the 2D feature point is obtained, and the rotation and translation matrix of the 2D feature point and the preset camera intrinsic parameter matrix are used to obtain the second basic matrix, wherein the second basic matrix can be obtained by formula (6):

[0084] F2=K -T EK -1 ......(10);

[0085] Among them, F2 is the second basic matrix, K is the camera intrinsic parameter matrix, and E is the rotation and translation matrix.

[0086] It should be noted that determining the rotation and translation matrix between two 2D feature points based on the positions of the two 2D feature points belongs to the method in the prior art and will not be described in detail in the embodiments of the present application.

[0087] In the embodiment of the present application, after obtaining two sets of epipolar distances for any 2D feature point, for any set of epipolar distances in the 2D feature point, the first epipolar distance and the second epipolar distance in the set are added to obtain an intermediate distance, and the intermediate distances of the two sets of epipolar distances are weighted summed to obtain a target epipolar distance of the 2D feature point. According to the target epipolar distance of the 2D feature point, a first intermediate probability of the 2D feature point is obtained, wherein the first intermediate probability of the 2D feature point can be obtained by formula (11):

[0088]

[0089] in, is the first intermediate probability of the mth 2D feature point, D m is the target epipolar distance of the mth 2D feature point, μ D is the average value of the target epipolar distance of each 2D feature point in the current frame indoor image, σ D is the standard deviation of the target epipolar distance of each 2D feature point in the current frame indoor image.

[0090] It should be noted that the weights of the first epipolar distance and the second epipolar distance in any group of epipolar distances in the embodiment of the present application are both 0.5. However, the weights of the first epipolar distance and the second epipolar distance in each group of epipolar distances are not limited in the embodiment of the present application, and the weights of the first epipolar distance and the second epipolar distance in each group of epipolar distances can be set according to actual conditions.

[0091] Step 302: Based on the observed value of the 2D feature point, a second intermediate probability of the 2D feature point is obtained, wherein the observed value is the total number of target indoor images corresponding to the 2D feature point, and the target indoor image is an indoor image including the 2D feature point among the 2D feature points of the image; wherein the second intermediate probability can be obtained by formula (12):

[0092]

[0093] in is the second intermediate probability of the mth 2D feature point, N m is the observed value of the mth 2D feature point, μ N is the average value of the observed values ​​of each 2D feature point in the current frame indoor image, σ N is the standard deviation of the observed values ​​of each 2D feature point in the current frame indoor image.

[0094] For example, if the indoor images are: indoor image 1, indoor image 2, indoor image 3 and indoor image 4. If indoor image 3 and indoor image 4 include a 2D feature point i, then the observation value of the 2D feature point i is determined to be 2.

[0095] Step 303: Obtain a third intermediate probability of the 2D feature point through a projection error of the 2D feature point, wherein the projection error is obtained based on the projection position coordinates of the first target 3D feature point in the current frame indoor image and the image position coordinates of the 2D feature point, and the first target 3D feature point is a 3D feature point corresponding to the 2D feature point in the previous frame indoor image; wherein the third intermediate probability can be obtained through formula (13):

[0096]

[0097] in is the third intermediate probability of the mth 2D feature point, R m is the projection error of the mth 2D feature point, μ R is the average value of the projection error of each 2D feature point in the current frame indoor image, σ R is the standard deviation of the projection error of each 2D feature point in the current frame indoor image.

[0098] The projection error in the embodiment of the present application is determined by: based on the projection position coordinates of the first target 3D feature point in the current frame indoor image and the image position coordinates of the 2D feature point, a first distance is determined, and the first distance is determined as the projection error. Among them, determining the distance between two points by the positions of the two points belongs to the method in the prior art, and the embodiment of the present application will not be repeated.

[0099] Since dynamic 3D feature points are in motion during the process, the 3D position coordinates of the 3D feature points are constantly changing. This means that the distance (projection error) between the image position coordinates of the 3D feature points obtained in the previous frame of the indoor image reprojected to the current frame of the indoor image and the image position coordinates of the corresponding 2D feature points obtained in the current frame of the indoor image has a larger projection error than that of static feature points. Figure 5 As shown in the figure, it is a schematic diagram of projection error. Figure 5 It can be seen that the projection error of dynamic feature points is greater than that of static feature points.

[0100] It should be noted that the execution sequence of step 301, step 302 and step 303 in the embodiment of the present application is not limited in the embodiment of the present application. They can be executed successively or simultaneously, and can be set according to the specific actual situation.

[0101] Step 304: performing weighted summation on the first intermediate probability, the second intermediate probability and the third intermediate probability to obtain a static probability value of the 2D feature point.

[0102] Among them, the weights of the first intermediate probability, the second intermediate probability, and the third intermediate probability in the embodiment of the present application are all 1 / 3, but the embodiment of the present application does not limit the weights of the first intermediate probability, the second intermediate probability, and the third intermediate probability. The weights of the first intermediate probability, the second intermediate probability, and the third intermediate probability can be set according to actual conditions.

[0103] After determining the static probability value of the 2D feature point, the category probability of any 2D feature point can be obtained by formula (14):

[0104]

[0105] in, is the category probability of the mth 2D feature point, x m is the predicted category of the mth 2D feature point. When the predicted category of the 2D feature point is a dynamic category, then x m = 0, when the predicted category of the 2D feature point is a static category, then x m =1, c is the preset constant value, S m is the static probability value of the mth 2D feature point, t is the specified value, when S m > t, I(S m >t)=A, when S m ≤t, I(S m >t)=B. And A and B are constants, A in the embodiment of the present application is 1, B is 0, and when (S m >t)=0, then. The value of is a fixed value, which is a constant.

[0106] It should be noted that in the embodiment of the present application, A, B and the fixed value can be set according to actual conditions, and the embodiment of the present application does not limit the specific values ​​of A, B and the fixed value.

[0107] Step 203: obtaining the confidence of the category set through the category probabilities of the respective 2D feature points and the difference between any two 2D feature points in the category set, wherein the difference between any two 2D feature points is used to represent the probability that the actual categories of the any two 2D feature points are different feature point categories, and the difference between any two 2D feature points is obtained based on the image position coordinates of the any two 2D feature points;

[0108] In one embodiment, step 203 can be specifically implemented as follows: adding the category probabilities of each 2D feature point in the category set to obtain a first intermediate confidence; and adding the target differences in the category set to obtain a second intermediate confidence; adding the first intermediate confidence and the second intermediate confidence to obtain the confidence of the category set, wherein the target difference is the difference between any two 2D feature points in the category set.

[0109] like Figure 6 FIG. 1 is a flow chart of determining the difference between any two 2D feature points in an embodiment of the present application, which may include the following steps:

[0110] Step 601: Based on the target 2D feature point among the arbitrary two 2D feature points, a first difference parameter is obtained according to the observed value and projection error of the target 2D feature point, wherein the target 2D feature point is the feature point with a smaller sequence number among the arbitrary two 2D feature points; wherein the first difference parameter can be obtained by formula (15):

[0111]

[0112] Wherein, k1 is the first difference parameter, N i is the observed value of the target 2D feature point, R i is the projection error of the target 2D feature point, μ N is the average value of the observed values ​​of each 2D feature point in the current frame indoor image, μ R is the average projection error of each 2D feature point in the current frame indoor image, σ N is the standard deviation of the observed values ​​of each 2D feature point in the current frame indoor image, σ R is the standard deviation of the projection error of each 2D feature point in the current frame indoor image.

[0113] It should be noted that the serial number of each 2D feature point in each frame of the indoor image in the embodiment of the present application is obtained when the feature point is extracted. The method of extracting the feature point can be set according to the specific actual situation, and the embodiment of the present application is no longer limited here.

[0114] Step 602: Obtain a second difference parameter according to the image position coordinates of the arbitrary two 2D feature points and the 3D position coordinates corresponding to the arbitrary two 2D feature points; wherein the difference between the arbitrary two feature points can be obtained by formula (16):

[0115]

[0116] Wherein, k2 is the difference, P i is the matrix of the 3D position coordinates corresponding to the target 2D feature points among the two arbitrary 2D feature points, P j is the matrix of the 3D position coordinates corresponding to the other 2D feature point of any two 2D feature points, p i is the matrix of the image position coordinates of the target 2D feature points among the two arbitrary 2D feature points, p j is the matrix of the image position coordinates of the other 2D feature point among any two 2D feature points, σ P is the standard deviation matrix of the 3D position coordinates of any two 2D feature points, σ p is the standard deviation matrix of the image position coordinates of any two 2D feature points.

[0117] Step 603: performing weighted summation on the first difference parameter and the second difference parameter to obtain the difference between the arbitrary two feature points.

[0118] It should be noted that the weights of the first difference parameter and the second difference parameter in the embodiment of the present application can be set according to the actual situation, and the embodiment of the present application does not limit the weight of the first difference parameter and the weight of the second difference parameter. And the mean matrix and the standard deviation matrix described in the embodiment of the present application are all methods in the prior art, so the embodiment of the present application will not be repeated here.

[0119] Step 204: obtaining a target category set according to the confidence of each category set, and obtaining the actual category of each 2D feature point through the target category set;

[0120] In one embodiment, step 204 may be specifically implemented as follows: determining the category set with the largest confidence as the target category set, and for any 2D feature point, determining the predicted category of the 2D feature point in the target category set as the actual category of the 2D feature point.

[0121] like Figure 7 As shown in FIG. 1 , it is a schematic diagram of the actual category of each feature point. Static category feature points and dynamic category feature points can be represented by different colors to distinguish the actual category of each feature point. Figure 7 As can be seen from the figure, the category of white 2D feature points is static, and the category of gray 2D feature points is dynamic.

[0122] Step 205: using the actual categories of the 2D feature points, obtaining the 2D static feature points corresponding to the static categories, and obtaining the positions of indoor stationary objects based on the 2D static feature points.

[0123] In one embodiment, step 205 can be specifically implemented as follows: deleting the feature points of the current frame indoor image whose feature point category is dynamic category, obtaining the target current frame indoor, and using the slam algorithm to locate the stationary objects in the room through the 2D static feature points in the target current frame indoor image, to obtain the position of the stationary objects in the room.

[0124] It should be noted that the method of using the SLAM algorithm for mapping and positioning is a common method in the prior art and will not be described in detail in the embodiments of this application.

[0125] In order to reduce the amount of calculation, in one embodiment, before executing step 201, for any 2D feature point, if the vertical coordinate value of the 3D feature point corresponding to the 2D feature point in the current frame indoor image is not within a specified range, and the first intermediate probability of the 2D feature point is not less than a specified probability value, the 2D feature point is deleted.

[0126] The specified range in the embodiment of the present application is [20, 60], but the embodiment of the present application does not limit the specified range, and the specified range in the embodiment of the present application can be set according to the actual situation. And the specified probability value in the embodiment of the present application can also be set according to the actual situation, and the implementation of the present application does not limit it here.

[0127] In order to further connect the method in this application, the indoor stationary object positioning method in this application is introduced in combination with the usage scenario. Figure 8 As shown, it is a schematic diagram of the flow of the use scenario of indoor stationary object positioning in the present application, which may include the following steps:

[0128] Step 801: traverse each 2D feature point in the current frame indoor image, and for any traversed 2D feature point, determine whether the vertical coordinate value of the 3D feature point corresponding to the 2D feature point in the current frame indoor image is not within a specified range, and whether the first intermediate probability of the 2D feature point is not less than the specified probability value. If so, execute step 802, if not, execute step 803;

[0129] Step 802: Delete the 2D feature points;

[0130] Step 803: determine whether there are any 2D feature points that have not been traversed, if so, return to step 801, if not, execute step 804;

[0131] Step 804: obtaining a plurality of category sets according to each 2D feature point in the current frame indoor image and a preset feature point category, wherein the feature point category includes a static category and a dynamic category, and any one of the category sets includes a predicted category of each of the 2D feature points, and the predicted category is any one of the feature point categories;

[0132] Step 805: for any 2D feature point, according to the image position coordinates of the 2D feature point in the current frame indoor image and the image position coordinates of the 2D feature point in the previous frame indoor image, obtain a first intermediate probability of the 2D feature point;

[0133] Step 806: obtaining a second intermediate probability of the 2D feature point based on the observed value of the 2D feature point, wherein the observed value is the total number of target indoor images corresponding to the 2D feature point, and the target indoor image is an indoor image including the 2D feature point among the 2D feature points of the image;

[0134] Step 807: obtaining a third intermediate probability of the 2D feature point through a projection error of the 2D feature point, wherein the projection error is obtained based on the projection position coordinates of the first target 3D feature point in the current frame indoor image and the image position coordinates of the 2D feature point, and the first target 3D feature point is a 3D feature point corresponding to the 2D feature point in the previous frame indoor image;

[0135] Step 808: performing weighted summation on the first intermediate probability, the second intermediate probability and the third intermediate probability to obtain a static probability value of the 2D feature point;

[0136] Step 809: Obtaining the category probability of the 2D feature point according to the probability function corresponding to the predicted category of the 2D feature point and the static probability value of the 2D feature point;

[0137] Step 810: based on a target 2D feature point among any two 2D feature points in the current frame indoor image, obtaining a first difference parameter according to an observation value and a projection error of the target 2D feature point;

[0138] Wherein, the target 2D feature point is the feature point with a smaller sequence number among the any two 2D feature points, and the observation value is the total number of target indoor images corresponding to the target 2D feature point, and the target indoor image is an indoor image including the target 2D feature point among the 2D feature points of the image, and the projection error is obtained based on the projection position coordinates of the second target 3D feature point in the current frame indoor image and the image position coordinates of the target 2D feature point, and the second target 3D feature point is a 3D feature point corresponding to the target 2D feature point in the previous frame indoor image;

[0139] Step 811: Obtain a second difference parameter according to the image position coordinates of the arbitrary two 2D feature points and the 3D position coordinates corresponding to the arbitrary two 2D feature points;

[0140] Step 812: performing weighted summation on the first difference parameter and the second difference parameter to obtain the difference between the arbitrary two feature points;

[0141] Step 813: Obtain the confidence of the category set through the category probability of each of the 2D feature points and the difference between any two 2D feature points in the category set;

[0142] Step 814: obtaining a target category set according to the confidence of each category set, and obtaining the actual category of each 2D feature point through the target category set;

[0143] Step 815: using the actual categories of the 2D feature points, obtain the 2D static feature points corresponding to the static category, and obtain the position of the indoor stationary object based on the 2D static feature points.

[0144] Based on the same inventive concept, the indoor stationary object positioning method disclosed above can also be implemented by a indoor stationary object positioning device. The effect of the indoor stationary object positioning device is similar to that of the aforementioned method, and will not be described in detail here.

[0145] Fig. 9 The figure is a schematic diagram of the structure of a positioning device for indoor stationary objects according to an embodiment of the present disclosure.

[0146] like Fig. 9 As shown, the indoor stationary object positioning device 900 of the present disclosure may include a category set determination module 910 , a category probability determination module 920 , a category set confidence determination module 930 , a feature point category determination module 940 and a positioning module 950 .

[0147] A category set determination module 910 is used to obtain multiple category sets according to each 2D feature point in the current frame indoor image and a preset feature point category, wherein the feature point category includes a static category and a dynamic category, and any one of the category sets includes a predicted category of each of the 2D feature points, and the predicted category is any one of the feature point categories;

[0148] A category probability determination module 920 is used to determine, for any category set, the category probability of each 2D feature point in the category set based on the predicted category of each 2D feature point; wherein the category probability of any 2D feature point is the probability that the actual category of the 2D feature point is the same as the predicted category; and

[0149] A category set confidence determination module 930, configured to obtain the confidence of the category set through the category probabilities of the respective 2D feature points and the difference between any two 2D feature points in the category set, wherein the difference between the any two 2D feature points is used to indicate the probability that the actual categories of the any two 2D feature points are different feature point categories, and the difference between the any two 2D feature points is obtained based on the image position coordinates of the respective two 2D feature points;

[0150] The feature point category determination module 940 is used to obtain a target category set according to the confidence of each category set, and obtain the actual category of each 2D feature point through the target category set;

[0151] The positioning module 950 is used to obtain each 2D static feature point corresponding to the static category by using the actual category of each 2D feature point, and obtain the position of the indoor stationary object based on the each 2D static feature point.

[0152] In one embodiment, the category set confidence determination module 930 is specifically configured to:

[0153] Adding the category probabilities of the 2D feature points in the category set to obtain a first intermediate confidence; and

[0154] Adding the target differences in the category set to obtain a second intermediate confidence, wherein the target difference is the difference between any two 2D feature points in the category set;

[0155] The first intermediate confidence score and the second intermediate confidence score are added to obtain the confidence score of the category set.

[0156] In one embodiment, the category probability determination module 920 is specifically configured to:

[0157] For any 2D feature point, the category probability of the 2D feature point is obtained according to the probability function corresponding to the predicted category of the 2D feature point and the static probability value of the 2D feature point, wherein the static probability value of the 2D feature point is used to represent the probability that the actual category of the 2D feature point is a static category.

[0158] In one embodiment, the category probability determination module 920 is further configured to:

[0159] The static probability value of the 2D feature point is determined by:

[0160] For any 2D feature point, according to the image position coordinates of the 2D feature point in the current frame indoor image and the image position coordinates of the 2D feature point in the previous frame indoor image, a first intermediate probability of the 2D feature point is obtained; and,

[0161] Based on the observed value of the 2D feature point, a second intermediate probability of the 2D feature point is obtained, wherein the observed value is the total number of target indoor images corresponding to the 2D feature point, and the target indoor image is an indoor image including the 2D feature point among the 2D feature points of the image; and

[0162] Obtaining a third intermediate probability of the 2D feature point through a projection error of the 2D feature point, wherein the projection error is obtained based on a projection position coordinate of a first target 3D feature point in the current frame indoor image and an image position coordinate of the 2D feature point, and the first target 3D feature point is a 3D feature point corresponding to the 2D feature point in the previous frame indoor image;

[0163] The first intermediate probability, the second intermediate probability and the third intermediate probability are weightedly summed to obtain a static probability value of the 2D feature point.

[0164] In one embodiment, the category probability determination module 920 is further configured to:

[0165] Based on the image position coordinates of the 2D feature points in the current frame indoor image and the image position coordinates of the 2D feature points in the previous frame indoor image, two sets of epipolar line distances of the 2D feature points are obtained, and according to the two sets of epipolar line distances of the 2D feature points, a first intermediate probability of the 2D feature points is obtained;

[0166] Among them, any set of epipolar line distances includes a first epipolar line distance and a second epipolar line distance, the first epipolar line distance is the distance between the 2D feature point in the current frame indoor image and the epipolar line in the previous frame indoor image, and the epipolar line in the previous frame indoor image is the intersection line between the previous frame indoor image and the epipolar plane, and the epipolar plane is a plane composed of the 3D feature point corresponding to the 2D feature point in the current frame indoor image, the camera center point of the current frame indoor image, and the camera center point of the previous frame indoor image; the second epipolar line distance is the distance between the 2D feature point in the previous frame indoor image and the epipolar line in the current frame indoor image, and the epipolar line in the current frame indoor image is the intersection line between the current frame indoor image and the epipolar plane;

[0167] The second intermediate probability is obtained by the following formula:

[0168]

[0169] in is the second intermediate probability of the mth 2D feature point, N m is the observed value of the mth 2D feature point, μ N is the average value of the observed values ​​of each 2D feature point in the current frame indoor image, σ N is the standard deviation of the observed values ​​of each 2D feature point in the current frame indoor image;

[0170] The third intermediate probability is obtained by:

[0171]

[0172] in is the third intermediate probability of the mth 2D feature point, R m is the projection error of the mth 2D feature point, μ R is the average value of the projection error of each 2D feature point in the current frame indoor image, σ R is the standard deviation of the projection error of each 2D feature point in the current frame indoor image.

[0173] In one embodiment, the apparatus further comprises:

[0174] The feature point screening module 960 is used to delete any 2D feature point before obtaining multiple category sets based on the 2D feature points in the current frame indoor image and the preset feature point categories, if the vertical coordinate value of the 3D feature point corresponding to the 2D feature point in the current frame indoor image is not within a specified range and the first intermediate probability of the 2D feature point is not less than a specified probability value.

[0175] In one embodiment, the apparatus further comprises:

[0176] The difference determination module 970 is used to obtain the difference between any two 2D feature points by:

[0177] Based on the target 2D feature point among the arbitrary two 2D feature points, a first difference parameter is obtained according to the observation value and the projection error of the target 2D feature point, wherein the target 2D feature point is the feature point with a smaller sequence number among the arbitrary two 2D feature points, and the observation value is the total number of target indoor images corresponding to the target 2D feature point, and the target indoor image is an indoor image including the target 2D feature point among the 2D feature points of the image, the projection error is obtained based on the projection position coordinates of the second target 3D feature point in the current frame indoor image and the image position coordinates of the target 2D feature point, and the second target 3D feature point is the 3D feature point corresponding to the target 2D feature point in the previous frame indoor image;

[0178] Obtaining a second difference parameter according to the image position coordinates of the arbitrary two 2D feature points and the 3D position coordinates corresponding to the arbitrary two 2D feature points;

[0179] The first difference parameter and the second difference parameter are weightedly summed to obtain the difference between the arbitrary two feature points.

[0180] In one embodiment, the difference determination module 970 is further used to:

[0181] The first difference parameter is obtained by the following formula:

[0182]

[0183] Wherein, k1 is the first difference parameter, N i is the observed value of the target 2D feature point, R i is the projection error of the target 2D feature point, μ N is the mean of the observed values ​​of each 2D feature point in the current frame indoor image, μ R is the average projection error of each 2D feature point in the current frame indoor image, σ N is the standard deviation of the observed values ​​of each 2D feature point in the current frame indoor image, σ R is the standard deviation of the projection error of each 2D feature point in the current frame indoor image;

[0184] The difference between any two feature points is obtained by the following formula:

[0185]

[0186] Wherein, k2 is the difference, P i is the matrix of the 3D position coordinates corresponding to the target 2D feature points among the two arbitrary 2D feature points, P jis the matrix of the 3D position coordinates corresponding to the other 2D feature point of any two 2D feature points, p i is the matrix of the image position coordinates of the target 2D feature points among the two arbitrary 2D feature points, p j is the matrix of the image position coordinates of the other 2D feature point among any two 2D feature points, σ P is the standard deviation matrix of the 3D position coordinates of any two 2D feature points, σ p is the standard deviation matrix of the image position coordinates of any two 2D feature points.

[0187] In one embodiment, the feature point category determination module 940 is specifically used to:

[0188] Determine the category set with the largest confidence as the target category set;

[0189] For any 2D feature point, the predicted category of the 2D feature point in the target category set is determined as the actual category of the 2D feature point.

[0190] After introducing a method and apparatus for positioning a stationary object in an indoor environment according to an exemplary embodiment of the present invention, a VR device according to another exemplary embodiment of the present invention is introduced next.

[0191] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0192] In some possible implementations, the VR device according to the present invention may include at least one processor and at least one computer storage medium. The computer storage medium stores program code, and when the program code is executed by the processor, the processor executes the steps of the method for locating a stationary object in an indoor environment according to various exemplary implementations of the present invention described above in this specification. For example, the processor may execute the following steps: Figure 1 Steps 201-205 shown in .

[0193] Refer to the following Fig.10 The VR device 1000 according to this embodiment of the present invention is described. Fig.10 The VR device 1000 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0194] like Fig.10As shown, the VR device 1000 is presented in the form of a general VR device. The components of the VR device 1000 may include, but are not limited to: the at least one processor 1001, the at least one computer storage medium 1002, and a bus 1003 connecting different system components (including the computer storage medium 1002 and the processor 1001).

[0195] Bus 1003 represents one or more of several types of bus structures, including a computer storage media bus or computer storage media controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0196] The computer storage medium 1002 may include readable media in the form of volatile computer storage media, such as random access computer storage media (RAM) 1021 and / or cache storage media 1022 , and may further include read-only computer storage media (ROM) 1023 .

[0197] The computer storage medium 1002 may also include a program / utility 1025 having a set (at least one) of program modules 424, such program modules 1024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0198] The VR device 1000 may also communicate with one or more external devices 1004 (e.g., keyboards, pointing devices, etc.), one or more devices that enable a user to interact with the VR device 1000, and / or any device that enables the VR device 1000 to communicate with one or more other VR devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1005. Furthermore, the VR device 1000 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 1006. As shown, the network adapter 1006 communicates with other modules for the VR device 1000 via a bus 1003. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the VR device 1000, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0199] In some possible implementations, various aspects of a method for positioning an indoor stationary object provided by the present invention may also be implemented in the form of a program product, which includes a program code. When the program product runs on a computer device, the program code is used to enable the computer device to execute the steps of the method for positioning an indoor stationary object according to various exemplary embodiments of the present invention described above in this specification.

[0200] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for positioning a stationary object indoors, characterized in that: Applied to VR equipment, the method includes: According to each 2D feature point in the current frame indoor image and the preset feature point category, a plurality of category sets are obtained, wherein the feature point category includes a static category and a dynamic category, and any one of the category sets includes the predicted category of each 2D feature point, and the predicted category is any one of the feature point categories; For any one of the category sets, based on the predicted category of each of the 2D feature points in the category set, determining the category probability of each of the 2D feature points; wherein the category probability of any one of the 2D feature points is the probability that the actual category of the 2D feature point is the same as the predicted category; and, Obtaining the confidence of the category set through the category probabilities of the respective 2D feature points and the difference between any two 2D feature points in the category set, wherein the difference between the any two 2D feature points is used to indicate the probability that the actual categories of the any two 2D feature points are different feature point categories, and the difference between the any two 2D feature points is obtained based on the image position coordinates of the respective two 2D feature points; According to the confidence of each category set, a target category set is obtained, and the actual category of each 2D feature point is obtained through the target category set; The actual categories of the 2D feature points are used to obtain the 2D static feature points corresponding to the static categories, and the positions of the indoor stationary objects are obtained based on the 2D static feature points.

2. The method according to claim 1, characterized in that The obtaining the confidence of the category set by the category probability of each of the 2D feature points and the difference between any two 2D feature points in the category set includes: Adding the category probabilities of the 2D feature points in the category set to obtain a first intermediate confidence; and Adding the target differences in the category set to obtain a second intermediate confidence, wherein the target difference is the difference between any two 2D feature points in the category set; The first intermediate confidence score and the second intermediate confidence score are added to obtain the confidence score of the category set.

3. The method according to claim 1 or 2, characterized in that: The determining, based on the predicted category of each 2D feature point in the category set, the category probability of each 2D feature point includes: For any 2D feature point, the category probability of the 2D feature point is obtained according to the probability function corresponding to the predicted category of the 2D feature point and the static probability value of the 2D feature point, wherein the static probability value of the 2D feature point is used to represent the probability that the actual category of the 2D feature point is a static category.

4. The method according to claim 3, characterized in that The static probability value of the 2D feature point is determined by: For any 2D feature point, according to the image position coordinates of the 2D feature point in the current frame indoor image and the image position coordinates of the 2D feature point in the previous frame indoor image, a first intermediate probability of the 2D feature point is obtained; and, Based on the observed value of the 2D feature point, a second intermediate probability of the 2D feature point is obtained, wherein the observed value is the total number of target indoor images corresponding to the 2D feature point, and the target indoor image is an indoor image including the 2D feature point among the 2D feature points of the image; and Obtaining a third intermediate probability of the 2D feature point through a projection error of the 2D feature point, wherein the projection error is obtained based on the projection position coordinates of the first target 3D feature point in the current frame indoor image and the image position coordinates of the 2D feature point, and the first target 3D feature point is a 3D feature point corresponding to the 2D feature point in the previous frame indoor image; The first intermediate probability, the second intermediate probability and the third intermediate probability are weightedly summed to obtain a static probability value of the 2D feature point.

5. The method according to claim 4, characterized in that The obtaining, according to the image position coordinates of the 2D feature point in the current frame indoor image and the image position coordinates of the 2D feature point in the previous frame indoor image, a first intermediate probability of the 2D feature point comprises: Based on the image position coordinates of the 2D feature points in the current frame indoor image and the image position coordinates of the 2D feature points in the previous frame indoor image, two sets of epipolar line distances of the 2D feature points are obtained, and according to the two sets of epipolar line distances of the 2D feature points, a first intermediate probability of the 2D feature points is obtained; Among them, any set of epipolar line distances includes a first epipolar line distance and a second epipolar line distance, the first epipolar line distance is the distance between the 2D feature point in the current frame indoor image and the epipolar line in the previous frame indoor image, and the epipolar line in the previous frame indoor image is the intersection line between the previous frame indoor image and the epipolar plane, and the epipolar plane is a plane composed of the 3D feature point corresponding to the 2D feature point in the current frame indoor image, the camera center point corresponding to the current frame indoor image, and the camera center point corresponding to the previous frame indoor image; the second epipolar line distance is the distance between the 2D feature point in the previous frame indoor image and the epipolar line in the current frame indoor image, and the epipolar line in the current frame indoor image is the intersection line between the current frame indoor image and the epipolar plane; The obtaining, based on the observed value of the 2D feature point, a second intermediate probability of the 2D feature point comprises: The second intermediate probability is obtained by the following formula: in is the second intermediate probability of the mth 2D feature point, N m is the observed value of the mth 2D feature point, μ N is the average value of the observed values ​​of each 2D feature point in the current frame indoor image, σ N is the standard deviation of the observed values ​​of each 2D feature point in the current frame indoor image; The obtaining a third intermediate probability of the 2D feature point through the projection error of the 2D feature point includes: The third intermediate probability is obtained by: in is the third intermediate probability of the mth 2D feature point, R m is the projection error of the mth 2D feature point, μ R is the average projection error of each 2D feature point in the current frame indoor image, σ R is the standard deviation of the projection error of each 2D feature point in the current frame indoor image.

6. The method according to claim 4, characterized in that Before obtaining a plurality of category sets according to each 2D feature point in the current frame indoor image and the preset feature point category, the method further includes: For any 2D feature point, if the vertical coordinate value of the 3D feature point corresponding to the 2D feature point in the current frame indoor image is not within the specified range, and the first intermediate probability of the 2D feature point is not less than the specified probability value, the 2D feature point is deleted.

7. The method according to claim 1 or 2, characterized in that: The difference between any two 2D feature points is obtained by: Based on the target 2D feature point among the arbitrary two 2D feature points, a first difference parameter is obtained according to the observation value and the projection error of the target 2D feature point, wherein the target 2D feature point is the feature point with a smaller sequence number among the arbitrary two 2D feature points, and the observation value is the total number of target indoor images corresponding to the target 2D feature point, and the target indoor image is an indoor image including the target 2D feature point among the 2D feature points of the image, the projection error is obtained based on the projection position coordinates of the second target 3D feature point in the indoor image of the current frame and the image position coordinates of the target 2D feature point, and the second target 3D feature point is the 3D feature point corresponding to the target 2D feature point in the indoor image of the previous frame; Obtaining a second difference parameter according to the image position coordinates of the arbitrary two 2D feature points and the 3D position coordinates corresponding to the arbitrary two 2D feature points; The first difference parameter and the second difference parameter are weightedly summed to obtain the difference between the arbitrary two feature points.

8. The method according to claim 7, characterized in that The obtaining, based on the target 2D feature point among the arbitrary two 2D feature points and according to the observed value and the projection error of the target 2D feature point, a first difference parameter comprises: The first difference parameter is obtained by the following formula: Wherein, k1 is the first difference parameter, N i is the observed value of the target 2D feature point, R i is the projection error of the target 2D feature point, μ N is the mean of the observed values ​​of each 2D feature point in the current frame indoor image, μ R is the average projection error of each 2D feature point in the current frame indoor image, σ N is the standard deviation of the observed values ​​of each 2D feature point in the current frame indoor image, σ R is the standard deviation of the projection error of each 2D feature point in the current frame indoor image; The obtaining a second difference parameter according to the image position coordinates of the arbitrary two 2D feature points and the 3D position coordinates corresponding to the arbitrary two 2D feature points includes: The difference between any two feature points is obtained by the following formula: Wherein, k2 is the difference, P i is the matrix of the 3D position coordinates corresponding to the target 2D feature points among the two arbitrary 2D feature points, P j is the matrix of the 3D position coordinates corresponding to the other 2D feature point of any two 2D feature points, i and j are the serial numbers of the feature points, and i <j,p i is the matrix of the image position coordinates of the target 2D feature points among the two arbitrary 2D feature points, p j is the matrix of the image position coordinates of the other 2D feature point among any two 2D feature points, σ P is the standard deviation matrix of the 3D position coordinates of any two 2D feature points, σ p is the standard deviation matrix of the image position coordinates of any two 2D feature points.

9. The method according to claim 1, characterized in that: The target category set is obtained according to the confidence of each category set, including: Determine the category set with the largest confidence as the target category set; The actual category of each 2D feature point is obtained through the target category set, including: For any 2D feature point, the predicted category of the 2D feature point in the target category set is determined as the actual category of the 2D feature point.

10. A VR device, characterized in that: comprising a processor and a memory, wherein the processor and the memory are connected via a bus; The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program: According to each 2D feature point in the current frame indoor image and the preset feature point category, a plurality of category sets are obtained, wherein the feature point category includes a static category and a dynamic category, and any one of the category sets includes the predicted category of each 2D feature point, and the predicted category is any one of the feature point categories; For any one of the category sets, based on the predicted category of each of the 2D feature points in the category set, determining the category probability of each of the 2D feature points; wherein the category probability of any one of the 2D feature points is the probability that the actual category of the 2D feature point is the same as the predicted category; and, Obtaining the confidence of the category set through the category probabilities of the respective 2D feature points and the difference between any two 2D feature points in the category set, wherein the difference between the any two 2D feature points is used to indicate the probability that the actual categories of the any two 2D feature points are different feature point categories, and the difference between the any two 2D feature points is obtained based on the image position coordinates of the respective two 2D feature points; According to the confidence of each category set, a target category set is obtained, and the actual category of each 2D feature point is obtained through the target category set; The actual categories of the 2D feature points are used to obtain the 2D static feature points corresponding to the static categories, and the positions of the indoor stationary objects are obtained based on the 2D static feature points.

Images