Method, device and storage medium for determining interior points

Abstract

Disclosed are a method, device, and storage medium for determining inliers. Through the solution provided in the embodiments of the present application, based on a determined first set of associated points, corresponding multiple parameters and a covariance matrix are calculated, thereby recalculating the calculated pixel coordinates of each first associated point based on the multiple parameters. Furthermore, based on the covariance matrix, the spatial distance between the true pixel coordinates and the calculated pixel coordinates is determined. First associated points whose spatial distance is not less than a preset value are then identified as outliers and eliminated, thereby generating a first set of inliers.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of visual positioning technology, and in particular to a method and device for determining an interior point and a storage medium thereof. Background Art

[0002] Visual positioning technology has been widely used in scenarios requiring high-precision positioning, such as assisted driving, autonomous driving, lane-level navigation, visual AR (pedestrian / car AR navigation, AR short video live broadcast), and high-precision map production. Visual positioning basically relies on visual simultaneous localization and mapping (SLAM) technology.

[0003] SLAM technology performs positioning through matching feature points between frame images taken by visual sensors. Among them, feature points can be divided into inliers and outliers according to their matching degree. Inliers are feature points with high matching degree, and outliers are feature points with low matching degree. In the positioning process, how to accurately screen out inliers and ensure the accuracy of visual positioning is a problem that technicians in this field need to solve. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a more accurate visual positioning solution.

[0005] In a first aspect, an embodiment of the present application provides an inlier determination method, comprising: determining a first associated point set of an Nth frame image and the real pixel coordinates of a first associated point in the set; determining parameter values ​​of a plurality of parameters related to the real pixel coordinates based on the real pixel coordinates of the first associated point; determining a covariance matrix corresponding to the plurality of parameters; determining the calculated pixel coordinates of the first associated point based on the parameter values ​​of the plurality of parameters; determining a spatial distance between the real pixel coordinates and the calculated pixel coordinates of the first associated point based on the covariance matrix; filtering out the first associated point whose spatial distance is not less than a preset value from the first associated point set to generate a first inlier set

[0006] In a second aspect, an embodiment of the present application provides a positioning and map construction method, comprising: acquiring an environmental image; determining interior points in the environmental image, wherein the interior points are obtained based on the interior point determination method described in the first aspect; and performing positioning and map construction based on the determined interior points.

[0007] In a third aspect, an embodiment of the present application provides an internal point determination device, comprising: a first associated point determination module, which determines the first associated point set of the Nth frame image and the real pixel coordinates of the first associated point in the set; a parameter value determination module, which determines the parameter values ​​of multiple parameters related to the real pixel coordinates based on the real pixel coordinates of the first associated point; a covariance determination module, which determines the covariance matrix corresponding to the multiple parameters; a calculated pixel coordinate determination module, which determines the calculated pixel coordinates of the first associated point based on the parameter values ​​of the multiple parameters; a spatial distance determination module, which determines the spatial distance between the real pixel coordinates and the calculated pixel coordinates of the first associated point based on the covariance matrix; and an internal point determination module, which filters out the first associated points whose spatial distance is not less than a preset value from the first associated point set to generate a first internal point set.

[0008] In a fourth aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0009] In a fifth aspect, an embodiment of the present application provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in the first aspect is implemented.

[0010] Through the solution provided in the embodiments of the present application, based on the first associated point set that has been determined, the corresponding multiple parameters and covariance matrix are calculated, so that the calculated pixel coordinates of each first associated point are recalculated according to the multiple parameters, and then the spatial distance between the real pixel coordinates and the calculated pixel coordinates is determined based on the covariance matrix, so that the first associated points whose spatial distance is not less than the preset value are determined as external points for elimination, and the first internal point set is generated to avoid the impact of erroneous association on subsequent positioning and improve the accuracy of visual positioning.

[0011] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory and do not limit the embodiments of the present application.

[0012] In addition, any embodiment of the present application does not need to achieve all of the above-mentioned effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0014] Figure 1A flow chart of an interior point determination method provided in an embodiment of the present application;

[0015] Figure 2a This is a schematic diagram of the first correlation point in the Nth frame image given in an embodiment of the present application;

[0016] Figure 2b A schematic diagram of a region division provided in an embodiment of the present application;

[0017] Figure 2c Schematic diagram of the distribution radius of the reliable first inner point provided in the embodiment of the present application

[0018] Figure 3 A schematic diagram of a process for further eliminating outliers based on RANSAC provided in an embodiment of the present application;

[0019] Figure 4 A flowchart of a positioning and map construction method provided in an embodiment of the present application;

[0020] Figure 5 A schematic structural diagram of an interior point determination device provided in an embodiment of the present application;

[0021] Figure 6 It is a structural diagram of a device used to configure the method of the embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described in detail below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art should fall within the scope of protection.

[0023] In visual positioning, it basically relies on simultaneous localization and mapping (SLAM) technology. In SLAM, feature point matching is required. Feature point matching often includes correctly matched feature points and incorrectly matched feature points. In other words, feature points in the image that are considered to meet the current posture judgment are inliers, while feature points that do not meet the current posture judgment are outliers. The posture here includes the inclination angle of the camera in the world coordinate system when the image is taken, the coordinates of the corner points (the points in the real space corresponding to the feature points) in the world coordinate system, and the coordinates of the camera in the world coordinate system when the image is taken, etc.

[0024] Based on this, the embodiment of the present application provides a more accurate method for determining interior points, such as Figure 1 As shown, Figure 1 A flow chart of an interior point determination method provided in an embodiment of the present application includes:

[0025] S101, determining a first associated point set of the Nth frame image and the real pixel coordinates of the first associated point in the set.

[0026] In visual positioning, multiple frames of images are usually captured continuously to perform feature point search and image positioning. In the embodiment of the present application, the inliers in the image of the N-1th frame have been determined, and N>1.

[0027] The first associated point refers to a feature point in the Nth frame image that has a one-to-one correspondence (for example, it may correspond to the same corner point in reality) with the second inner point in the previous frame image (i.e., the N-1th frame image). The real pixel coordinates of the first associated point refer to the pixel coordinate values ​​of the first associated point in the Nth frame image. There are usually multiple first associated points in the first associated point set.

[0028] Since the second inlier point has been determined as an inlier point in the N-1th frame image, the first associated point and the second inlier point both correspond to a corner point in reality (which can be a static corner point or a moving corner point). Therefore, there are still some incorrectly associated feature points in the first associated point set.

[0029] S103: Determine parameter values ​​of multiple parameters related to the real pixel coordinates according to the real pixel coordinates of the first associated point.

[0030] The multiple parameters include the camera's tilt parameter matrix R in the world coordinate system when capturing the Nth frame, the camera's coordinate parameter matrix t in the world coordinate system, and the corner point coordinate parameter matrix P in the world coordinate system. Obviously, the parameter values ​​are usually different in different images due to the different camera poses and coordinates. The coordinates of the corner points in real space may also be different (for example, moving objects may have different world coordinates).

[0031] In fact, if the internal parameter K of the camera is determined to be unchanged, the relationship between the pixel coordinates and the pose parameters has been determined to exist as follows, that is, the pixel coordinate h p =K*(R T *(Pt)). Here R T Refers to the transposed matrix of the tilt matrix R.

[0032] S105: Determine a covariance matrix corresponding to the multiple parameters.

[0033] Therefore, based on the multiple real pixel coordinates h that have been obtained p, that is, the parameter values ​​of each pose parameter can be solved. Specifically, the incremental ISAM2 based on the Bayesian tree and the recursive method based on the sparse Cholesky decomposition can be used to solve the pose parameter values ​​in the N-th frame image, and the covariance of each pose parameter can be determined respectively, so as to obtain the pixel coordinate h p The covariance matrix ∑ of each variable.

[0034] The covariance matrix is ​​a symmetric matrix. Each element in the matrix is ​​the covariance of the real pixel coordinates between two variables, and the elements on the diagonal are the variance of the real pixel coordinates on one variable. In this application, the real pixel coordinates h p The covariance between the two variables includes Cov(R, t), Cov(R, P) and Cov(P, t), that is, the covariance matrix

[0035] Among them, Cov(R, t) is equal to Cov(t, R), Cov(R, R) represents the variance of the real pixel coordinate on the variable R, and the rest are similar.

[0036] S107: Determine the calculated pixel coordinates of the first associated point according to the parameter values ​​of the multiple parameters.

[0037] For any coordinate, after the values ​​of R, P and t have been determined, the corner point corresponding to any associated point can be re-projected into the Nth frame image, and according to the above formula h p =K*(R T *(Pt)) to calculate the pixel coordinate Z of any first associated point k .

[0038] Since the parameters R, P and t are obtained based on the real pixel coordinates of the first associated points, the calculated pixel coordinate Z is actually obtained by reprojecting the parameters R, P and t. k There is always a deviation from the true pixel coordinates, and whether a first associated point is incorrectly associated can be evaluated based on the degree of this deviation.

[0039] S109: Determine the spatial distance between the real pixel coordinates and the calculated pixel coordinates of the first associated point according to the covariance matrix.

[0040] Specifically, the spatial distance D between the real pixel coordinates and the calculated pixel coordinates is calculated in the following manner: ML , or, where ∑ -1 is the inverse matrix of the covariance matrix ∑, and the superscript T represents the transposed matrix.

[0041] S111 , filtering out first associated points whose spatial distances are not less than a preset value from the first associated point set to generate a first inlier point set.

[0042] For any first associated point, if the calculated spatial distance D ML Less than a certain preset value χ 2 (This value can be given in advance based on experience), that is, it is considered that the deviation between the calculated pixel coordinates and the real pixel coordinates is very small. Therefore, the first associated point can be considered as a correctly associated inner point, otherwise it is considered as an incorrectly associated outer point, that is, D ML ≤χ 2 The point is confirmed as the first inlier point, and the first inlier point set is generated. In the subsequent visual positioning and map construction, the first inlier point is used as the correctly associated feature point.

[0043] like Figure 2a As shown, Figure 2a This is a schematic diagram of the first correlation points provided in the embodiments of this application. Both the black and white circles represent first correlation points. The black circles are identified as outliers and need to be removed, while the white circles represent first inliers that have been confirmed to be accurately correlated. The accurately correlated first inliers obtained in one frame can be used to determine inliers in the next frame.

[0044] Through the solution provided in the embodiments of the present application, based on the first associated point set that has been determined, the corresponding multiple parameters and covariance matrix are calculated, so that the calculated pixel coordinates of each first associated point are recalculated according to the multiple parameters, and then the spatial distance between the real pixel coordinates and the calculated pixel coordinates is determined based on the covariance matrix, so that the first associated points whose spatial distance is not less than the preset value are determined as external points for elimination, thereby preventing erroneous associations from affecting subsequent positioning and improving the accuracy of visual positioning.

[0045] On the basis that the aforementioned scheme has determined multiple first inliers, further outlier removal can be performed. Specifically, it can be performed based on the Random Sample Consensus (RANSAC) method. In RANSAC, it is necessary to calculate the polar constraint matrix based on the coordinates of multiple points in different images. Therefore, how to select reliable associated matching points is the most critical step. In this application, based on the aforementioned scheme, multiple first inliers have been determined in the Nth frame image. Therefore, further outlier removal based on RANSAC can be performed by taking the following steps, such as Figure 3 As shown, Figure 3 The flowchart of further removing outliers based on RANSAC provided in the embodiment of the present application includes:

[0046] S301: Determine a reliable area of ​​the Nth frame image.

[0047] The Nth frame image may be divided into a plurality of regions, and a reliable region may be selected from the regions. For example, the Nth frame image may be divided into a plurality of regions, and for each region, a ratio of the number of first inliers to first associated points in the grid is determined; and a region for which the ratio of the number of first inliers to first associated points exceeds a preset value is determined as a reliable region.

[0048] When dividing the image, it can be divided randomly, or the image of the Nth frame can be divided into a plurality of equally spaced grids, each grid corresponding to a region. For example, the image of the Nth frame can be divided into a plurality of grid regions according to a preset grid size, that is, the grids are of the same size. Figure 2b As shown, Figure 2b A schematic diagram of a regional division provided in an embodiment of the present application. Figure 2b The Nth frame image is divided into multiple square grids at equal distances, and each grid corresponds to a region. Assuming the preset ratio is 40%, that is, if the number of white circles in a grid exceeds 40%, the grid is confirmed as a reliable region, and then the inner point is selected from the reliable region. Figure 2b In three areas, the number of black circles exceeds 60%, so these three grids are unreliable areas and will not be selected as the first inlier point. At the same time, the number of white circles in some grids is zero, so the first inlier point will not be selected from these grids either.

[0049] S303, selecting a credible first interior point from the first interior points whose real pixel coordinates fall into the reliable area;

[0050] A credible first inlier point is the first inlier point whose true pixel coordinates fall within the reliable region. Specifically, a credible first inlier point can be selected from each reliable region. Alternatively, each reliable region can be sorted based on its inlier point ratio (i.e., the ratio of the number of first inliers to the number of first associated points in the reliable region), thereby preferentially selecting the first inlier point from the reliable regions with the highest ratio.

[0051] The method for selecting the reliable first inlier may also be a combination of the two aforementioned methods. For example, one reliable first inlier is selected from each of the seven reliable regions ranked at the top of the inlier ratio, thereby obtaining seven credible first inliers.

[0052] By dividing the points into reliable regions, the RANSAC point selection process selects the first reliable inliers from multiple different regions, thus preventing the problem of excessive concentration of points leading to large errors in solving the epipolar constraint matrix.

[0053] S305 , for each selected credible first interior point, determine an epipolar constraint matrix according to the real pixel coordinates of the first interior point and the real pixel coordinates of the second interior point in the (N−1)th frame image.

[0054] Here, the N-1th frame image is the frame image preceding the Nth frame image, and the second inlier point is a feature point that has been identified as an inlier point in the N-1th frame image. The second inlier point and the first associated point correspond to the same corner point in reality, so there is a one-to-one correspondence between the two. Based on this, an epipolar constraint matrix can be determined using limit constraints.

[0055] For a corner point P in three-dimensional space, the coordinate matrix of the first associated point projected in the Nth frame image is recorded as P1, and the coordinate matrix of the second inner point in the N-1th frame image is P2, then there is an association relationship P1 between them. T *F*P2=0, where F is the polar constraint matrix, and F is a 3*3 basic matrix with a rank of 2.

[0056] Since P1 and P2 are actually already determined, F can be determined by selecting multiple credible first inner points and corresponding second inner points. The specific number of selected points can be 1, 2, 5, 7, or 8, etc., and can be determined based on actual needs.

[0057] S307 : Filter out first inliers with low confidence in the first inlier set according to the epipolar constraint matrix.

[0058] The limit constraint matrix determines the two epipolar lines corresponding to the feature points in the two images. In theory, corresponding feature points in the two images should lie on the corresponding epipolar lines after mapping. The greater the distance between the mapped point and the epipolar line, the lower the confidence that the point is an inlier. Therefore, the confidence level of the first inlier point can be determined based on its distance from the epipolar line or the projection error after mapping using the limit constraint matrix.

[0059] In this application, the second inlier point is actually already confirmed as an inlier point. The feature points confirmed as inliers in the Nth frame image will also be used to determine inliers in subsequent frames. Therefore, the epipolar equation corresponding to the first inlier point can be determined based on the already determined epipolar constraint matrix, thereby determining the projection error corresponding to each first inlier point in the first set of inliers.

[0060] Furthermore, first inliers whose projection errors exceed a preset error threshold can be determined as outliers and eliminated, while first inliers whose projection errors do not exceed the preset error threshold can be determined as remaining first inliers, thereby obtaining the remaining first inliers in the first inlier point set. If the number and distribution radius of the remaining first inliers in the first inlier point set meet preset conditions, the remaining first inliers in the first inlier point set are retained, and the remaining first inliers can become the remaining inlier point set.

[0061] Obviously, the remaining inlier point set is a subset of the first inlier point set. In the subsequent positioning and image construction, using the remaining inlier point set as the correctly associated points will further improve the accuracy compared to using the first inlier point set.

[0062] For the remaining first inliers in the first inlier set, the epipolar constraint matrix is ​​essentially calculated based on a small number of selected first inliers, and it is assumed that the remaining first inliers will subsequently conform to this epipolar constraint matrix. Therefore, due to the randomness of the selected first inliers, it often happens that the majority of points do not conform to the epipolar constraint matrix, resulting in the elimination of most first inliers. Therefore, the preset condition can be set to ensure that the number of candidate inliers in the candidate inlier set exceeds a preset threshold, thereby improving the robustness of the outlier elimination algorithm.

[0063] In one embodiment, the distribution radius of the credible first inner point may be pre-calculated. The distribution radius of the credible first inner point refers to the radius r1 of a circle that can encompass the selected credible first inner point.

[0064] For example, one practicable method is to determine the mean of the real pixel coordinates of multiple credible first inner points as the center of the circle, so that the distribution radius is the maximum value of the distances from the center of the circle to the multiple credible first inner points. Figure 2c As shown, Figure 2c A schematic diagram of the distribution radius of the credible first inliers provided in an embodiment of the present application. The dashed white circle in the figure represents the selected credible first inliers, and the center of the circle is the mean of the selected credible first inliers.

[0065] Furthermore, the same method can be used to calculate the distribution radius r2 of the remaining first inliers in the remaining inlier set. If r2 / r1>k, where k is a preset distribution ratio (e.g., k=0.3), the remaining inlier set can be retained. This method combines the distribution radius of the credible first inlier as the prior distribution radius for point detection, ensuring that the remaining first inliers after screening are evenly distributed, which is beneficial for subsequent positioning and map construction.

[0066] In RANSAC, since selecting multiple credible inliers at a time will result in a corresponding set of remaining inliers, multiple samplings are actually always used, that is, each time the same number of credible inliers is selected to calculate a set of remaining inliers, and then multiple sets of remaining inliers may be obtained. Since not every set of remaining inliers can meet the preset conditions, the number of remaining inlier sets that are finally retained is usually less than the number of samplings.

[0067] Therefore, on the one hand, the actual preset conditions may also include that the number of sampling times (ie, the number of random selections) does not exceed a preset value, for example, not more than 30 times, thereby improving calculation efficiency.

[0068] On the other hand, since the number of reliable inliers selected in each sampling is different, the number of remaining first inliers contained in the remaining remaining inlier set is often different. Therefore, when there are multiple remaining inlier sets that meet the preset conditions, the remaining inlier set with the largest number of remaining first inliers can be determined as the target inlier set, and the target inlier set can be used for subsequent positioning and map construction, thereby further improving the accuracy of subsequent positioning and map construction.

[0069] Furthermore, since RANSAC may result in too few or too many first inliers being eliminated, in order to maintain the stability of the sampled target inlier set, the number of sampling times M (i.e., the number of random selections) can be adjusted based on the number of inliers N3 in the target inlier set and the number of inliers N1 in the first inlier set.

[0070] That is, the value of N3 / N1 is determined. When N3 / N1 is less than the preset value, it means that the number of the largest number of inliers in the sampled set is still insufficient. Therefore, the number of samplings can be increased again to achieve sufficient sampling selection to obtain more remaining inliers, thereby increasing the number of remaining first inliers contained in the remaining inlier set to meet the needs of subsequent positioning or map construction. When N3 / N1 is greater than the preset value, it means that the number of inliers in the remaining inlier set is sufficient to meet the needs, and the number of samplings can be reduced again to improve the calculation efficiency.

[0071] In a second aspect of the present application, a positioning and map construction method is also provided, such as Figure 4 As shown, Figure 4 This is a flow chart of a positioning and map construction method provided in an embodiment of the present application, the method comprising:

[0072] S401, acquiring an environment image; the environment image may be an image in an open space, such as an image captured while a vehicle is moving on a road, or an image in a closed environment, such as an image captured indoors.

[0073] S403, determining the interior points in the environment image, wherein the interior points are obtained based on the aforementioned method; the interior points include at least one of the interior points included in the aforementioned first interior point set, the remaining interior point set, or the target interior point set.

[0074] S405: Perform positioning and map construction based on the determined interior points.

[0075] Correspondingly, the embodiment of the present application also provides an interior point determination device, such as Figure 5 As shown, Figure 5 : is a schematic structural diagram of an interior point determination device provided in an embodiment of the present application, comprising:

[0076] The first connection point determination module 501 determines a first connection point set of the Nth frame image and the real pixel coordinates of the first connection point in the set;

[0077] A parameter value determination module 503 is configured to determine parameter values ​​of a plurality of parameters related to the real pixel coordinates according to the real pixel coordinates of the first associated point;

[0078] A covariance determination module 505 is configured to determine a covariance matrix corresponding to the plurality of parameters;

[0079] A calculated pixel coordinate determining module 507 is configured to determine the calculated pixel coordinates of the first associated point according to the parameter values ​​of the plurality of parameters;

[0080] A spatial distance determination module 509 is configured to determine the spatial distance between the real pixel coordinates and the calculated pixel coordinates of the first associated point according to the covariance matrix;

[0081] The inlier point determination module 511 filters out first associated points whose spatial distances are not less than a preset value from the first associated point set to generate a first inlier point set.

[0082] Optionally, the inlier determination module 511 determines a reliable area of ​​the Nth frame image; selects a credible first inlier from the first inliers whose real pixel coordinates fall into the reliable area; for each selected credible first inlier, determines an epipolar constraint matrix based on the real pixel coordinates of the first inlier and the real pixel coordinates of the second inlier in the N-1th frame image, wherein the credible first inlier corresponds one-to-one to the second inlier; and filters out low-confidence first inliers in the first inlier set based on the epipolar constraint matrix.

[0083] Optionally, the inlier determination module 511 determines the projection error corresponding to the first inlier point based on the polar constraint matrix, the real pixel coordinates of the first inlier point and the real pixel coordinates of the second inlier point; filters out the first inlier points in the first inlier point set whose projection errors exceed a preset error threshold; determines whether the number and distribution radius of the remaining first inlier points in the first inlier point set meet preset conditions, and retains the remaining first inlier points if the conditions are met.

[0084] Optionally, the preset condition includes at least one of the following conditions: the number of first inliers remaining in the first inlier set exceeds a preset number threshold; or the ratio of the distribution radius of the first inliers remaining in the first inlier set to the distribution radius of the credible first inliers exceeds a preset distribution ratio.

[0085] Optionally, the interior point determination module 511 selects reliable first interior points M times from the first interior points whose real pixel coordinates fall within the reliable area, where M is an integer greater than or equal to 2; after processing the reliable first interior points selected M times, the method further includes: comparing the number of remaining first interior points, and selecting the largest number of remaining first interior points as the target interior point set of the Nth frame image.

[0086] Optionally, the apparatus further includes an adjustment module 513 that determines a new value of M based on a ratio of the number of inliers in the target inlier set to the number of inliers in the first inlier set, where the new value of M is used to determine the target inlier set for the N+1th frame image.

[0087] Optionally, the interior point determination module 511 divides the Nth frame image into multiple regions; determines the region to which the first interior point belongs based on the real pixel coordinates of the first interior point in the first interior point set; determines the ratio of the number of the first interior point to the first associated point in each region; and determines the region where the ratio of the number of the first interior point to the first associated point exceeds a preset value as a reliable region.

[0088] Optionally, the inlier determination module 511 selects at least one reliable first inlier from each reliable region; or randomly selects a first inlier from the reliable regions ranked at the top in terms of the proportion of first inliers.

[0089] Optionally, in the device, the parameters include: posture parameters of the camera in the world coordinate system when shooting the Nth frame of image; and coordinates of the corner point corresponding to the first associated point in the world coordinate system.

[0090] The embodiment of the present application also provides a computer device, which includes at least a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to realize Figure 1The interior point determination method shown.

[0091] Figure 6 10 is a schematic diagram showing a more specific hardware structure of a computing device provided in an embodiment of the present application. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other within the device via the bus 1050.

[0092] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0093] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of the present application are implemented through software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0094] The input / output interface 1030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0095] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0096] The bus 1050 comprises a path for transmitting information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0097] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of the present application, and does not necessarily include all the components shown in the figure.

[0098] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, Figure 1 The interior point determination method shown.

[0099] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0100] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that the embodiments of the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions of the embodiments of the present application are essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments of the present application.

[0101] The systems, methods, modules, or units described in the above embodiments may be implemented by a computer chip or entity, or by a product having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0102] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the method embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The method embodiment described above is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the functions of each module can be implemented in the same one or more software and / or hardware when implementing the embodiment of this application. It is also possible to select some or all of the modules according to actual needs to achieve the purpose of the embodiment. Those of ordinary skill in the art can understand and implement it without paying any creative work.

[0103] The above is only a specific implementation of the embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the embodiment of the present application. These improvements and modifications should also be regarded as the scope of protection of the embodiment of the present application.

Claims

1. A method for determining interior points, comprising: Determine a first associated point set of the Nth frame image and the real pixel coordinates of the first associated point in the set, wherein the first associated point refers to a feature point in the Nth frame image that has a one-to-one correspondence with the second inner point in the (N-1)th frame image; Determining, based on the real pixel coordinates of the first associated point, parameter values ​​of a plurality of parameters associated with the real pixel coordinates, wherein the plurality of parameters include a tilt parameter matrix R of the camera in the world coordinate system when capturing the Nth frame of image, a coordinate parameter matrix t of the camera in the world coordinate system, and a coordinate parameter matrix P of the corner points in the world coordinate system; Determining a covariance matrix corresponding to the plurality of parameters; determining the calculated pixel coordinates of the first associated point according to the parameter values ​​of the plurality of parameters; Determining a spatial distance between the real pixel coordinates and the calculated pixel coordinates of the first associated point according to the covariance matrix; From the first associated point set, first associated points whose spatial distance is not less than a preset value are filtered out to generate a first inlier point set.

2. The method of claim 1, further comprising: Determining a reliable area of ​​the Nth frame image; Selecting a credible first inner point from the first inner points whose real pixel coordinates fall into the reliable area; For each selected credible first interior point, determine an epipolar constraint matrix based on the real pixel coordinates of the first interior point and the real pixel coordinates of the second interior point in the (N-1)th frame image, wherein the credible first interior point corresponds one-to-one to the second interior point; According to the epipolar constraint matrix, first inlier points with low confidence in the first inlier point set are filtered out.

3. The method of claim 2, wherein filtering out low-confidence first inliers in the first inlier set according to the epipolar constraint matrix comprises: determining a projection error corresponding to the first interior point based on the epipolar constraint matrix, the true pixel coordinates of the first interior point, and the true pixel coordinates of the second interior point; Filtering out first inlier points whose projection errors exceed a preset error threshold in the first inlier point set; It is determined whether the number and distribution radius of the remaining first inliers in the first inlier set meet preset conditions, and if so, the remaining first inliers are retained.

4. The method according to claim 3, wherein the preset condition comprises at least one of the following conditions: The number of remaining first inlier points in the first inlier point set exceeds a preset threshold; or A ratio of the distribution radius of the remaining first inliers in the first inlier set to the distribution radius of the credible first inliers exceeds a preset distribution ratio.

5. The method according to claim 3, wherein: The selecting of a credible first interior point from the first interior points whose real pixel coordinates fall within the reliable area specifically includes: Selecting a reliable first interior point M times from the first interior points whose real pixel coordinates fall within the reliable area, where M is an integer greater than or equal to 2; After the M selected credible first interior points are processed, the method further includes: The numbers of the remaining first inliers are compared, and the remaining first inliers with the largest number are selected as the target inlier set of the N-th frame image.

6. The method of claim 5, further comprising: A new value of M is determined according to the ratio of the number of inliers in the target inlier set to the number of inliers in the first inlier set, and the new value of M is used to determine the target inlier set of the N+1th frame image.

7. The method according to claim 2, wherein determining the reliable area of ​​the Nth frame image comprises: Dividing the Nth frame image into a plurality of regions; determining, according to the real pixel coordinates of a first inlier point in the first inlier point set, the region to which the first inlier point belongs; Determine the ratio of the number of first inliers and first associated points in each region; A region where the ratio of the number of the first inliers to the number of the first associated points exceeds a preset value is determined as a reliable region.

8. The method according to claim 7, wherein selecting a credible first interior point from the first interior points whose real pixel coordinates fall within the reliable region comprises: From each reliable region, select at least one reliable first inlier point; Alternatively, the first inlier point is randomly selected from the reliable region with the highest proportion of the first inlier point.

9. A positioning and mapping method comprising: Get the environment image; Determining interior points in the environment image, wherein the interior points are obtained based on the method according to any one of claims 1 to 8; Positioning and map construction are performed based on the determined interior points.

10. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images