A method and related device for realizing SLAM positioning based on monocular driving recorder

Abstract

The present application discloses a method and related device for realizing SLAM positioning based on a monocular driving recorder. The method includes: obtaining at least two frames of pictures collected by the monocular driving recorder during vehicle driving; extracting feature points after partitioning each frame of the picture; matching the feature points of each frame of the picture to obtain a first feature point set that successfully matches; constructing the three-dimensional spatial coordinates of the first feature point set; obtaining the next frame of the picture, extracting feature points after partitioning; determining the position of the monocular driving recorder when shooting the next frame of the picture according to the feature points of the next frame of the picture and the three-dimensional spatial coordinates of the first feature point set; determining the position of the vehicle according to the position of the monocular driving recorder. The solution provided by the present application can realize the real-time positioning of the vehicle by using the picture under monocular vision, so as to obtain the moving trajectory of the vehicle by continuously updating the positioning later; and extracting features after partitioning the picture can make the feature distribution uniform, which is conducive to the construction of three-dimensional space.

Description

Technical Field

[0001] The present application relates to the field of navigation technology, and in particular to a method and related device for realizing SLAM positioning based on a monocular driving recorder. Background Art

[0002] SLAM (Simultaneous Localization And Mapping) is mainly used to solve the problem of positioning, navigation and mapping of mobile devices when they are running in unknown environments. To perform positioning and mapping, data must be collected first. Currently, most of them use binocular cameras or laser sensors to collect data. However, for devices with only a single camera, such as a single-lens dash cam, when real-time positioning, navigation and mapping are required, conventional methods cannot be used to achieve this.

[0003] In addition, the images captured by the dashcam generally show more features on both sides of the road, but fewer features in the sky and on the road surface. Therefore, when extracting feature points on the image, it is easy for the feature points to be concentrated in a local area, while feature points in other places will be ignored, which is not conducive to map construction. Therefore, how to use the video images collected by the monocular dashcam to effectively achieve real-time positioning and map construction is a technical issue that is worth studying. Summary of the invention

[0004] The present application provides a method and related device for realizing SLAM positioning based on a monocular driving recorder, which can realize real-time positioning of a vehicle by using images under monocular vision, so as to obtain the movement trajectory of the vehicle by continuously updating the positioning; and partitioning the image and then extracting features can make the feature distribution uniform, which is conducive to the construction of three-dimensional space.

[0005] The first aspect of the present application provides a method for implementing SLAM positioning based on a monocular driving recorder, comprising:

[0006] Acquire at least two frames of pictures collected by the monocular driving recorder during the vehicle driving process;

[0007] Performing partition processing on each of the at least two frames of pictures to obtain each frame of pictures after partitioning;

[0008] Extract a preset number of feature points from each region in each frame of the partitioned image, and use the set of feature points extracted from each region as the feature points of the frame of the image;

[0009] Matching the feature points of the at least two frames of images to obtain a first feature point set that successfully matches the at least two frames of images;

[0010] Constructing the three-dimensional space coordinates of the first feature point set;

[0011] Obtaining a next frame of image captured by the monocular driving recorder, and extracting feature points of the next frame of image after performing partition processing on the next frame of image;

[0012] Determine the position and posture of the monocular driving recorder when taking the next frame of picture according to the feature points of the next frame of picture and the three-dimensional space coordinates of the first feature point set;

[0013] The position of the vehicle is determined according to the posture of the monocular driving recorder when the next frame of picture is taken.

[0014] As an optional implementation manner, in the first aspect of the present application, partitioning each of the at least two frames of pictures to obtain each frame of pictures after partitioning includes:

[0015] Partitioning each of the at least two frames of images according to a preset rule to obtain each frame of images after partitioning;

[0016] Alternatively, obtain picture information of each frame of the at least two frames of pictures, determine a partition strategy for each frame of pictures according to the picture information of each frame of pictures, and partition each frame of pictures according to the partition strategy of each frame of pictures to obtain each frame of pictures after partitioning, wherein the picture information includes at least one of a pixel value, a grayscale value, and an RGB value of the picture.

[0017] As an optional implementation, in the first aspect of the present application, extracting a preset number of feature points from each region in each frame of the partitioned image, and using the set of feature points extracted from each region as the feature points of the frame of the image, includes:

[0018] A preset number of feature points are extracted from each region in each frame of the partitioned image using the brisk operator, and the feature points extracted from each region are described, and the set of feature points of each region after the description is used as the feature points of the frame of the image;

[0019] The step of matching the feature points of the at least two frames of images to obtain a first feature point set that successfully matches the at least two frames of images includes:

[0020] The feature points described by the at least two frames of images are matched, and the feature points whose matching distance is less than a preset value are determined as a first feature point set with successful matching.

[0021] As an optional implementation manner, in the first aspect of the present application, constructing the three-dimensional space coordinates of the first feature point set includes:

[0022] Using the first feature point set and adopting epipolar constraints to calculate a rotation matrix and a translation matrix between the at least two frames of images;

[0023] Generate three-dimensional spatial coordinates of the first feature point set according to the rotation matrix and the translation matrix between the at least two frames of pictures.

[0024] As an optional implementation, in the first aspect of the present application, the method further includes:

[0025] Iteratively process each frame of the image subsequently collected by the monocular driving recorder to obtain the position and posture of the monocular driving recorder when taking each frame of the image;

[0026] The moving trajectory of the vehicle is determined according to the position and posture of the monocular driving recorder when taking each frame of the picture.

[0027] As an optional implementation, in the first aspect of the present application, determining the position and posture of the monocular driving recorder when shooting the next frame of the picture according to the feature points of the next frame of the picture and the three-dimensional space coordinates of the first feature point set includes:

[0028] Matching the next frame of picture with each of the at least two frames of picture to obtain a set of feature points where the next frame of picture successfully matches each of the frames of picture;

[0029] According to the feature point set that the next frame of picture successfully matches with each frame of picture, feature points in the next frame of picture that successfully match with at least a preset number of frames of pictures in the at least two frames of pictures are determined as a second feature point set;

[0030] Determine the three-dimensional space coordinates of the second feature point set according to the three-dimensional space coordinates of the first feature point set;

[0031] The position and posture of the monocular driving recorder when taking the next frame of picture are determined by using the three-dimensional spatial coordinates of the second feature point set and the positions of the feature points in the second feature point set located on the next frame of picture.

[0032] As an optional implementation, in the first aspect of the present application, the method further includes:

[0033] Using the feature point set that successfully matches the next frame of the picture with each frame of the picture, the remaining feature point set after removing the second feature point set is removed, and triangulating the three-dimensional space coordinates of the remaining feature point set;

[0034] The three-dimensional space coordinates of the first feature point set and the three-dimensional space coordinates of the second feature point set are adjusted using the three-dimensional space coordinates of the remaining feature point set.

[0035] The second aspect of the present application provides a device for implementing SLAM positioning based on a monocular driving recorder, comprising:

[0036] An acquisition unit, used to acquire at least two frames of pictures collected by the monocular driving recorder during the driving of the vehicle;

[0037] A partitioning unit, used to perform partitioning processing on each of the at least two frames of pictures to obtain each frame of pictures after partitioning;

[0038] An extraction unit, used to extract a preset number of feature points from each region in each frame of the partitioned image, and use the set of feature points extracted from each region as feature points of the frame of the image;

[0039] A matching unit, configured to match the feature points of the at least two frames of images to obtain a first feature point set that successfully matches the at least two frames of images;

[0040] A construction unit, used to construct the three-dimensional space coordinates of the first feature point set;

[0041] The acquisition unit is further used to acquire the next frame of image collected by the monocular driving recorder;

[0042] The partitioning unit is further used to perform partitioning processing on the next frame of the picture;

[0043] The extraction unit is further used to extract feature points of the next frame of the image after partitioning;

[0044] A determination unit, configured to determine the position and posture of the monocular driving recorder when taking the next frame of picture according to the feature points of the next frame of picture and the three-dimensional spatial coordinates of the first feature point set;

[0045] The determination unit is further used to determine the position of the vehicle according to the posture of the monocular driving recorder when taking the next frame of picture.

[0046] The third aspect of the present application provides a device for implementing SLAM positioning based on a monocular driving recorder, comprising:

[0047] Processor; and

[0048] The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0049] A fourth aspect of the present application provides a non-transitory machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0050] The technical solution provided by the present application can obtain two or more frames of pictures collected successively by a monocular driving recorder during vehicle driving, partition each frame of the picture, obtain each frame of the picture after partitioning, and extract a certain number of feature points from each area of ​​each frame of the picture after partitioning, regard the set of feature points extracted from each area as the feature points of the frame of the picture, and then match the feature points of each frame of the picture to obtain the first feature point set of successful matching, so as to construct the three-dimensional space coordinates using the first feature point set; further, the next frame of the picture collected by the monocular driving recorder can be obtained, and the feature points can be extracted after partitioning the next frame of the picture according to the above method, and the position of the monocular driving recorder when shooting the next frame of the picture can be determined according to the feature points of the next frame of the picture and the three-dimensional space coordinates of the first feature point set, so as to obtain the position of the vehicle when shooting the next frame of the picture. The technical solution of the present application uses the picture under monocular vision to realize the real-time positioning of the vehicle, so that the subsequent collected pictures can obtain the moving trajectory of the vehicle by continuously updating the positioning; and the extraction of features after partitioning the picture can make the feature distribution uniform, which is conducive to the establishment of three-dimensional space, and thus helps to improve the construction accuracy of three-dimensional space.

[0051] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and other objects, features and advantages of the present application will become more apparent through a more detailed description of exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0053] Figure 1 It is a flow chart of a method for realizing SLAM positioning based on a monocular driving recorder shown in an embodiment of the present application;

[0054] Figure 2a It is a schematic diagram of a picture collected by a monocular driving recorder shown in an embodiment of the present application;

[0055] Figure 2b is a schematic diagram of the translation matrix and rotation matrix algorithm shown in the embodiment of the present application;

[0056] Figure 3 It is a schematic diagram of a vehicle movement trajectory obtained by SLAM positioning based on a monocular driving recorder shown in an embodiment of the present application;

[0057] Figure 4 It is a structural schematic diagram of a device for realizing SLAM positioning based on a monocular driving recorder shown in an embodiment of the present application;

[0058] Figure 5 It is a structural schematic diagram of another device for realizing SLAM positioning based on a monocular driving recorder shown in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The preferred embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0060] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0061] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0062] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0063] See also Figure 1 , the embodiment of the present application provides a method for implementing SLAM positioning based on a monocular driving recorder. Figure 1 As shown, the method may at least include the following steps:

[0064] 110. Obtain at least two frames of images captured by a monocular driving recorder during the driving of the vehicle.

[0065] In an embodiment of the present application, a monocular driving recorder can be arranged at the front windshield of the vehicle. During the driving of the vehicle, the monocular driving recorder can be used to collect video data in front of the vehicle. In order to obtain a picture, it is necessary to extract frames from the acquired video data. Usually, the frame rate of the video is 30 frames per second, and the video can be extracted according to preset rules to obtain a picture. The above-mentioned at least two frames of pictures can be two or more consecutive frames of pictures collected by the monocular driving recorder in succession. Specifically, the above-mentioned at least two frames of pictures can be real-time pictures obtained by extracting frames from the real-time video collected by the monocular driving recorder during the driving of the vehicle, or they can be a few frames in a sequence of frames of pictures obtained by extracting frames from the entire video collected by the monocular driving recorder during the entire driving process of the vehicle, which is not limited here.

[0066] It is understandable that the embodiments of the present application are described by taking a monocular driving recorder on a vehicle as an example, and other monocular devices on the vehicle, such as a monocular camera, a mobile phone, etc., which can collect monocular video. In addition, the monocular device can be set at the front of the vehicle to collect the video in front of it, or at the rear of the vehicle to collect the video behind it, which is not limited here.

[0067] 120. Perform partition processing on each of the at least two frames of images to obtain each frame of images after partitioning.

[0068] 130. Extract a preset number of feature points from each region in each frame of the partitioned image, and use the set of feature points extracted from each region as feature points of the frame of the image.

[0069] In the embodiment of the present application, the feature points on the image can be used to identify some target objects on the image. Generally, the points where the grayscale value on the image changes dramatically or the points with large curvature on the edge of the image (such as the intersection of two edges) are regarded as the feature points of the image. In order to better perform image matching later, stable points in the image that do not change with the movement, rotation or change of illumination of the camera can generally be selected as feature points. Figure 2a FIG. 2 shows one of the frames of an image captured by a monocular driving recorder while a vehicle is driving. In FIG. 2a , feature points in fixed buildings (such as houses on the roadside), fixed trees, or billboards can be selected instead of feature points in the sky or on the ground.

[0070] like Figure 2aAs shown, the pictures collected by the driving recorder generally have more features on both sides of the road and fewer features in the sky and on the road surface. When extracting feature points from the pictures, it may be possible to concentrate on extracting the feature points in a certain area, resulting in the phenomenon that the feature points are concentrated in a local area. This will lead to an overly large weight of local features, while the feature points in other places will be ignored. This situation is not conducive to the construction of a three-dimensional space. Based on this, the embodiments of the present application perform partition processing on each frame of picture to divide each frame of picture into several regions, and the number of these regions can be 2, 4, 5, 6, 8 or other values, etc. Each region can contain feature points. When extracting the feature points in the picture, a preset number of feature points can be extracted from each region on the frame of picture, and these feature points are combined to form the feature points of the frame of picture, so that the positions of the extracted feature points can be distributed as evenly as possible, which is more conducive to the construction of a three-dimensional space and can further improve the construction accuracy of the three-dimensional space.

[0071] In an alternative embodiment, the specific implementation of step 120 for performing partition processing on each of the at least two frames of pictures to obtain each partitioned frame of picture may include the following steps:

[0072] 11) Perform partition processing on each of the at least two frames of pictures according to a preset rule to obtain each partitioned frame of picture; or,

[0073] 12) Obtain the picture information of each of the at least two frames of pictures, determine the partition strategy for each frame of picture according to the picture information of each frame of picture, and perform partition processing on each frame of picture according to the partition strategy of each frame of picture to obtain each partitioned frame of picture, where the picture information may include but is not limited to at least one of the pixel value, grayscale value, and RGB value of the picture.

[0074] Specifically, the partition rule can be set in advance according to the pictures collected by the monocular driving recorder, and then each frame of picture is partitioned according to the partition rule. For example, according to the fact that the pictures collected by the driving recorder generally have more features on both sides of the road and fewer features in the sky and on the road surface, the picture can be divided into 4 regions in a "field" shape. Or, in order to extract feature points more evenly, the regions can be divided into more, such as 6, 8, 10 regions, etc.

[0075] In addition, the pictures can also be partitioned according to the situation of each frame of picture itself. Specifically, the distribution of the picture content can be determined according to the distribution of the pixel value, grayscale value, and / or RGB value, etc. of the picture, and then the partition strategy of the picture is determined according to the distribution of the picture content, and the picture will be partitioned according to the partition strategy of the picture. The partition strategy may include information such as the number of regions and the positions of the regions of the picture.

[0076] 140. Match feature points of the at least two frames of images to obtain a first feature point set that is successfully matched in the at least two frames of images.

[0077] In an embodiment of the present application, the at least two frames of pictures may contain the same object (such as a building, a billboard, a road sign, etc.) under different viewing angles. By matching the feature points on the pictures, some feature points of the same object on different pictures can be matched successfully. Among them, the first feature point set is a set of feature points that are successfully matched on each of the at least two frames of pictures. For example, when the at least two frames of pictures contain only two frames of pictures (such as two frames of pictures A and B), the first feature point set is the feature points that are successfully matched by the two frames of pictures A and B; when the at least two frames of pictures contain three frames of pictures A, B, and C, the first feature point set is the feature points that are successfully matched by the three frames of pictures A, B, and C at the same time, that is, the successfully matched feature points must appear on the three frames of pictures A, B, and C at the same time, and cannot appear on only one or two of the frames.

[0078] In an optional implementation, step 130 extracts a preset number of feature points from each region in each frame of the partitioned image, and uses the set of feature points extracted from each region as the feature points of the frame of the image. The specific implementation may include the following steps:

[0079] 13) Use the brisk operator to extract a preset number of feature points from each region in each frame of the partitioned image, and describe the feature points extracted from each region, and use the set of feature points of each region after description as the feature points of the frame of the image.

[0080] The specific implementation of step 140 of matching the feature points of the at least two frames of images to obtain a first feature point set that successfully matches the at least two frames of images may include the following steps:

[0081] 14) Matching the feature points described by the at least two frames of images, and determining the feature points whose matching distance is less than a preset value as a first feature point set with successful matching.

[0082] Specifically, the Brisk algorithm has good performance in image registration applications due to its good rotation invariance, scale invariance and good robustness. A feature point of an image can be composed of two parts: key points and descriptors. The Brisk algorithm mainly uses FAST9-16 to detect feature points, and obtains points with larger scores as feature points (i.e., key points), that is, completes the extraction of feature points. It is not possible to match feature points well using only the information of key points, so it is necessary to further obtain more detailed information to distinguish the features. Therefore, it is necessary to describe the feature points and obtain feature descriptors. The feature descriptor can eliminate the changes in the scale and direction of the image caused by the change in perspective, and can better match between images. Each feature descriptor on an image is unique and exclusive, and minimizes the similarity between each other. The Brisk feature descriptor can be represented by a binary number, such as a 256-bit or 512-bit binary number.

[0083] Match the feature descriptors of each frame of the picture. Specifically, a feature descriptor on a frame of the picture can be matched with all feature descriptors on other frames of the picture, and the matching distance (such as Hamming distance) is calculated respectively. The feature points on other frames of the picture with the smallest matching distance and a matching distance less than a preset value are taken as matching points. According to the above method, all feature points on each frame of the picture can be matched one by one to find the successfully matched feature points. It can be understood that after the matching distance is obtained, the uv coordinates of the feature points on the picture can be combined to jointly determine the matching feature points. For example, when the matching distance is less than the preset value and the difference in the uv coordinates of the feature points is also within the allowed range, it is determined as a matching feature point, otherwise it is not matched.

[0084] When a feature point on a frame of an image only matches the feature points on one or several frames of other frames, but does not match the feature points on one or some frames, the feature point can be regarded as an invalid feature point and can be discarded. When a feature point on a frame of an image can find matching feature points on other frames of an image, the feature point can be regarded as a valid feature point. All valid feature points are collected together and can be regarded as the first feature point set.

[0085] For example, when the at least two frames of images only include the A and B frames collected successively, it is assumed that 100 feature points are extracted from the A frame and 200 feature points are extracted from the B frame using the brisk algorithm. The feature points in the A and B frames are described to obtain corresponding feature descriptors; after matching all feature descriptors on the A and B frames one by one, 50 feature points with successful matching are obtained, that is, 50 feature points on the A frame match the 50 feature points on the B frame one by one, then the first feature point set can include the 50 feature points on the A frame and the 50 feature points on the B frame that have successfully matched, that is, the first feature point set can be regarded as 50 pairs of feature points.

[0086] For another example, when the above-mentioned at least two frames of images include three frames of images A, B, and C collected successively, it is assumed that 100 feature points are extracted from the A frame of images, 150 feature points are extracted from the B frame of images, and 120 points are extracted from the C frame of images using the brisk algorithm. The feature points in the A, B, and C frames of images are described to obtain corresponding feature descriptors; after matching all feature descriptors on the A, B, and C frames of images one by one, a total of 50 successfully matched feature points are obtained, that is, 50 feature points on the A frame of images, 50 feature points on the B frame of images, and 50 feature points on the C frame of images are all successfully matched, then the first feature point set can include the 50 successfully matched feature points on the A frame of images, 50 feature points on the B frame of images, and 50 feature points on the C frame of images, that is, the first feature point set can be regarded as 50 groups of feature points.

[0087] It is understandable that other algorithms (such as ORB, SURF or SIFT algorithms, etc.) can also be used to extract and describe image feature points. Using different algorithms may result in different image registration results, that is, the registration results may be different.

[0088] 150. Construct the three-dimensional space coordinates of the first feature point set.

[0089] In the embodiment of the present application, based on the successfully matched first feature point set, the epipolar geometry can be used to calculate the position change of the monocular driving recorder when collecting each frame of the picture, that is, the translation and rotation. The three-dimensional spatial coordinates of the first feature point set can then be calculated using the translation and rotation between each frame of the picture.

[0090] Specifically, in an optional implementation, the specific implementation of step 150 to construct the three-dimensional space coordinates of the first feature point set may include the following steps:

[0091] 15) Using the first feature point set and adopting the epipolar constraint, calculate the rotation matrix and the translation matrix between the at least two frames of images;

[0092] 16) Generate three-dimensional spatial coordinates of the first feature point set according to the rotation matrix and translation matrix between the at least two frames of images.

[0093] For example, when the at least two frames of images only include the two frames A and B captured successively, the feature points on the two frames A and B are matched to obtain 8 matching points, that is, the first feature point set includes 8 point pairs. Based on these 8 point pairs, the rotation matrix and translation matrix of the B frame image relative to the A frame image can be calculated.

[0094] Specifically, Figure 2b As shown in Figure 1, two frames of images of the same target object are taken at different positions, and the pixel points corresponding to the same object in the images satisfy the epipolar constraint relationship. Where P is a real object in the world coordinate system, such as a point on a building. 1 , O 2 are the optical center positions of the monocular driving recorder when taking A frame and B frame respectively. 1 ,I 2 Represents A frame picture and B frame picture respectively. 1 、p 2 are the projections of point P in frame A and frame B, that is, a pair of points that are successfully matched in frame A and frame B. 1 P is projected on the B frame image as e 2 p 2 , denoted as l 2 , O 2 P is projected on the A frame image as e 1 p 1 , denoted as l 1 , where l 1 , l 2 It is called the polar line, e 1 、e 2 is called a pole. According to the pole constraint:

[0095]

[0096] get:

[0097]

[0098] in:

[0099]

[0100] E is the essential matrix, t is the translation matrix, and R is the rotation matrix.

[0101] Use the 8-point method to find E:

[0102]

[0103] Among them, (u1 , v 1 ) is p 1 The image pixel coordinates, (u 2 , v 2 ) is p 2 The image pixel coordinates.

[0104] get:

[0105]

[0106] in:

[0107]

[0108] The same representation method is used for other point pairs. Putting all the equations together, we get a linear system of equations, (u i , v i ) represents the i-th matching point pair.

[0109]

[0110] The essential matrix E is obtained through the above linear equations.

[0111] Decomposing the singular value of E, we get 4 sets of t and R values, which are:

[0112]

[0113] Among the four groups of results, only one depth value is positive. The combination of t and R values ​​with positive depth value is the translation matrix and rotation matrix of the B frame image relative to the A frame image.

[0114] It is understandable that the above process is described by taking the eight-point method as an example, but is not limited to this. When there are more than eight pairs of matching feature points on the two frames A and B, the least squares method can be constructed using the epipolar constraint to obtain the translation matrix and rotation matrix between the two frames. The least squares method is a relatively mature existing technology, and its specific implementation process will not be described here.

[0115] In addition, after the rotation matrix R and the translation matrix t between each frame image are obtained using the first feature point set, the three-dimensional spatial coordinates (ie, the 3D position of the feature point) of each feature point in the first feature point set can be calculated by triangulation.

[0116] 160. Obtain a next frame of image captured by the monocular driving recorder, and extract feature points of the next frame of image after performing partition processing on the next frame of image.

[0117] In the embodiment of the present application, after constructing the three-dimensional space coordinates according to the above-mentioned at least two frames of pictures, the next frame of pictures collected by the monocular driving recorder can be obtained in real time, or the next frame of pictures located after the above-mentioned at least two frames of pictures can be obtained from the picture sequence, which is not limited here. After the next frame of pictures is partitioned, the feature points of the next frame of pictures can be extracted using the brisk algorithm, and the extracted feature points can be described to obtain a feature descriptor.

[0118] 170. Determine the position and posture of the monocular driving recorder when taking the next frame of image according to the feature points of the next frame of image and the three-dimensional space coordinates of the first feature point set.

[0119] In an optional implementation, the specific implementation of step 170 determining the position and posture of the monocular driving recorder when shooting the next frame of picture according to the feature points of the next frame of picture and the three-dimensional space coordinates of the first feature point set may include the following steps:

[0120] 17) Matching the next frame of the image with each of the at least two frames of the image, and obtaining a set of feature points that successfully matches the next frame of the image with each frame of the image;

[0121] 18) According to the feature point set that successfully matches the next frame of picture with each frame of picture, feature points in the next frame of picture that successfully matches at least a preset number of frames of pictures in the at least two frames of pictures are determined as a second feature point set;

[0122] 19) Determining the three-dimensional space coordinates of the second feature point set according to the three-dimensional space coordinates of the first feature point set;

[0123] 20) Determine the position and posture of the monocular driving recorder when taking the next frame of image by using the three-dimensional spatial coordinates of the second feature point set and the positions of the feature points in the second feature point set located on the next frame of image.

[0124] Take two frames of pictures as windows for example. Assume that the above at least two frames of pictures include two frames of pictures A and B, and the next frame of picture is frame C. There are 100 feature points in frame A and 200 feature points in frame B. There are 50 feature points that are successfully matched between frame A and frame B, that is, the first feature point set contains 50 pairs of points. 200 feature points are extracted from frame C, 70 of which are successfully matched with feature points in frame A and 60 are successfully matched with feature points in frame B. The feature points in frame C that are successfully matched with both feature points in frame A and frame B are classified into the second feature point set. For example, the feature point numbered c1 on frame C matches the feature point numbered a3 on frame A and the feature point numbered b2 on frame B. Then the feature point c1 is a valid feature point, and the feature point (a3, b2, c1) is one of the feature points in the second feature point set. When the feature point numbered c1 on the C frame image only matches the feature point numbered a3 on the A frame image, and no matching feature point is found in the B frame image, the feature point c1 is an invalid feature point (or noise point) and will not be included in the second feature point set. According to the above method, feature points that match in all three frames can be found to form the second feature point set.

[0125] Assume that among the 70 feature points that successfully match the C frame image and the A frame image, and the 60 feature points that successfully match the C frame image and the B frame image, there are 30 feature points that exist in all three frames, and these 30 feature points are included in the 50 feature points that successfully match the A and B frames. Therefore, the three-dimensional spatial coordinates of the 30 feature points can be extracted from the three-dimensional spatial coordinates of the 50 feature points. Of course, the three-dimensional spatial coordinates of the 30 feature points can also be directly calculated by triangulation, which is not limited here. Furthermore, based on the three-dimensional spatial coordinates of the 30 feature points and the positions of the 30 feature points on the C frame image (i.e., uv coordinates), the PnP optimization method can be used to calculate the position and posture of the monocular driving recorder when shooting the C frame image.

[0126] Take three frames of pictures as windows for example, assuming that the above at least two frames of pictures include three frames of pictures A, B, and C, and the next frame of picture is frame D. There are 100 feature points in frame A, 200 feature points in frame B, and 150 feature points in frame C. There are 50 feature points that are successfully matched in the three frames of pictures A, B, and C, that is, the first feature point set contains 50 pairs of points. 200 feature points are extracted from frame D, 70 of which are successfully matched with feature points in frame A, 60 with feature points in frame B, and 65 with feature points in frame C. The feature points in the D frame image that are successfully matched with at least two of the three frames of images A, B, and C can be included in the second feature point set. For example, if a feature point in the D frame image can find a matching feature point in all three frames of images A, B, and C, or can find a matching feature point in two of the three frames of images A, B, and C, then the feature point can be considered as a valid feature point, and combined with the feature points that successfully match other images as a set of feature points in the second feature point set. If a feature point in the D frame image is only found to match a feature point in one of the three frames of images A, B, and C, then the feature point can be considered to be an invalid feature point (or noise point) and will not be included in the second feature point set. Match one by one according to the above method to find matching feature points that meet the conditions to form the second feature point set. Furthermore, the PnP optimization method is used to calculate the position and posture of the monocular driving recorder when shooting the D frame image through the three-dimensional spatial coordinates of the second feature point set and its position in the D frame image.

[0127] In practical applications, other numbers of frame images may be used as reference windows, such as 4 frames, 5 frames, 6 frames or other values. When the number of windows is different, the preset number in step 18) also changes accordingly. For example, when the window takes 4 frames of images, the preset number may be set to 2, 3 or 4; when the window takes 5 frames of images, the preset number may be set to 3, 4 or 5.

[0128] 180. Determine the position of the vehicle according to the posture of the monocular driving recorder when taking the next frame of the picture.

[0129] In the embodiment of the present application, since the monocular driving recorder is set on the vehicle, the position of the monocular driving recorder when taking a certain frame of picture can be regarded as the position of the vehicle at that time, so that the position of the vehicle can be obtained, that is, the positioning of the vehicle is realized. Of course, there can also be a preset position relationship between the monocular driving recorder and the vehicle, and the position of the monocular driving recorder can be converted according to the position relationship to obtain the position of the vehicle.

[0130] In an alternative embodiment, Figure 1 The described method may further comprise the following steps:

[0131] 21) using the feature point set that successfully matches the next frame of the picture with each frame of the picture, excluding the second feature point set, and calculating the three-dimensional space coordinates of the remaining feature point set by triangulation;

[0132] 22) Using the three-dimensional spatial coordinates of the remaining feature point sets, the three-dimensional spatial coordinates of the first feature point set and the three-dimensional spatial coordinates of the second feature point set are adjusted.

[0133] Still taking the above two frames of pictures as an example of windows, there are 70-30=40 remaining feature points between the C frame picture and the A frame picture, and there are 60-30=30 remaining feature points between the C frame picture and the B frame picture. The three-dimensional spatial coordinates of the above 40 remaining feature points and the 30 remaining feature points are calculated respectively by triangulation, and then the three-dimensional spatial coordinates of the remaining feature points are used to adjust the three-dimensional spatial coordinates of the first feature point set and the three-dimensional spatial coordinates of the second feature point set, so that the three-dimensional spatial range corresponding to the first feature point set and the three-dimensional spatial range corresponding to the second feature point set can be expanded, and a three-dimensional map containing more information is constructed, which is beneficial to the subsequent image registration and improves the registration accuracy.

[0134] In an alternative embodiment, Figure 1 The described method may further comprise the following steps:

[0135] 23) Iteratively process each frame of images subsequently collected by the monocular driving recorder to obtain the position and posture of the monocular driving recorder when taking each frame of images;

[0136] 24) Determine the vehicle's moving trajectory based on the position and posture of the monocular driving recorder when taking each frame of the picture.

[0137] Still taking the above two frames of pictures as windows as an example. The specific process of iterative processing can be: when the next frame (D frame) of the picture is to be registered, the two frames of pictures B and C are selected as reference windows, the three-dimensional spatial coordinates of the first feature point set of the two frames of pictures B and C are constructed, and the feature points of the D frame are matched with the feature points of the two frames of pictures B and C respectively, and the second feature point set that is successfully matched is obtained. The three-dimensional spatial coordinates of the second feature point set and its position on the D frame picture are used to determine the position of the monocular driving recorder when the D frame picture is taken, so as to obtain the position of the vehicle when the D frame picture is taken. When the next frame of pictures (E frame) is to be registered, the two frames of pictures C and D are selected as reference windows, the three-dimensional spatial coordinates of the first feature point set of the two frames of pictures C and D are constructed, and the feature points of the E frame are matched with the feature points of the two frames of pictures C and D respectively, so as to obtain the second feature point set that is successfully matched. The three-dimensional spatial coordinates of the second feature point set and its position on the E frame picture are used to determine the position of the monocular driving recorder when the E frame picture is taken, so as to obtain the position of the vehicle when the E frame picture is taken. The above process is iterated until the last frame of the picture is taken to obtain the position of the vehicle when the last frame of the picture is taken.

[0138] Still taking the above three frames of pictures as windows as an example. The specific process of iterative processing can be: when the next frame (frame E) is to be registered, the three frames of pictures B, C, and D are selected as reference windows, and the three-dimensional spatial coordinates of the first feature point set of the three frames of pictures B, C, and D are constructed, and the feature points of the E frame are matched with the feature points of the three frames of pictures B, C, and D respectively, to obtain the second feature point set that is successfully matched. The three-dimensional spatial coordinates of the second feature point set and its position on the E frame are used to determine the position of the monocular driving recorder when the E frame picture is taken, so as to obtain the position of the vehicle when the E frame picture is taken. When the next frame (frame F) is to be registered, the three frames of pictures C, D, and E are selected as reference windows, and the three-dimensional spatial coordinates of the first feature point set of the three frames of pictures C, D, and E are constructed, and the feature points of the F frame are matched with the feature points of the three frames of pictures C, D, and E respectively, to obtain the second feature point set that is successfully matched. The three-dimensional spatial coordinates of the second feature point set and their positions on the F frame image are used to determine the position of the monocular driving recorder when the F frame image is taken, so as to obtain the position of the vehicle when the F frame image is taken. The above process is iterated backward until the last frame image is taken to obtain the position of the vehicle when the last frame image is taken.

[0139] like Figure 3As shown, the relative movement trajectory of the vehicle can be determined based on the position of the vehicle when each frame of the picture is taken. If the starting position of the vehicle is known when the first few frames of the picture are taken (which can be obtained through the GPS positioning module or Beidou positioning device or IMU on the vehicle), the actual movement trajectory of the vehicle can be determined based on the known starting position. The embodiment of the present application only needs to use GPS or IMU for positioning at the beginning, and it will no longer be needed later. Instead, the position of the vehicle is estimated by using the change in the posture of the monocular driving recorder when collecting different pictures.

[0140] It can be seen that the embodiment of the present application obtains two or more frames of pictures successively collected by the monocular driving recorder during vehicle driving, and partitions each frame of the picture to obtain each frame of the picture after partitioning, and extracts a certain number of feature points from each area of ​​each frame of the picture after partitioning, and regards the set of feature points extracted from each area as the feature points of the frame of the picture, and then matches the feature points of each frame of the picture to obtain a first feature point set that successfully matches, so as to construct three-dimensional space coordinates using the first feature point set; further, the next frame of the picture collected by the monocular driving recorder can be obtained, and the feature points of the next frame of the picture can be extracted after partitioning according to the above method, and the position of the monocular driving recorder when the next frame of the picture is taken is determined according to the feature points of the next frame of the picture and the three-dimensional space coordinates of the first feature point set, and then the position of the vehicle when the next frame of the picture is taken can be obtained, that is, the real-time positioning of the vehicle is realized by using the picture under monocular vision, so that the subsequent collected pictures can obtain the moving trajectory of the vehicle by continuously updating the positioning; and the extraction of features after partitioning the picture can make the feature distribution uniform, which is conducive to the establishment of three-dimensional space, and thus helps to improve the construction accuracy of three-dimensional space.

[0141] See also Figure 4 , the embodiment of the present application provides a device for implementing SLAM positioning based on a monocular driving recorder. The device can be used to execute the method for implementing SLAM positioning based on a monocular driving recorder provided in the above embodiment. Specifically, Figure 4 As shown, the device may include:

[0142] An acquisition unit 41 is used to acquire at least two frames of images collected by a monocular driving recorder during the driving of the vehicle;

[0143] A partitioning unit 42 is used to perform partitioning processing on each of the at least two frames of pictures to obtain each frame of pictures after partitioning;

[0144] An extraction unit 43 is used to extract a preset number of feature points from each region in each frame of the partitioned image, and use the set of feature points extracted from each region as feature points of the frame of the image;

[0145] A matching unit 44 is used to match the feature points of the at least two frames of images to obtain a first feature point set that successfully matches the at least two frames of images;

[0146] A construction unit 45, configured to construct the three-dimensional space coordinates of the first feature point set;

[0147] The acquisition unit 41 is also used to acquire the next frame of the image captured by the monocular driving recorder;

[0148] The partitioning unit 42 is further used to perform partitioning processing on the next frame of the picture;

[0149] The extraction unit 43 is further used to extract feature points of the next frame of the image after partitioning;

[0150] A determination unit 46, configured to determine the position and posture of the monocular driving recorder when taking the next frame of picture according to the feature points of the next frame of picture and the three-dimensional space coordinates of the first feature point set;

[0151] The determination unit 46 is further configured to determine the position of the vehicle according to the position and posture of the monocular driving recorder when taking the next frame of picture.

[0152] Optionally, the partitioning unit 42 performs partitioning processing on each of the at least two frames of pictures to obtain each frame of the picture after partitioning. The specific implementation method may be:

[0153] Partitioning each of the at least two frames of images according to a preset rule to obtain each frame of images after partitioning;

[0154] Alternatively, obtain image information of each of the at least two frames of images, determine a partitioning strategy for each frame of images based on the image information of each frame of images, and perform partitioning processing on each frame of images according to the partitioning strategy of each frame of images to obtain each frame of images after partitioning, wherein the image information may include but is not limited to at least one of the pixel value, grayscale value, and RGB value of the image.

[0155] Optionally, the extraction unit 43 extracts a preset number of feature points from each region in each frame of the partitioned image, and uses the set of feature points extracted from each region as the feature points of the frame of the image. The specific implementation method may be:

[0156] A preset number of feature points are extracted from each region in each frame of the partitioned image using the brisk operator, and the feature points extracted from each region are described, and the set of feature points of each region after the description is used as the feature points of the frame of the image;

[0157] Accordingly, the matching unit 44 may be specifically configured to match the feature points described by the at least two frames of images, and determine the feature points whose matching distance is less than a preset value as a first feature point set with successful matching.

[0158] Optionally, the construction unit 45 can be specifically used to use the first feature point set and adopt the epipolar constraint to calculate the rotation matrix and translation matrix between the above-mentioned at least two frames of pictures; based on the rotation matrix and translation matrix between the above-mentioned at least two frames of pictures, generate the three-dimensional spatial coordinates of the first feature point set.

[0159] Optional, Figure 4 The device shown may also include:

[0160] An iteration unit, used for iteratively processing each frame of images subsequently collected by the monocular driving recorder, to obtain the position and posture of the monocular driving recorder when taking each frame of images;

[0161] The determination unit 46 may also be used to determine the moving trajectory of the vehicle according to the position and posture of the monocular driving recorder when taking each frame of the picture.

[0162] Optionally, the specific implementation of the determination unit 46 determining the position and posture of the monocular driving recorder when shooting the next frame of the picture according to the feature points of the next frame of the picture and the three-dimensional space coordinates of the first feature point set may be:

[0163] Matching the next frame of the picture with each of the at least two frames of the picture, and obtaining a set of feature points that successfully match the next frame of the picture with each frame of the picture;

[0164] According to the feature point set that the next frame of picture successfully matches with each frame of picture, feature points in the next frame of picture that successfully match with at least a preset number of frames of pictures in the at least two frames of pictures are determined as a second feature point set;

[0165] Determine the three-dimensional space coordinates of the second feature point set according to the three-dimensional space coordinates of the first feature point set;

[0166] The position and posture of the monocular driving recorder when shooting the next frame of the picture are determined by using the three-dimensional spatial coordinates of the second feature point set and the positions of the feature points in the second feature point set located on the next frame of the picture.

[0167] Optional, Figure 4 The device shown may also include:

[0168] A calculation unit, used to use the feature point set that successfully matches the next frame of the picture with each frame of the picture to remove the second feature point set from the remaining feature point set, and to calculate the three-dimensional space coordinates of the remaining feature point set by triangulation;

[0169] The adjustment unit is used to adjust the three-dimensional space coordinates of the first feature point set and the three-dimensional space coordinates of the second feature point set using the three-dimensional space coordinates of the remaining feature point set.

[0170] Implementation Figure 4 The device shown can use images under monocular vision to achieve real-time positioning of the vehicle, so that the subsequent collected images can obtain the vehicle's movement trajectory by continuously updating the positioning; and partitioning the image and then extracting features can make the features evenly distributed, which is conducive to the establishment of three-dimensional space, and thus helps to improve the construction accuracy of three-dimensional space.

[0171] Regarding the device in the above embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0172] See also Figure 5 , the embodiment of the present application also provides another device for implementing SLAM positioning based on a monocular driving recorder. The device can be used to execute the method for implementing SLAM positioning based on a monocular driving recorder provided in the above embodiment. The device can be any device with a computing unit, such as a computer, a server, a handheld device (such as a smart phone, a tablet computer, etc.), or a driving recorder, etc., and the embodiment of the present application is not limited. Specifically, Figure 5 As shown, the device 500 may include: at least one processor 501, a memory 502, at least one communication interface 503 and other components. Among them, these components can be connected to each other through one or more communication buses 504. Those skilled in the art will understand that Figure 5 The structure of the device 500 shown in the figure does not constitute a limitation on the embodiments of the present application, and it can be a bus structure or a star structure, and can also include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0173] The processor 501 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0174] The memory 502 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 501 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 502 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 502 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0175] The communication interface 503 may include a wired communication interface, a wireless communication interface, etc., and may be used to communicate and interact with the driving recorder, such as obtaining video images captured by the driving recorder.

[0176] The memory 502 stores executable codes. When the executable codes are processed by the processor 501 , the processor 501 can execute part or all of the steps in the above-mentioned method.

[0177] Specifically, the processor 501 may be used to call one or more executable codes stored in the memory 502 to perform the following operations:

[0178] Obtain at least two frames of images captured by a monocular driving recorder during vehicle driving;

[0179] Partitioning each of the at least two frames of images to obtain each frame of images after partitioning;

[0180] Extract a preset number of feature points from each region in each frame of the partitioned image, and use the set of feature points extracted from each region as the feature points of the frame of the image;

[0181] Matching the feature points of the at least two frames of images to obtain a first feature point set that successfully matches the at least two frames of images;

[0182] Constructing the three-dimensional space coordinates of the first feature point set;

[0183] Obtain the next frame of image captured by the monocular driving recorder, and extract feature points of the next frame of image after performing partition processing on the next frame of image;

[0184] Determine the position and posture of the monocular driving recorder when taking the next frame of picture according to the feature points of the next frame of picture and the three-dimensional space coordinates of the first feature point set;

[0185] The position of the vehicle is determined according to the posture of the monocular driving recorder when the next frame of the picture is taken.

[0186] Optionally, the processor 501 performs partition processing on each of the at least two frames of pictures, and a specific implementation method of obtaining each frame of pictures after partitioning may be:

[0187] Partitioning each of the at least two frames of images according to a preset rule to obtain each frame of images after partitioning;

[0188] Alternatively, obtain image information of each of the at least two frames of images, determine a partitioning strategy for each frame of images based on the image information of each frame of images, and perform partitioning processing on each frame of images according to the partitioning strategy of each frame of images to obtain each frame of images after partitioning, wherein the image information may include but is not limited to at least one of the pixel value, grayscale value, and RGB value of the image.

[0189] Optionally, the processor 501 extracts a preset number of feature points from each region in each frame of the partitioned image, and uses the set of feature points extracted from each region as the feature points of the frame of the image. A specific implementation method may be:

[0190] A preset number of feature points are extracted from each region in each frame of the partitioned image using the brisk operator, and the feature points extracted from each region are described, and the set of feature points of each region after the description is used as the feature points of the frame of the image;

[0191] The specific implementation of the processor 501 matching the feature points of the at least two frames of images to obtain the first feature point set that successfully matches the at least two frames of images may be:

[0192] The feature points described by the at least two frames of images are matched, and the feature points whose matching distance is less than a preset value are determined as a first feature point set with successful matching.

[0193] Optionally, the specific implementation of the processor 501 constructing the three-dimensional space coordinates of the first feature point set may be:

[0194] Using the first feature point set and adopting epipolar constraints, a rotation matrix and a translation matrix between the at least two frames of images are calculated;

[0195] The three-dimensional space coordinates of the first feature point set are generated according to the rotation matrix and the translation matrix between the at least two frames of images.

[0196] Optionally, the processor 501 may also call one or more executable codes stored in the memory 502 to perform the following operations:

[0197] Iteratively process each frame of images subsequently collected by the monocular driving recorder to obtain the position and posture of the monocular driving recorder when taking each frame of images;

[0198] The moving trajectory of the vehicle is determined according to the posture of the monocular driving recorder when taking each frame of the picture.

[0199] Optionally, the specific implementation of the processor 501 determining the position and posture of the monocular driving recorder when shooting the next frame of picture according to the feature points of the next frame of picture and the three-dimensional space coordinates of the first feature point set may be:

[0200] Matching the next frame of the picture with each of the at least two frames of the picture, and obtaining a set of feature points that successfully match the next frame of the picture with each frame of the picture;

[0201] According to the feature point set that the next frame of picture successfully matches with each frame of picture, feature points in the next frame of picture that successfully match with at least a preset number of frames of pictures in the at least two frames of pictures are determined as a second feature point set;

[0202] Determine the three-dimensional space coordinates of the second feature point set according to the three-dimensional space coordinates of the first feature point set;

[0203] The position and posture of the monocular driving recorder when shooting the next frame of the picture are determined by using the three-dimensional spatial coordinates of the second feature point set and the positions of the feature points in the second feature point set located on the next frame of the picture.

[0204] Optionally, the processor 501 may also call one or more executable codes stored in the memory 502 to perform the following operations:

[0205] Using the feature point set that successfully matches the next frame of the picture with each frame of the picture, the remaining feature point set after removing the second feature point set is removed, and triangulating to calculate the three-dimensional space coordinates of the remaining feature point set;

[0206] The three-dimensional space coordinates of the remaining feature point set are used to adjust the three-dimensional space coordinates of the first feature point set and the three-dimensional space coordinates of the second feature point set.

[0207] The scheme of the present application has been described in detail above with reference to the accompanying drawings. In the above embodiments, the description of each embodiment has its own emphasis. For the part that is not described in detail in a certain embodiment, refer to the relevant description of other embodiments. Those skilled in the art should also know that the actions and modules involved in the description are not necessarily required for the present application. In addition, it can be understood that the steps in the method of the embodiment of the present application can be adjusted in order, merged and deleted according to actual needs, and the modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0208] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0209] Alternatively, the present application can also be implemented as a non-temporary machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) on which executable code (or computer program, or computer instruction code) is stored. When the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device, the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0210] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the application herein may be implemented as electronic hardware, computer software, or combinations of both.

[0211] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system and method according to multiple embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a part of a module, a program segment or a code, and the part of the module, the program segment or the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or the flow chart, and the combination of the square boxes in the block diagram and / or the flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of special hardware and computer instructions.

[0212] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for realizing SLAM positioning based on a monocular driving recorder, characterized in that: include: Acquire at least two frames of pictures collected by the monocular driving recorder during the vehicle driving process; Performing partition processing on each of the at least two frames of pictures to obtain each frame of pictures after partitioning; Extract a preset number of feature points from each region in each frame of the partitioned image, and use the set of feature points extracted from each region as the feature points of the frame of the image; Matching the feature points of the at least two frames of images to obtain a first feature point set that successfully matches the at least two frames of images; Constructing the three-dimensional space coordinates of the first feature point set; Obtaining a next frame of image captured by the monocular driving recorder, and extracting feature points of the next frame of image after performing partition processing on the next frame of image; Determine the position and posture of the monocular driving recorder when taking the next frame of picture according to the feature points of the next frame of picture and the three-dimensional space coordinates of the first feature point set; The method includes: matching the next frame of the picture with each of the at least two frames of the picture, and obtaining a set of feature points where the next frame of the picture successfully matches each of the frames of the picture; According to the feature point set that successfully matches the next frame of the picture with each frame of the picture, the feature points in the next frame of the picture that successfully match at least a preset number of frames of the at least two frames of the picture are determined as a second feature point set; the three-dimensional spatial coordinates of the second feature point set are determined according to the three-dimensional spatial coordinates of the first feature point set; the position and posture of the monocular driving recorder when shooting the next frame of the picture are determined by using the three-dimensional spatial coordinates of the second feature point set and the positions of the feature points in the second feature point set located on the next frame of the picture; The position of the vehicle is determined according to the posture of the monocular driving recorder when the next frame of picture is taken.

2. The method for realizing SLAM positioning based on a monocular driving recorder according to claim 1, characterized in that: The partitioning process is performed on each of the at least two frames of pictures to obtain each frame of pictures after partitioning, including: Partitioning each of the at least two frames of images according to a preset rule to obtain each frame of images after partitioning; Alternatively, obtain picture information of each frame of the at least two frames of pictures, determine a partition strategy for each frame of pictures according to the picture information of each frame of pictures, and partition each frame of pictures according to the partition strategy of each frame of pictures to obtain each frame of pictures after partitioning, wherein the picture information includes at least one of a pixel value, a grayscale value, and an RGB value of the picture.

3. The method for realizing SLAM positioning based on a monocular driving recorder according to claim 1, characterized in that: The step of extracting a preset number of feature points from each region in each frame of the partitioned image, and using the set of feature points extracted from each region as feature points of the frame of the image, includes: A preset number of feature points are extracted from each region in each frame of the partitioned image using the brisk operator, and the feature points extracted from each region are described, and the set of feature points of each region after the description is used as the feature points of the frame of the image; The step of matching the feature points of the at least two frames of images to obtain a first feature point set that successfully matches the at least two frames of images includes: The feature points described by the at least two frames of images are matched, and the feature points whose matching distance is less than a preset value are determined as a first feature point set with successful matching.

4. The method for realizing SLAM positioning based on a monocular driving recorder according to claim 1, characterized in that: The constructing the three-dimensional space coordinates of the first feature point set includes: Using the first feature point set and adopting epipolar constraints to calculate a rotation matrix and a translation matrix between the at least two frames of images; Generate three-dimensional spatial coordinates of the first feature point set according to the rotation matrix and the translation matrix between the at least two frames of pictures.

5. The method for realizing SLAM positioning based on a monocular driving recorder according to claim 1, characterized in that: The method further comprises: Iteratively process each frame of the image subsequently collected by the monocular driving recorder to obtain the position and posture of the monocular driving recorder when taking each frame of the image; The moving trajectory of the vehicle is determined according to the position and posture of the monocular driving recorder when taking each frame of the picture.

6. The method for realizing SLAM positioning based on a monocular driving recorder according to claim 1, characterized in that: The method further comprises: Using the feature point set that successfully matches the next frame of the picture with each frame of the picture, the remaining feature point set after removing the second feature point set is removed, and triangulating the three-dimensional space coordinates of the remaining feature point set; The three-dimensional space coordinates of the first feature point set and the three-dimensional space coordinates of the second feature point set are adjusted using the three-dimensional space coordinates of the remaining feature point set.

7. A device for realizing SLAM positioning based on a monocular driving recorder, characterized in that: include: An acquisition unit, used to acquire at least two frames of pictures collected by the monocular driving recorder during the driving of the vehicle; A partitioning unit, used to perform partitioning processing on each of the at least two frames of pictures to obtain each frame of pictures after partitioning; An extraction unit, used to extract a preset number of feature points from each region in each frame of the partitioned image, and use the set of feature points extracted from each region as feature points of the frame of the image; A matching unit, configured to match the feature points of the at least two frames of images to obtain a first feature point set that successfully matches the at least two frames of images; A construction unit, used to construct the three-dimensional space coordinates of the first feature point set; The acquisition unit is further used to acquire the next frame of image collected by the monocular driving recorder; The partitioning unit is further used to perform partitioning processing on the next frame of the picture; The extraction unit is further used to extract feature points of the next frame of the image after partitioning; A determination unit, configured to determine the position and posture of the monocular driving recorder when shooting the next frame of picture according to the feature points of the next frame of picture and the three-dimensional spatial coordinates of the first feature point set; wherein the implementation method of the determination unit is: matching the next frame of picture with each of the at least two frames of pictures, and obtaining a feature point set where the next frame of picture successfully matches each frame of picture; According to the feature point set that successfully matches the next frame of the picture with each frame of the picture, the feature points in the next frame of the picture that successfully match at least a preset number of frames of the above at least two frames of the picture are determined as a second feature point set; the three-dimensional spatial coordinates of the second feature point set are determined according to the three-dimensional spatial coordinates of the first feature point set; and the position of the monocular driving recorder when shooting the next frame of the picture is determined by using the three-dimensional spatial coordinates of the second feature point set and the positions of the feature points in the second feature point set located on the next frame of the picture; The determination unit is further used to determine the position of the vehicle according to the posture of the monocular driving recorder when taking the next frame of picture.

8. A device for realizing SLAM positioning based on a monocular driving recorder, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 6.

9. A non-transitory machine-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 6.

Images