An image feature point extraction method and device, a visual odometer and a mobile tool

Abstract

The application provides an image feature point extraction method, device, visual odometer and mobile tool. The image feature point extraction method comprises: acquiring an image frame; extracting first feature points of the image frame by using a feature extraction network; performing instance segmentation on the image frame by using an instance segmentation network to obtain an original mask, and obtaining a prior dynamic object region and a prior static object region based on the original mask; eliminating the first feature points in the prior dynamic object region; continuing to extract a set number of second feature points in the prior static object region by using the feature extraction network, and the number of the second feature points is not less than the number of the eliminated first feature points; and taking the second feature points and the remaining first feature points as feature points of the image frame. After the first feature points in the prior dynamic region are eliminated, the set number of second feature points are extracted in the prior static object region, so that a sufficient number of high-quality feature points are transmitted into a SLAM back end, and the positioning accuracy of the visual odometer in a dynamic environment is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image feature point extraction method and device, a visual odometry and a mobile tool. BACKGROUND

[0002] An odometry refers to self-positioning of a mobile robot by using a sensor carried by the mobile robot, thereby providing a guarantee for implementation of various complex tasks. Among various types of sensors, a camera has a wider application range due to low cost, simple structure and larger amount of information collected. At present, many products are positioned by using visual odometry technology, such as unmanned vehicles, unmanned aerial vehicles and household sweeping robots.

[0003] There are many frameworks of visual odometry, but most of them are constructed on the premise of static environment assumption. The most classic one is the ORB-SLAM2 algorithm, which is a system integrating various visual sensor devices, and can calculate the self-position only by using a visual sensor, and is extremely widely applied. However, in the actual environment, there is inevitably interference of dynamic objects, but the algorithm does not fully consider the influence of dynamic objects on the positioning accuracy of the system, resulting in that the positioning accuracy of the method is obviously poor in an indoor environment with multiple dynamic objects, and the positioning error sharply increases.

[0004] In view of the above problems, a mobile robot visual odometry design method in a dynamic environment is disclosed in Chinese Patent No. CN114612494A. The method uses an instance segmentation network YOLACT to extract semantic information of objects and generate a semantic binary mask, and combines image semantic information and an LK optical flow method to detect motion consistency to determine moving objects, so as to eliminate dynamic feature points. The patent mainly uses semantic information and an LK optical flow method to detect motion consistency and eliminate dynamic feature points. However, the method requires that feature points are completely uniformly distributed, and static region feature points need to account for a certain proportion of all feature points. Although the traditional ORB feature points (ORB is an abbreviation of Oriented FAST and Rotated BRIEF, including feature points and descriptors) have a certain dispersion, under the influence of light and other factors in an indoor environment, it is difficult to avoid dense distribution of feature points, which may result in that dynamic feature points account for a majority, and the region distribution is more extensive, and finally there are not enough static feature points to execute a RANSAC method and estimate a camera pose in a subsequent visual odometry, thereby causing the method to have decreased or even failed visual odometry accuracy in a dynamic environment. SUMMARY

[0005] The present application aims to at least solve the problems of uneven feature point extraction in a dynamic environment and insufficient feature points after removing dynamic feature points in the prior art, and the problem of insufficient positioning accuracy of visual odometry in a dynamic environment caused by uneven feature points and insufficient feature points, and provides an image feature point extraction method, device, visual odometry and mobile tool.

[0006] In order to achieve the above-mentioned purpose of the present application, according to a first aspect of the present application, the present application provides an image feature point extraction method, comprising: acquiring an image frame; extracting first feature points of the image frame by using a pre-trained feature extraction network; performing instance segmentation on the image frame by using a pre-trained instance segmentation network to obtain an original mask, and obtaining a prior dynamic object region and a prior static object region of the image frame based on the original mask; removing the first feature points in the prior dynamic object region of the image frame; continuing to extract a second feature point in the prior static object region of the image frame by using the feature extraction network, and the number of the second feature points is not less than the number of the removed first feature points; and taking the second feature points and the remaining first feature points as the feature points of the image frame.

[0007] In order to achieve the above-mentioned purpose of the present application, according to a second aspect of the present application, the present application provides an image feature point extraction device, comprising: an acquisition module, which acquires an image frame; a first feature point extraction module, which extracts first feature points of the image frame by using a pre-trained feature extraction network; a region division module, which performs instance segmentation on the image frame by using a pre-trained instance segmentation network to obtain an original mask, and obtains a prior dynamic object region and a prior static object region of the image frame based on the original mask; a first feature point removal module, which removes the first feature points in the prior dynamic object region of the image frame; and a second feature point extraction module, which continues to extract a second feature point in the prior static object region of the image frame by using the feature extraction network, and the number of the second feature points is not less than the number of the removed first feature points, and takes the second feature points and the remaining first feature points as the feature points of the image frame.

[0008] In order to achieve the above-mentioned purpose of the present application, according to a third aspect of the present application, the present application provides a visual odometry, comprising: a camera, which is used for shooting driving images of a mobile tool; the image feature point extraction device provided by the second aspect of the present application, which is used for extracting feature points in image frames output by the camera; a RANSAC algorithm module, which obtains an optimal basis matrix based on the feature points extracted by the image feature point extraction device, and obtains a preliminary pose based on the optimal basis matrix; a key frame detection module, which selects key frames; and a SLAM backend optimization module, which completes trajectory optimization and loop detection based on the key frames, the optimized basis matrix and the preliminary pose, and generates a camera motion trajectory curve.

[0009] To achieve the above object of the present application, according to a fourth aspect of the present application, the present application provides a mobile tool comprising a tool body and a visual odometer provided by the third aspect of the present application mounted on the tool body.

[0010] The present application fully utilizes the feature extraction network (such as GCNv2) to extract uniform first feature points in the front end, fully utilizes the semantic information of the instance segmentation network (such as SOLOv2), and after eliminating the first feature points in the prior dynamic region, the feature extraction network is used again to extract a certain number of second feature points in the prior static object region of the image frame, and the second feature points and the remaining first feature points are used as the feature points of the image frame, which ensures that a sufficient number of high-quality feature points are transmitted to the SLAM back end for optimization calculation, thereby improving the positioning accuracy of the visual odometer in a dynamic environment. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a flowchart of an image feature point extraction method in an embodiment of the present application;

[0012] Figure 2 is a schematic diagram of a feature point re-projection process in an embodiment of the present application;

[0013] Figure 3 is a whole flowchart of a visual odometer in an embodiment of the present application;

[0014] Figure 4 is a comparison diagram of the traditional ORB feature extraction algorithm and the GCNv2 extracted ORB feature points. DETAILED DESCRIPTION

[0015] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application.

[0016] In the description of the present application, it should be understood that the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0017] In the description of the present application, unless otherwise specified and limited, it is necessary to explain that the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be mechanical connection or electrical connection, it can be the communication inside two elements, it can be direct connection or indirect connection through intermediate medium, and the specific meaning of the above terms can be understood by those skilled in the art according to the specific circumstances.

[0018] The present application discloses a kind of image feature point extraction method, in one embodiment, the flow chart of image feature point extraction method is as shown in Figure 1 As shown, comprising:

[0019] Step S1, obtain image frame. Image frame is the video image that camera shoots, preferably but not limited to RGB image.

[0020] Step S2, the first feature point of image frame is extracted using pre-trained feature extraction network. Feature extraction network can use GCN or GCNv2. Let the number of first feature points extracted be N.

[0021] Step S3, the original mask is obtained by using pre-trained instance segmentation network to perform instance segmentation on image frame, and the prior dynamic object region and prior static object region of image frame are obtained based on the original mask.

[0022] Image frame has at least one or more prior static object regions. Specifically, the original mask is a binary mask mask of each object surface. The pixel with value 1 in the binary mask mask is the pixel of the prior dynamic object, and the pixel with value 0 is the pixel of the prior static object. The region composed of the pixels with value 1 in the binary mask mask corresponding to the image frame is regarded as the prior dynamic object region, and the region composed of the pixels with value 0 in the binary mask mask corresponding to the image frame is regarded as the prior static object region.

[0023] Let the image frame contain i prior static object regions, i.e. contain i prior static objects, the prior static object set is Φ={O1, O2,...O i}O i represents the tth prior static object, and the region set occupied by the prior static object in the image frame is represented as Λ={S1, S2,...,S i}S i represents the region occupied by the ith prior static object in the image frame.

[0024] Step S4, remove the first feature points in the prior dynamic object region of image frame. Preferably, record the number of first feature points proposed in image frame, and use the number as the basis for the number of second feature points supplemented subsequently.

[0025] Step S5, continue to extract a set number of second feature points in the prior static object region of the image frame using the feature extraction network, and the number of second feature points is not less than the number of removed first feature points.

[0026] Step S6, take the second feature points and the remaining first feature points as the feature points of the image frame. The remaining first feature points represent the first feature point set in all prior static object regions in the image frame.

[0027] In the embodiment, it should be noted that steps S2 and S3 can be executed synchronously, or step S2 can be executed first and then step S3, or step S3 can be executed first and then step S2.

[0028] In the embodiment, preferably, the feature extraction network adopts GCNv2, which can extract uniform ORB feature points and is beneficial to improve the positioning accuracy of the visual odometry backend module in a dynamic environment. Figure 4 A comparison diagram of the traditional ORB feature extraction algorithm and the GCNv2 extracted ORB feature points is shown, Figure 4 The left side is a distribution diagram of the ORB feature points extracted by the traditional ORB feature extraction algorithm, Figure 4 The right side is a distribution diagram of the ORB feature points extracted by GCNv2, and it can be obviously seen that the ORB feature points on the right side are more uniform. Preferably, a preset N first feature points are extracted in step S2. The camera video image is input, and each frame of image information is input into the feature extraction network GCNv2 for operation:

[0029] Define f as a dense feature mapping under low resolution, and o as a pixel original resolution. When a two-dimensional coordinate x=(u x , v y ) of an image plane is given as the input of f(·) and o(·), the corresponding feature descriptor and key point confidence are returned. The descriptor is obtained by bilinear sampling, and f cur and f tar represent the feature maps of the current frame and the target frame respectively, L feat and L det represent the loss of the training descriptor and the detector. The features are binarized

[0030] Forward:

[0031] Backward:

[0032] Wherein, b is the binarized version of the feature f, |f|≤1 is to eliminate the gradient of a single feature response whose absolute value is greater than 1, and the size of b is set to 256. Pixel metric learning is used to train the nearest neighbor descriptor:

[0033]

[0034] d(x cur , x tar ) = ||b cur (x cur )-b tar (x tar )||2 (2)

[0035] where m is the truncated distance margin and J(·, ·) is the square of Hamming distance of binary features. is the matched pair obtained using the real trajectory of camera pose in the training data.

[0036]

[0037] where R gt is the ground truth rotation matrix and t gt is the ground truth translation vector. π projects a pixel from the image plane to 3D space using the given 2D coordinate and depth z i is the retrieved non-matched pair.

[0038] The keypoint detection is considered as a binary classification problem, the goal of the probability map o from the network is a mask composed of 1 and 0, which indicates whether a pixel is a keypoint or not. The weighted cross-entropy is used as the objective function for training, the loss is always evaluated on two consecutive frames to enhance the consistency of the extracted keypoint, the loss of detection can be written as

[0039]

[0040]

[0041] where α1 and α2 are used to handle the unbalanced classes and prevent non-keypoint pixels from dominating the loss.

[0042] In the embodiment, in order to ensure that a sufficient number of second feature points are extracted, further preferably, the feature point threshold is set to T1 when the feature extraction network extracts the first feature points, and the feature point threshold is set to T2 when the feature extraction network extracts the second feature points, T1>T2. The feature point threshold is a network parameter of the feature extraction network. In the process of extracting the feature points by the feature extraction network, if a pixel point is taken as the center, and the brightness of the pixel point is I p , if the brightness of the second preset number of pixel points (such as 3, 9, 11 or 12) in the first preset number of adjacent pixel points (such as 16 adjacent pixel points) around the pixel point is greater than I p +T or less than I​p If R < T, the pixel point is considered as a feature point. Wherein, T represents a feature point threshold, when the feature extraction network extracts the first feature point, T = T1, when the feature extraction network extracts the second feature point, R = T2.

[0043] In this embodiment, the instance segmentation network can adopt SOLOv1 or SOLOv2 network based on semantic information. Preferably, the instance segmentation network adopts SOLOv2 network, which can segment objects by position, complete the instance segmentation task, and also has real-time performance, which excellently balances accuracy and speed.

[0044] In this embodiment, preferably, the number of the second feature points is equal to the number of the removed first feature points. Assuming that the number of the removed first feature points in step S4 is ndelete, the number of the supplemented second feature points is also ndelete, so that the number of the finally obtained feature points is still N.

[0045] In another embodiment, in order to make the extracted feature points as high-quality feature points as possible and make them more static, in step S5, a certain number of second feature points are continuously extracted in the prior static object region of the image frame by using the feature extraction network, including:

[0046] Step A, priority sorting is performed on the prior static object region of the image frame. The higher the static attribute of the prior static object region, the higher the priority level.

[0047] Step B, according to the order from high to low priority, the second feature points are sequentially extracted in the prior static object region of the image frame by using the feature extraction network, and the second feature point extraction is stopped when the cumulative number of the extracted second feature points is equal to the number of the removed first feature points.

[0048] In this embodiment, the regions corresponding to each object in the prior static object set Φ are sorted according to the priority order. First, in the region with the highest priority, the feature extraction network GCNv2 is called to extract feature points in the prior static region with threshold T2. Assuming that the number of valid feature points that can be extracted in this region is n1, the relative size of n1 and ndelete is judged. If n1 ≥ ndelete, it means that enough second feature points have been extracted in the region with the highest priority, and the second feature point extraction is stopped. Otherwise, with the same threshold, static feature points are continuously extracted in the regions with lower priority, until the cumulative number of the extracted second feature points reaches ndelete. The specific algorithm flow is as follows:

[0049]

[0050]

[0051] In the embodiment, further preferably, only the prior static object region with the number of first feature points contained in the image frame greater than or equal to 7 is prioritized to ensure that the large brightness change of the pixel points in the prior static object region can smoothly extract the second feature points.

[0052] In the embodiment, further preferably, the step A comprises:

[0053] The average epipolar distance of the prior static object region is calculated; the average epipolar distance can accurately evaluate the static property of the prior static object region, and the smaller the average epipolar distance, the higher the static property of the prior static object region.

[0054] The static object regions of the image frame are prioritized according to the size of the average epipolar distance, wherein the smaller the average epipolar distance of the static object region, the higher the priority level.

[0055] In the embodiment, further preferably, the process of calculating the average epipolar distance of the jth prior static object region comprises:

[0056] Step 1, the first feature points of the jth prior static object region in the image frame and the next frame image of the image frame are matched to obtain a plurality of matching point pairs, and the basis matrix F corresponding to the jth prior static object region is obtained based on the plurality of matching point pairs.

[0057] For convenience of distinction, the input image frame is called the current frame, the next frame image of the image frame is called the next frame, the first feature points of the next frame are extracted by using the feature extraction network, the first feature point matching of the current frame and the next frame in the jth prior static object region is performed by using the existing ORB feature point matching algorithm, and it is assumed that the mth first feature point in the jth prior static object region of the current frame is matched with the m'th first feature point in the jth prior static object region of the next frame. The normalized coordinates of the mth first feature point in the current frame are The normalized coordinates of the m'th first feature point in the next frame are

[0058] Step 2, the epipolar line corresponding to the matching point pair of the jth prior static object region in the next frame image of the image frame is calculated based on the basis matrix. Figure 2 The re-projection process of the feature points is shown, F r represents the current frame, F e represents the next frame, F r The feature point x1 on F e is x2, x1 and x2 form a matching point pair, L1 is the epipolar line corresponding to the feature point x1, and L2 is the epipolar line corresponding to the feature point x2.

[0059] Specifically, the distance of the mth first feature point of the jth prior static object region in the current frame to the epipolar line corresponding to the matching point m' in the next frame image is calculated wherein, [X, Y, Z] represents the three-axis vector of the epipolar line l m′ .

[0060] Step 3, the distance of the matching point in the jth prior static object region in the next frame image of the image frame to the corresponding epipolar line is calculated, and is recorded as the epipolar distance. Specifically, the distance (epipolar distance) of the m'th matching point in the next frame to the corresponding epipolar line l m′ is:

[0061]

[0062] Step 4, the average value of the epipolar distances of all matching points in the jth prior static object region in the next frame image of the image frame is calculated, and the average value is taken as the average epipolar distance of the jth prior static object region. Specifically, it is assumed that the jth prior static object region in the next frame also has k first feature points, and the average epipolar distance of the jth prior static object region is k is a positive integer. Wherein, J is a positive integer, and j∈[1, i], i represents the number of prior static object regions in the image frame.

[0063] The application further discloses an image feature point extraction device for realizing the function of the image feature point extraction method provided by the application. In an embodiment, the device comprises: an acquisition module for acquiring an image frame; a first feature point extraction module for extracting first feature points of the image frame by using a pre-trained feature extraction network; a region division module for performing instance segmentation on the image frame by using a pre-trained instance segmentation network to obtain an original mask, and obtaining a prior dynamic object region and a prior static object region of the image frame based on the original mask; a first feature point elimination module for eliminating the first feature points in the prior dynamic object region of the image frame; and a second feature point extraction module for continuing to extract a set number of second feature points in the prior static object region of the image frame by using the feature extraction network, wherein the number of the second feature points is not less than the number of the eliminated first feature points, and the second feature points and the remaining first feature points are taken as feature points of the image frame.

[0064] In the embodiment, the specific execution process of each module of the device can refer to the specific steps in the image feature point extraction method provided by the application, and will not be described here.

[0065] The application further discloses a visual odometer, which, in an embodiment, comprises a camera, an image feature point extraction device provided by the application, a RANSAC algorithm module, a key frame detection module and a SLAM backend optimization module.

[0066] The camera is used for shooting driving images of a mobile tool.

[0067] The image feature point extraction device provided by the application is used for extracting feature points in a camera output image frame.

[0068] The RANSAC algorithm module obtains an optimal fundamental matrix based on the feature points extracted by the image feature point extraction device and obtains a preliminary pose based on the optimal fundamental matrix. The RANSAC algorithm module performs feature matching operation on the input N feature points. First, s points are randomly selected from the plurality of feature points for calculating the optimal fundamental matrix F', and each pair of matching points in all the feature points is traversed according to the F' matrix to determine whether they satisfy the RANSAC model, that is, assuming that the current frame feature point is x1, when the next frame feature point is x2, the re-projection error is defined as:

[0069] e = x2 - F'x1 (7)

[0070] If the error e satisfies the distance threshold t, the point is an inlier and is added to a consistent set S, otherwise, the point is an outlier. If the number of points in S is greater than a feature point number threshold N t , the RANSAC model is re-estimated using the S and the iteration is ended, otherwise, the above process is repeated.

[0071] After M experiments of the RANSAC algorithm module, a maximum number of consistent sets S is selected, all the points in the S are used to re-estimate the RANSAC model, an optimal fundamental matrix is obtained, and a preliminary pose is obtained based on the optimal fundamental matrix.

[0072] RANSAC is the abbreviation of Random Sample Consensus, which is an algorithm for calculating the mathematical model parameters of data and obtaining effective sample data according to a sample data set containing abnormal data. The RANSAC algorithm is often used in computer vision. For example, the matching point problem of a pair of cameras and the calculation of the fundamental matrix are solved in the field of stereo vision.

[0073] The key frame detection module selects a key frame. Specifically, the similarity of each frame of image in a continuous image frame sequence and the adjacent frame is calculated, and if the similarity is less than a preset similarity threshold, the frame of image is selected as a key frame.

[0074] The SLAM backend optimization module completes trajectory optimization and loop detection based on the key frame, the optimized fundamental matrix and the preliminary pose, and generates a camera motion trajectory curve.

[0075] As Figure 3The overall execution flow of the visual odometer is shown, the SLAM backend optimization module includes a key frame insertion unit, a map point selection unit, a local BA optimization unit, a redundant key frame unit, a database query unit, a Sim3 calculation unit, a loop fusion unit and an essential graph optimization unit, the SLAM backend optimization module is prior art, and details are not repeated here.

[0076] The application further provides a mobile tool, comprising a tool body and the visual odometer provided by the application installed on the tool body.

[0077] The tool body is preferably but not limited to an unmanned vehicle, an unmanned aerial vehicle, a sweeping robot or a patrol robot. The camera in the visual odometer is installed on the tool body for shooting tool body driving images, and the functions of other modules in the visual odometer except the camera can be realized by a special processor or the original processor of the tool body.

[0078] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0079] Although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and purposes of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A method for extracting image feature points, characterized in that, include: Acquire image frames; The first feature points of the image frame are extracted using a pre-trained feature extraction network; The image frame is segmented using a pre-trained instance segmentation network to obtain an original mask, and the prior dynamic object region and prior static object region of the image frame are obtained based on the original mask. Remove the first feature point within the prior dynamic object region of the image frame; The feature extraction network is used to extract a set number of second feature points within the prior static object region of the image frame. The number of second feature points is not less than the number of first feature points that were removed. The second feature point and the remaining first feature points are used as feature points of the image frame; Wherein, the feature point threshold is set when the feature extraction network extracts the first feature point. When the feature extraction network extracts the second feature point, the feature point threshold is set to... , ; In the process of feature extraction networks extracting feature points, if a certain pixel is taken as the center, and the brightness of that pixel is set to... If among the first preset number of neighboring pixels of a given pixel, there is a second preset number of pixels with a brightness greater than [a certain value], or less than If the pixel is a feature point, then it is considered to be a feature point. This represents the feature point threshold, used when the feature extraction network extracts the first feature point. When the feature extraction network extracts the second feature point, .

2. The image feature point extraction method as described in claim 1, characterized in that, The number of the second feature points is equal to the number of the first feature points that were removed; And / or, the feature extraction network is a GCNv2 network; And / or, the instance segmentation network is a SOLOv2 network.

3. The image feature point extraction method as described in claim 1 or 2, characterized in that, The step of using a feature extraction network to further extract a predetermined number of second feature points within the prior static object region of the image frame includes: Prioritize the prior static object regions of the image frame. Following the order of priority from high to low, the feature extraction network sequentially extracts the second feature points in the prior static object region of the image frame. The extraction of the second feature points stops when the cumulative number of extracted second feature points equals the number of first feature points removed.

4. The image feature point extraction method as described in claim 3, characterized in that, Priority sorting is performed only on prior static object regions in image frames that contain 7 or more first feature points.

5. The image feature point extraction method as described in claim 4, characterized in that, The prioritization of prior static object regions in the image frame includes: Calculate the average epipolar distance of the prior static object region; The static object regions in the image frame are prioritized according to the magnitude of the average epipolar distance, with the smaller the average epipolar distance of the static object regions, the higher the priority.

6. The image feature point extraction method as described in claim 5, characterized in that, Calculate the first The process of averaging the epipolar distance of a prior static object region includes: For the Each prior static object region is matched with a first feature point in the image frame and the next frame image to obtain multiple matching point pairs. Based on the multiple matching point pairs, the first... The fundamental matrix corresponding to each prior static object region; Based on the fundamental matrix, the matching point pair located in the next frame of the image frame is calculated. The epipolar line corresponding to the matching point of a prior static object region; Calculate the first image in the next frame of the image frame. The distance between a matching point within a prior static object region and its corresponding epipolar line is denoted as the epipolar distance. Calculate the next frame of the image frame. The average epipolar distance of all matching points within a prior static object region, where the average value is the mean epipolar distance; in, It is a positive integer, and , This indicates the number of prior static object regions in an image frame.

7. An image feature point extraction device, characterized in that, include: The acquisition module acquires image frames; The first feature point extraction module extracts the first feature points of the image frame using a pre-trained feature extraction network. The region segmentation module uses a pre-trained instance segmentation network to perform instance segmentation on the image frame to obtain an original mask, and obtains the prior dynamic object region and prior static object region of the image frame based on the original mask. The first feature point removal module removes the first feature points within the prior dynamic object region of the image frame; The second feature point extraction module uses a feature extraction network to extract a set number of second feature points in the prior static object region of the image frame. The number of second feature points is not less than the number of first feature points that are removed. The second feature points and the remaining first feature points are used as feature points of the image frame. Wherein, the feature point threshold is set when the feature extraction network extracts the first feature point. When the feature extraction network extracts the second feature point, the feature point threshold is set to... , ; In the process of feature extraction networks extracting feature points, if a certain pixel is taken as the center, and the brightness of that pixel is set to... If among the first preset number of neighboring pixels of a given pixel, there is a second preset number of pixels with a brightness greater than [a certain value], or less than If the pixel is a feature point, then it is considered to be a feature point. This represents the feature point threshold, used when the feature extraction network extracts the first feature point. When the feature extraction network extracts the second feature point, .

8. A visual odometer, characterized in that, include: A camera used to capture images of mobile vehicles in motion; The image feature point extraction device as described in claim 7 is used to extract feature points in the camera output image frame; The RANSAC algorithm module obtains the optimal basis matrix based on the feature points extracted by the image feature point extraction device, and then obtains the preliminary pose based on the optimal basis matrix. The keyframe detection module selects keyframes. The SLAM backend optimization module performs trajectory optimization and loop closure detection based on keyframes, optimized base matrices, and preliminary pose, generating camera motion trajectory curves.

9. A mobile tool, characterized in that, It includes a tool body and a visual odometer as described in claim 8 mounted on the tool body.

Images