SLAM method based on static feature point extraction

Abstract

The invention discloses an SLAM (Simultaneous Localization and Mapping) method based on static feature point extraction in the technical field of robot positioning and navigation. The method comprises the following steps of: firstly, sharpening and downsampling an original image, constructing an image pyramid, extracting FAST angular points as feature points in each layer, screening out static feature points, and calculating an image moment of the static feature points to determine a main direction and generate a descriptor; according to the method, static feature points are matched among different frames, key frames are determined, a key frame set is updated, camera poses are calculated and updated through the key frame set, the positions of the feature points are recorded, and map construction and optimization are conducted. Map information and the positions of the feature points are refined in the process. According to the invention, the distribution of the extracted static feature points is more uniform, and the accuracy and anti-interference performance of the SLAM system in a complex scene are improved.

Description

Technical Field

[0001] The invention relates to a SLAM method based on static feature point extraction, belonging to the technical field of robot positioning and navigation. Background Art

[0002] Robot navigation technology is the key to robots' autonomous actions. It allows robots to use sensors to perceive the surroundings in unknown environments and determine their specific positions and directions in order to plan paths and make decisions effectively. SLAM (Simultaneous Localization and Mapping) uses visual sensors to capture environmental information and build maps based on it, thereby helping robots or cameras to achieve self-positioning and navigation. This is crucial for intelligent devices such as service robots, driverless cars, and drones, enabling them to better adapt to complex and unknown environments and improve their autonomy and intelligence.

[0003] However, early SLAM systems were usually designed based on the assumption that the robot is in a static environment, while the environment that robots face in real life is often dynamic, with moving objects and constantly changing scenes. The dynamic environment will cause the robot to obtain inaccurate information about its own position, and the inaccuracy of this information will directly affect the robot's decision-making and actions. At the same time, due to the constant changes in the environment, if the robot cannot effectively update its map data, it will not be able to accurately reflect the true state of the current environment. This will not only cause the map information to become outdated, but may also cause errors in path planning, which in turn affects the robot's navigation efficiency and safety. Therefore, studying how to achieve precise positioning and accurate map updates in a dynamic environment is crucial to improving the robot's autonomous navigation capabilities. Summary of the invention

[0004] The purpose of the present invention is to provide a SLAM method based on static feature point extraction, which can solve the problems of inaccurate camera or robot positioning and untimely map updating in a dynamic environment, so as to ensure that the robot can make correct decisions, perform effective path planning, and ensure the efficiency and safety of navigation.

[0005] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions.

[0006] In a first aspect, the present invention provides a SLAM method based on static feature point extraction, comprising: Acquire an original image, and perform sharpening processing on the original image; Downsampling the sharpened image layer by layer to generate multiple hierarchical images with different resolutions, using the hierarchical images to construct an image pyramid, extracting FAST corner points at each layer of the image pyramid, dividing the hierarchical images containing the FAST corner points into multiple grids, and extracting the optimal FAST corner points from each grid as feature points; Screening the extracted feature points, removing dynamic feature points from the feature points, and retaining static feature points from the feature points; A circular area is selected with the static feature point as the center, an image moment is calculated within the circular area, a main direction of the static feature point is determined according to the image moment, and based on the main direction, pixel points within the circular area are compared and encoded to generate a descriptor of the static feature point; Using the generated descriptor, static feature points are matched between different image frames. Based on the matched static feature points, a sliding window is established, and whether the current image frame is a key frame is determined by the number of matched static feature points. If the current image frame is a key frame, the key frame with the earliest residence time in the sliding window is replaced, and the key frame set in the sliding window is updated; The updated key frame set is used to calculate and update the camera pose, and the position of the static feature points is recorded. The map is constructed and optimized through the continuously updated camera pose and the position of the static feature points to refine the map information and the position of the static feature points.

[0007] In combination with the first aspect, further, performing sharpening processing on the original image includes: Use the convolution kernel in the Laplace operator to move pixel by pixel on the original image, perform weighted summation on the neighborhood of each pixel, and calculate the second-order derivative value of each pixel; The second-order derivative value of each pixel is added to the pixel value of the original image to obtain a sharpened image.

[0008] In combination with the first aspect, further, extracting FAST corner points at each layer of the image pyramid includes: Starting from the sharpened image, downsampling is performed in sequence according to a preset scaling ratio to obtain a plurality of hierarchical images with decreasing resolutions; FAST corner points are extracted for each level of image in the image pyramid, and the extracted FAST corner points are distributed to each level of image according to the area of ​​each level of image.

[0009] In combination with the first aspect, further, the extracted FAST corner points are distributed to images at each level according to the areas of the images at each level, including: Adding the areas of the images at each level to obtain the total area of ​​the image pyramid; The total area of ​​the image pyramid is: Among them, S represents the total area of ​​the image pyramid; L represents the length of the original image; W represents the width of the original image; k represents the scaling factor of the image pyramid; C represents the area of ​​the 0th layer image; Divide all the extracted FAST corner points by the total area of ​​the image pyramid to calculate the number of FAST corner points allocated per unit area. The expression for the number of FAST corner points allocated per unit area is: in, represents the number of FAST corner points allocated per unit area; N represents the total number of FAST corner points extracted; The number of FAST corner points is multiplied by the area of ​​each level of image, so as to distribute the extracted FAST corner points to each level of image; The expression of the extracted FAST corner points distributed to each level of image is:

[0010] in, Indicates The number of feature points assigned to the layer; n represents the image pyramid level.

[0011] In combination with the first aspect, further, extracting the best FAST corner point from each grid as a feature point includes: Selecting pixel points p from the images of each level, taking each pixel point p as the center, selecting multiple pixel points on a circle with a certain radius, and comparing the multiple pixel points with the pixel point p respectively, so as to screen out FAST corner points from the multiple pixel points; Divide the images at each level into a plurality of first grids, check whether there is a FAST corner point in each first grid, and if there is no FAST corner point in the first grid, lower the threshold of the FAST corner point detection operator until a FAST corner point is found in the first grid; if there is a FAST corner point in the first grid, split the first grid into four second grids, and continue the splitting process until the total number of split grids is greater than the total number of preset grids, and stop splitting; The best FAST corner points are selected from each split grid as feature points.

[0012] In combination with the first aspect, further, comparing the multiple pixel points with the pixel point p respectively includes: Calculate the brightness difference between the pixel point p and multiple pixel points on the surrounding circle respectively; The brightness difference expression between the pixel point p and multiple pixels on the surrounding circle is:

[0013] Where N represents the brightness difference between the surrounding points and the pixel point p; Represents the pixel values ​​of surrounding points; Indicates the brightness of pixel p; Indicates the set threshold; represents a circle with pixel p as the center; x represents a pixel; According to the brightness difference, multiple pixel points on the surrounding circle are classified. If the brightness of the current pixel point is higher than the brightness of the pixel point p plus the preset brightness threshold t, it is marked as bright; if the brightness of the current pixel point is lower than the brightness of the pixel point p minus the preset brightness threshold t, it is marked as dark; if the brightness of the current pixel point is between the brightness of the pixel point p minus the preset brightness threshold t and the brightness of the pixel point p plus the preset brightness threshold t, it is marked as similar; Count the number of pixels marked as bright and dark in the circular area. If the number of multiple pixels on the surrounding circle is greater than a preset pixel number threshold, the pixel point p is a FAST corner point. The expression for classifying multiple pixel points on the surrounding circle is: in, Represents the value of the pixel on the circle; d represents a dark pixel; s represents a similar pixel; b represents a bright pixel; and t represents 20% of the brightness value of pixel p.

[0014] In combination with the first aspect, further, screening the extracted feature points includes: Using the YOLO algorithm to identify the dynamic object coordinate region in the original image, excluding feature points within the dynamic object coordinate region, and retaining feature points outside the dynamic object coordinate region as first static feature points; Use the YOLO algorithm to perform semantic segmentation again on the feature points in the coordinate area of ​​the dynamic object to obtain dynamic pixel points, remove the dynamic feature points that coincide with the dynamic pixel points, and retain the dynamic feature points that are different from the dynamic pixel points as the second static feature points; The first static feature point and the second static feature point are combined to form a final static feature point set.

[0015] In combination with the first aspect, further, determining the main direction of the static feature point includes: A circular area with a radius of R is constructed with the static feature point as the center, and the first-order image moment along the x-axis and the y-axis in the circular area is calculated; The expressions of the first-order image moments along the x-axis and the y-axis are respectively: in, represents the first-order image moment along the y-axis; represents the first-order image moment along the x-axis; R represents the radius of the circular area; I(x,y) represents the pixel coordinates; x represents the horizontal coordinate of the pixel; y represents the vertical coordinate of the pixel; Calculate the sum of the brightness of all pixels in the circular area, and calculate the centroid of the circular area by combining the first-order image moments along the x-axis and the y-axis; The calculation expression of the centroid is: Where C represents the coordinates of the center of mass; Indicates the horizontal coordinate of the centroid point relative to the feature point; Indicates the ordinate of the centroid relative to the feature point; Represents the sum of the brightness of all pixels in the circular area; Calculating the main direction of the static feature point according to the position of the centroid relative to the position of the static feature point; The expression of the main direction of the static feature point is: = ( ) in, Indicates the main direction of the static feature point.

[0016] In combination with the first aspect, further, the calculating and updating the camera pose includes: If the ratio of the number of matched static feature points T to the total number of detected static feature points S is less than a preset first threshold , and the ratio of the number of matched static feature points T to the total number of landmark points N in the map is also less than the preset second threshold , then the current image frame is a key frame; Adding the key frame to the sliding window, if the key frames in the sliding window exceed a preset maximum number, removing the key frame with the earliest time to keep the number of key frames in the sliding window constant; Calculate and update the camera pose using the continuously updated keyframe information; Combined with the first aspect, further, the update expression of the camera pose is:

[0017] in, Represents the first key frame in the sliding window; represents the second key frame in the sliding window; Represents the Nth key frame in the sliding window; represents the mean pose of the first key frame; represents the mean pose of the second key frame; Represents the mean pose of the Nth key frame; It means that the result is obtained after marginalizing the H matrix of the entire BA.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention firstly sharpens the original image by adopting Laplace gradient sharpening technology, thereby improving the detail clarity and edge recognition ability of the image, generates multi-resolution hierarchical images by downsampling layer by layer, and constructs an image pyramid, thereby achieving feature scale invariance and uniformity of feature point distribution, and utilizing the quadtree method and FAST corner point detection to improve the extraction efficiency and quality of feature points; then, the YOLO algorithm (a deep learning algorithm for target detection) is used in combination with semantic segmentation technology to effectively eliminate the interference of dynamic objects and ensure the accuracy of static feature points, and the rotation invariance of the feature points is enhanced by calculating the main directions of the static feature points and generating descriptors, thereby improving the accuracy of feature matching; finally, by establishing a sliding window and a matching relationship between static feature points, real-time calculation and updating of the camera posture are realized, and while updating the camera posture, map data is continuously constructed, thereby achieving real-time perception of the environment and dynamic updating of the map, thereby ensuring the accuracy of the map and the positioning accuracy of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 FIG. 1 is a flow chart of a SLAM method based on static feature point extraction provided by an embodiment of the present invention; Figure 2 The figure is a flow chart of feature point extraction provided by an embodiment of the present invention; Figure 3 The figure shows a schematic diagram of FAST corner point extraction provided by an embodiment of the present invention; Figure 4 The figure shows a principle diagram of image pyramid calculation provided by an embodiment of the present invention; Figure 5 The figure shows a quadtree optimization principle diagram provided by an embodiment of the present invention; Figure 6 The figure shows a quadtree optimization result diagram provided by an embodiment of the present invention; Figure 7 The figure is a static feature point extraction flow chart provided by an embodiment of the present invention; Figure 8 Shown is a key frame screening principle diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0021] The term "and / or" is only a description of the association relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " generally indicates that the related objects are in an "or" relationship. Example 1

[0022] See also Figure 1 This embodiment introduces a SLAM method based on static feature point extraction, comprising the following steps: Step S1, obtaining an original image, and performing a sharpening process on the original image; Here, the original image is captured by a camera or a robot.

[0023] Step S2, downsampling the sharpened image layer by layer to generate a plurality of hierarchical images with different resolutions, constructing an image pyramid through the hierarchical images, and extracting FAST corner points at each layer of the image pyramid; dividing the hierarchical images containing the FAST corner points into a plurality of grids, and extracting the optimal FAST corner point from each grid as a feature point; Step S3, screening the extracted feature points, removing dynamic feature points from the feature points, and retaining static feature points from the feature points; Here, by eliminating dynamic feature points, static feature points are retained as much as possible to ensure the accuracy of camera or robot positioning.

[0024] Step S4, selecting a circular area with the static feature point as the center, calculating the image moment in the circular area, determining the main direction of the static feature point according to the image moment, and comparing and encoding the pixel points in the circular area based on the main direction to generate a descriptor of the static feature point; Step S5, using the generated descriptor, matching static feature points between different image frames, establishing a sliding window based on the matched static feature points, and judging whether the current image frame is a key frame by the number of matched static feature points, if the current image frame is a key frame, replacing the key frame with the earliest residence time in the sliding window, and updating the key frame set in the sliding window; Step S6: Calculate and update the camera pose using the updated key frame set, and record the position of the static feature points. Construct and optimize the map by continuously updating the camera pose and the position of the static feature points to refine the map information and the position of the static feature points.

[0025] The process achieves the preprocessing of the original image and the extraction of feature points through a series of steps, thereby achieving the update of the map and the precise positioning of the camera in the environment. First, the acquired original image is sharpened to enhance the image details and edges. Then, the sharpened image is downsampled layer by layer to construct an image pyramid. FAST corner points are extracted as feature points in each layer, and the distribution of feature points is optimized by grid partitioning. Then, static feature points are screened out from the feature points and dynamic feature points are eliminated. For each static feature point, its main direction is calculated and a descriptor is generated. Using these descriptors, static feature points are matched between different image frames, and whether the current frame is a key frame is determined based on the number of matches. If it is a key frame, its information is used to calculate and update the camera's pose. Finally, according to the updated camera pose, the position of the static feature points is recorded and updated, so that the map information is continuously updated to achieve the positioning of the camera in the map. This whole process improves the accuracy and anti-interference of the SLAM system in complex scenes, while ensuring the real-time performance of the system. Example 2

[0026] See also Figure 2 , using a camera to capture the original image in the environment, and using steps S1 to S2 to process the original image, specifically including: (1) Apply Laplace gradient sharpening technology to sharpen the original image.

[0027] The core of the Laplace operator is to perform a second-order differential operation on the original image to highlight the area where the grayscale value between pixels changes dramatically. The area where the grayscale value between pixels changes dramatically is usually the edge area, and it is expressed as a high value in the second-order derivative. In digital image processing, the implementation of the Laplace operator depends on the convolution kernel, which is used to move pixel by pixel on the original image and perform a weighted summation on the neighborhood of each pixel to calculate the second-order derivative value of each pixel. This value indicates the degree of change between the grayscale value of the pixel and the average grayscale value of the surrounding neighborhood. The calculation expression of the second-order derivative value is: (1) in, Represents the second-order derivative value of the pixel; Indicates the coordinates of the pixel to the right of the current pixel; Indicates the coordinates of the pixel to the left of the current pixel; Indicates the coordinates of the pixel above the current pixel; Indicates the coordinates of the pixel below the current pixel; Indicates the current pixel coordinates.

[0028] The second-order derivative value obtained by formula (1) is added to the pixel value of the original image. If the second-order derivative value is positive, the pixel value of the original image increases (large brightness change, usually an edge); if the second-order derivative value is negative (brightness decreases, usually a flat area), the pixel value of the original image decreases, thereby achieving the purpose of sharpening the image and highlighting the edge.

[0029] (2) Calculate the image pyramid and assign the number of FAST corner points extracted to each layer.

[0030] Step S31, starting from the image after the sharpening process, downsampling is performed in sequence by a preset scaling ratio to obtain a plurality of hierarchical images with decreasing resolutions; In order to ensure the scale invariance of feature points, the present invention constructs an image pyramid. Figure 4 , starting from the sharpened original image (layer 0), set the pyramid scaling factor k to 1.2, downsample the original image layer by layer, and generate 8 hierarchical images with decreasing resolution (including the original image), where each layer of the image is 1 / 1.2 (about 83.3%) of the size of the previous layer.

[0031] Step S32: extract FAST corner points from each level of the image in the image pyramid, and distribute the extracted FAST corner points to each level of the image according to the area of ​​each level of the image.

[0032] FAST corner points are extracted from images at each level, and the pyramid level where each FAST corner point is located is recorded to obtain a set of FAST corner points at different scales for each frame of image, ensuring that the FAST corner points can be detected regardless of how their size changes in the image, thereby achieving feature scale invariance.

[0033] As the number of pyramid layers increases, the area of ​​the image gradually decreases. Correspondingly, the number of FAST corner points assigned to each layer also decreases accordingly. In order to evenly extract FAST corner points at all scales, the number of FAST corner points can be allocated according to the area of ​​each pyramid layer to ensure that matching FAST corner points can be extracted on images of different resolutions.

[0034] The extracted FAST corner points are allocated to each level of the image, including the following steps: Step S41: Add the areas of each level of the image to obtain the total area of the image pyramid; Here, assume that the width of the image at level 0 is W, the length is L, the total area C of the original image (image at level 0) = L W, the scaling factor is k (0 < k < 1), and the total area of the image pyramid is: (2) Where S represents the total area of the image pyramid; L represents the length of the original image; W represents the width of the original image; k represents the scaling factor of the image pyramid; C represents the area of the image at level 0; n represents the level of the image pyramid.

[0035] Step S42: Divide all the extracted FAST corner points by the total area of the image pyramid to calculate the number of FAST corner points allocated per unit area. The expression for the number of FAST corner points allocated per unit area is: (3) Where represents the number of FAST corner points allocated per unit area; N represents the total number of FAST corner points to be allocated or extracted.

[0036] Step S43: Multiply the number of FAST corner points by the areas of each level of the image respectively to allocate the extracted FAST corner points to each level of the image; The expression for allocating the extracted FAST corner points to each level of the image is: (4) Where represents the number of feature points allocated at level ; n represents the level of the image pyramid.

[0037] According to formula (4), when = 0, the expression for the number of FAST corner points to be allocated at level 0 is: (5) Where represents the number of FAST corner points allocated at level 0.

[0038] (3) Extract FAST corner points as feature points.

[0039] Sample the FAST corner point algorithm to extract candidate corner points as feature points from each level of the image that has been allocated. The specific steps include: Step S51, selecting pixel points p from the images of each level, taking each pixel point p as the center (central pixel point), selecting multiple pixel points on a circle with a certain radius, and comparing the multiple pixel points with the pixel point p respectively, so as to screen out FAST corner points from the multiple pixel points; Specifically, the step of comparing the plurality of pixel points with the pixel point p respectively includes: Step S511, respectively calculating the brightness difference between the pixel point p and a plurality of pixel points on the surrounding circle; The brightness difference expression between the pixel point p and multiple pixels on the surrounding circle is: (6) Where N represents the brightness difference between the surrounding points and the center point; Represents the pixel values ​​of surrounding points; The brightness of the center pixel of the table; Indicates the set threshold; represents a circle with the central pixel as the center; x represents the pixel; Step S512: classify the multiple pixel points on the surrounding circle according to the brightness difference. If the brightness of the current pixel point is higher than the brightness of the pixel point p plus the preset brightness threshold t, it is marked as bright; if the brightness of the current pixel point is lower than the brightness of the pixel point p minus the preset brightness threshold t, it is marked as dark; if the brightness of the current pixel point is between the brightness of the pixel point p minus the preset brightness threshold t and the brightness of the pixel point p plus the preset brightness threshold t, it is marked as similar; Step S513, counting the number of pixels marked as bright and dark in the circular area, if the number of pixels is greater than a preset pixel number threshold, the pixel point p is a FAST corner point; The expression for classifying each pixel is: (7) Where N represents the brightness difference between the surrounding points and the pixel point p; Represents the pixel values ​​of surrounding points; Table: The brightness of pixel p; Indicates the set threshold; represents a circle with pixel p as the center; x represents a pixel; In the embodiment of the present invention, referring to FIG3 , pixel points p are selected from eight images with different resolutions, and the brightness of the pixel point p is , taking pixel p as the center, select 16 pixels on a circle with a radius of 3, and classify the pixels on these surrounding circles into dark (darker, d), bright (b), and similar (s); Compare the 16 pixels on the circle around pixel p with pixel p (the center pixel). If there are N consecutive pixels on the selected circle with a brightness greater than 16+3.2 ( +t) or 16-3.2 ( -t), that is, the number of d or b in the circular area is counted according to formula (7). If the number of d or b appears exceeds the set threshold n, the pixel point p is extracted as a FAST corner point.

[0040] (4) Quadtree evenly distributes feature points.

[0041] The sampling quadtree method extracts the FAST corner points from each level of the image and selects the best candidate corner points as feature points. The specific steps include the following: Step S52, dividing the images of each level into a plurality of first grids, checking whether there is a FAST corner point in each first grid, if there is no FAST corner point in the first grid, lowering the threshold of the FAST corner point detection operator until a FAST corner point is found in the first grid; if there is a FAST corner point in the first grid, splitting the first grid into 4 second grids, and continuing the splitting process until the total number of split grids is greater than the total number of preset grids, and then stopping the splitting; For each layer of the pyramid, the images of each level are divided into multiple grids with a size of 30×30 pixels. FAST corner points are extracted for each grid separately. If no FAST corner points can be extracted, the threshold is lowered and the extraction of FAST corner points is tried again to ensure that FAST corner points can also be extracted in areas with weaker textures.

[0042] Step S53: Select the best FAST corner point from each split grid as a feature point.

[0043] Take a 640x480 image as an example, see Figure 5 Initially, the quadtree method was introduced to treat each layer of the hierarchical image as a single node, i.e., a grid or region. If more than one FAST corner point is detected in a single node, it is split into four smaller nodes. If the current node does not detect a FAST corner point, the node will be removed. This splitting process will continue until the total number of nodes reaches a preset value (Na), or until no further splitting can be performed. See Figure 6 After the splitting is completed, the FAST corner point with the highest response intensity in each node is selected. The selected FAST corner point may be a real feature point. In this way, a more evenly distributed corner point set can be obtained, thereby significantly improving the distribution uniformity of the feature points and optimizing the feature point extraction process.

[0044] Step S3, screening the extracted feature points, removing dynamic feature points from the feature points, and retaining static feature points from the feature points; Referring to FIG. 7 , the YOLO algorithm is used to identify the dynamic object coordinate region in the original image, the feature points within the dynamic object coordinate region are excluded, and the feature points outside the dynamic object coordinate region are retained as the first static feature points; Use the YOLO algorithm to perform semantic segmentation again on the feature points in the coordinate area of ​​the dynamic object to obtain dynamic pixel points, remove the dynamic feature points that coincide with the dynamic pixel points, and retain the dynamic feature points that are different from the dynamic pixel points as the second static feature points; The first static feature point and the second static feature point are combined to form a final static feature point set.

[0045] Step S4, selecting a circular area with the static feature point as the center, calculating the image moment in the circular area, determining the main direction of the static feature point according to the image moment, and comparing and encoding the pixel points in the circular area based on the main direction to generate a descriptor of the static feature point; Step S61, constructing a circular area with a radius R with the static feature point as the center, and calculating the first-order image moment along the x and y axes in the circular area; First, define the image moment in the circular area, and its expression is as follows: (8) in, Represents the moment of the circular area image; R represents the radius; Represents the grayscale value at pixel coordinates (x, y); Takes 0 or 1; Takes 0 or 1.

[0046] Then, in the circular area with a radius of R, the expressions of the first-order image moments along the x-axis and y-axis are: (9) (10) in, represents the first-order image moment along the y-axis; Represents the first-order image moment along the x-axis; I(x,y) represents the pixel coordinates; x represents the horizontal coordinate of the pixel; y represents the vertical coordinate of the pixel.

[0047] Step S62, calculating the sum of the brightness of all pixels in the circular area, and calculating the centroid of the circular area in combination with the first-order image moments along the x-axis and the y-axis; The calculation expression of the centroid is: (11) Among them, C represents the center of mass; Indicates the horizontal coordinate of the centroid point relative to the feature point; Indicates the ordinate of the centroid relative to the feature point; Represents the sum of the brightness of all pixels within the circular area.

[0048] Step S63, calculating the main direction of the static feature point according to the position of the centroid relative to the position of the static feature point; The expression of the main direction of the static feature point is: (12) in, Indicates the main direction of the static feature point.

[0049] Here, the center of the static feature point is the centroid, the geometric center of the circular area is the centroid, and the vector from the centroid to the centroid is the main direction of the static feature point.

[0050] Furthermore, after determining the main direction of the static feature point, the information of each static feature point is quantified, that is, the features of the local area around the static feature point are captured and converted into binary descriptors. The specific conversion process includes: To reduce noise interference, Gaussian filtering is first performed on the images at each level. An image window P of a certain size is taken with the key point as the center. A pair of pixels ( )and( ),like( ) is larger than ( ), the current bit is set to 1, otherwise it is set to 0. A 256-bit BRIEF descriptor is generated by continuous comparison. This binary string is a quantitative representation of the pixel intensity distribution around the key point, which captures the local structural information around the key point. The expression for binary assignment based on the comparison result is: (13) in, The value of BRIEF is 0 or 1; P(x) represents the pixel value of point x; P(y) represents the pixel value of point y corresponding to point x; x represents the horizontal coordinate of the pixel pair; y represents the vertical coordinate of the pixel pair; P represents the image window P.

[0051] In the static feature point matching stage, the Hamming distance between the BRIEF descriptors of two static feature points can be compared to evaluate whether they match. The smaller the Hamming distance, the more similar the two descriptors are, and the greater the possibility of key point matching.

[0052] Step S5, using the generated descriptor, matching static feature points between different image frames, establishing a sliding window based on the matched static feature points, and judging whether the current image frame is a key frame by the number of matched static feature points. If the current image frame is a key frame, replacing the key frame with the earliest residence time in the sliding window, and updating the key frame set in the sliding window; Specifically, based on the matched static feature points, a sliding window is established, which includes several key frames processed recently. By continuously updating the key frames, the camera pose is updated.

[0053] Referring to FIG8 , the steps of continuously updating the key frames in the sliding window include: Step S71: If the ratio of the number of matched static feature points T to the total number of detected static features S is less than a preset first threshold value, ,Right now , and the ratio of the number of matched static feature points T to the total number of landmark points N in the map is also less than the preset second threshold ,Right now , then the current image frame is a key frame; Step S72, adding the key frame to the sliding window, and if the number of key frames in the sliding window exceeds a preset maximum number, removing the earliest key frame to keep the number of key frames in the sliding window constant; Step S73: Calculate and update the camera pose using the updated key frame set The camera pose update formula is: (14)

[0054] in, Represents the first key frame in the sliding window; represents the second key frame in the sliding window; Represents the Nth key frame in the sliding window; represents the mean pose of the first key frame; represents the mean pose of the second key frame; Represents the mean pose of the Nth key frame; It means that the result is obtained after marginalizing the H matrix of the entire BA.

[0055] Step S6: based on the updated camera pose, the positions of the static feature points are recorded, and the map is constructed and optimized through the continuously updated camera pose and the positions of the static feature points to refine the map information and the positions of the static feature points.

[0056] Assume that there are N key frame poses in this sliding window expressed as ( ), and M landmarks ( ), and M landmarks and N key frames form a local map, and the local map is optimized. The specific optimization process includes: Build a graph optimization model containing keyframe poses and landmarks, and construct the Hessian matrix. Through marginalization, some variables (in this case, landmarks) can be removed from the global optimization problem while retaining the marginal distribution of these variables (that is, the conditional probability distribution of these variables). Doing so can reduce the size of the Hessian matrix, thereby speeding up the solution of the optimization process. When marginalizing, the camera pose is continuously updated.

[0057] Finally, according to the camera posture, the position of the static feature point is updated and recorded to continuously update the information in the map, thereby determining the position of the camera in the map.

[0058] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A SLAM method based on static feature point extraction, characterized in that: include: Acquire an original image, and perform sharpening processing on the original image; Downsampling the sharpened image layer by layer to generate multiple hierarchical images with different resolutions, using the hierarchical images to construct an image pyramid, extracting FAST corner points at each layer of the image pyramid, dividing the hierarchical images containing the FAST corner points into multiple grids, and extracting the optimal FAST corner points from each grid as feature points; Screening the extracted feature points, removing dynamic feature points from the feature points, and retaining static feature points from the feature points; A circular area is selected with the static feature point as the center, an image moment is calculated within the circular area, a main direction of the static feature point is determined according to the image moment, and based on the main direction, pixel points within the circular area are compared and encoded to generate a descriptor of the static feature point; Using the generated descriptor, static feature points are matched between different image frames. Based on the matched static feature points, a sliding window is established. The number of matched static feature points is used to determine whether the current image frame is a key frame. If the current image frame is a key frame, the key frame with the longest stay time in the sliding window is replaced, and the key frame set in the sliding window is updated. The updated key frame set is used to calculate and update the camera pose, and the position of the static feature points is recorded. The map is constructed and optimized through the continuously updated camera pose and the position of the static feature points to refine the map information and the position of the static feature points.

2. The SLAM method based on static feature point extraction according to claim 1, characterized in that: The original image is sharpened, comprising: Use the convolution kernel in the Laplace operator to move pixel by pixel on the original image, perform weighted summation on the neighborhood of each pixel, and calculate the second-order derivative value of each pixel; The second-order derivative value of each pixel is added to the pixel value of the original image to obtain a sharpened image.

3. The SLAM method based on static feature point extraction according to claim 1, characterized in that: Extract FAST corner points at each layer of the image pyramid, including: Starting from the sharpened image, downsampling is performed in sequence according to a preset scaling ratio to obtain a plurality of hierarchical images with decreasing resolutions; FAST corner points are extracted for each level of image in the image pyramid, and the extracted FAST corner points are distributed to each level of image according to the area of ​​each level of image.

4. The SLAM method based on static feature point extraction according to claim 3, characterized in that: The extracted FAST corner points are distributed to each level of images according to the area of ​​each level of images, including: Adding the areas of the images at each level to obtain the total area of ​​the image pyramid; The total area of ​​the image pyramid is: ； Among them, S represents the total area of ​​the image pyramid; L represents the length of the original image; W represents the width of the original image; k represents the scaling factor of the image pyramid; C represents the area of ​​the 0th layer image; Divide all the extracted FAST corner points by the total area of ​​the image pyramid to calculate the number of FAST corner points allocated per unit area. The expression for the number of FAST corner points allocated per unit area is: ； in, represents the number of FAST corner points allocated per unit area; N represents the total number of FAST corner points extracted; The number of FAST corner points is multiplied by the area of ​​each level of image, so as to distribute the extracted FAST corner points to each level of image; The expression of the extracted FAST corner points distributed to each level of image is: ； in, Indicates The number of feature points assigned to the layer; n represents the image pyramid level.

5. The SLAM method based on static feature point extraction according to claim 1, characterized in that: The extracting of the best FAST corner point from each grid as a feature point includes: Selecting pixel points p from the images of each level, taking each pixel point p as the center, selecting multiple pixel points on a circle with a certain radius, and comparing the multiple pixel points with the pixel point p respectively, so as to screen out FAST corner points from the multiple pixel points; Divide the images at each level into a plurality of first grids, check whether there is a FAST corner point in each first grid, and if there is no FAST corner point in the first grid, lower the threshold of the FAST corner point detection operator until a FAST corner point is found in the first grid; if there is a FAST corner point in the first grid, split the first grid into four second grids, and continue the splitting process until the total number of split grids is greater than the total number of preset grids, and stop splitting; The best FAST corner points are selected from each split grid as feature points.

6. The SLAM method based on static feature point extraction according to claim 5, characterized in that: Comparing the plurality of pixel points with the pixel point p respectively comprises: Calculate the brightness difference between the pixel point p and multiple pixel points on the surrounding circle respectively; The brightness difference expression between the pixel point p and multiple pixels on the surrounding circle is: ； Where N represents the brightness difference between the surrounding points and the pixel point p; Represents the pixel values ​​of surrounding points; Indicates the brightness of pixel p; Indicates the set threshold; represents a circle with pixel p as the center; x represents a pixel; According to the brightness difference, multiple pixel points on the surrounding circle are classified. If the brightness of the current pixel point is higher than the brightness of the pixel point p plus the preset brightness threshold t, it is marked as bright; if the brightness of the current pixel point is lower than the brightness of the pixel point p minus the preset brightness threshold t, it is marked as dark; if the brightness of the current pixel point is between the brightness of the pixel point p minus the preset brightness threshold t and the brightness of the pixel point p plus the preset brightness threshold t, it is marked as similar; Count the number of pixels marked as bright and dark in the circular area. If the number of multiple pixels on the surrounding circle is greater than a preset pixel number threshold, the pixel point p is a FAST corner point. The expression for classifying multiple pixel points on the surrounding circle is: ； in, Represents the value of the pixel on the circle; d represents a dark pixel; s represents a similar pixel; b represents a bright pixel; and t represents 20% of the brightness value of pixel p.

7. The SLAM method based on static feature point extraction according to claim 1, characterized in that: The screening of the extracted feature points includes: Using the YOLO algorithm to identify the dynamic object coordinate region in the original image, excluding feature points within the dynamic object coordinate region, and retaining feature points outside the dynamic object coordinate region as first static feature points; Use the YOLO algorithm to perform semantic segmentation again on the feature points in the coordinate area of ​​the dynamic object to obtain dynamic pixel points, remove the dynamic feature points that coincide with the dynamic pixel points, and retain the dynamic feature points that are different from the dynamic pixel points as the second static feature points; The first static feature point and the second static feature point are combined to form a final static feature point set.

8. The SLAM method based on static feature point extraction according to claim 1, characterized in that: The determining the main direction of the static feature point includes: A circular area with a radius of R is constructed with the static feature point as the center, and the first-order image moment along the x-axis and the y-axis in the circular area is calculated; The expressions of the first-order image moments along the x-axis and the y-axis are respectively: ； ； in, represents the first-order image moment along the y-axis; represents the first-order image moment along the x-axis; R represents the radius of the circular area; I(x,y) represents the pixel coordinates; x represents the horizontal coordinate of the pixel; y represents the vertical coordinate of the pixel; Calculate the sum of the brightness of all pixels in the circular area, and calculate the centroid of the circular area by combining the first-order image moments along the x-axis and the y-axis; The calculation expression of the centroid is: ； Where C represents the coordinates of the center of mass; Indicates the horizontal coordinate of the centroid point relative to the feature point; Indicates the ordinate of the centroid relative to the feature point; Represents the sum of the brightness of all pixels in the circular area; Calculating the main direction of the static feature point according to the position of the centroid relative to the position of the static feature point; The expression of the main direction of the static feature point is: = ( )； in, Indicates the main direction of the static feature point.

9. The SLAM method based on static feature point extraction according to claim 1, characterized in that: The calculation and update of the camera pose includes: If the ratio of the number of matched static feature points T to the total number of detected static feature points S is less than a preset first threshold , and the ratio of the number of matched static feature points T to the total number of landmark points N in the map is also less than the preset second threshold , then the current image frame is a key frame; Adding the key frame to the sliding window, if the key frames in the sliding window exceed a preset maximum number, removing the key frame with the earliest time to keep the number of key frames in the sliding window constant; The camera pose is calculated and updated using the continuously updated key frame information.

10. The SLAM method based on static feature point extraction according to claim 9, characterized in that: The update expression of the camera pose is: ； in, Represents the first key frame in the sliding window; represents the second key frame in the sliding window; Represents the Nth key frame in the sliding window; represents the mean pose of the first key frame; represents the mean pose of the second key frame; Represents the mean pose of the Nth key frame; It means that the result is obtained after marginalizing the H matrix of the entire BA.

Images