ORB feature point extraction method, device and equipment and parking method
Fast corner points are screened through the double threshold method and grid filtering, and combined with Gaussian fuzzy calculation descriptors, which solves the problems of uneven distribution, insufficient descriptor smoothing, and failure to consider the mask area in the DSP module during the ORB feature point extraction process. , achieving a more uniform feature point distribution and smoother descriptor changes.
Patent Information
- Application Number
- CN202411321069.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2025-01-24
AI Technical Summary
During the ORB feature point extraction process, the existing DSP module has problems such as uneven distribution of feature points, insufficient descriptor smoothness, and failure to consider the mask area, resulting in the degradation of VSLAM's front-end perception ability and mismatch.
The double -threshold method is used to extract the FAST angle point, and the grid filtering is performed on the basis of the mask area to screen out the determined FAST angle point. Then, Gaussian is blurred by the Gaussian nucleus, and the descriptor of each FAST angle point is calculated to get the ORB feature point.
Improving the uniformity of the ORB feature point in the image position, enhancing the smoothness of the descriptor, reducing the wrong matching situation, and improving the front -end perception ability of VSLAM.
Smart Images
Figure CN119360050A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to an ORB feature point extraction method, device, equipment and parking method. Background Art
[0002] In Visual Simultaneous Localization and Mapping (VSLAM), ORB feature points are widely used in the feature point extraction module of VSLAM front-end due to their relatively simple extraction and descriptor calculation process, and serve the subsequent matching, positioning, mapping and other modules.
[0003] The ORB feature point extraction methods currently used in VSLAM are generally divided into two types of module processing methods: CPU module and non-CPU module, and most non-CPU modules are extracted with the help of DSP modules. The ORB feature point extraction of CPU modules has the disadvantages of slow extraction speed and large CPU computing resources. Under this premise, a solution for ORB feature point extraction through DSP modules has emerged on the market.
[0004] The ORB feature point extraction of the DSP module contains a large number of ORB feature points that are repeatedly calculated. The calculation time of the extraction process in the DSP module is significantly improved compared to the CPU module. Although there is an integrated ORB feature point extraction solution on the existing DSP side, which can extract a number of ORB feature points from the input image, the ORB feature point extraction solution of the existing DSP module still has the problem that the extracted ORB feature points are unevenly distributed on the image, and a large number of extracted ORB feature points will gather in the area with obvious image texture, which reduces the effective extraction area and reduces the front-end perception ability of VSLAM. In addition, the ORB feature point extraction solution of the existing DSP module also has the problem of directly calculating the descriptor of the original grayscale image of the image when calculating the descriptor, which makes the descriptor not smooth enough, thereby reducing the success rate of key point matching between subsequent frames. And the mask area of the original image is not considered, so that the points in the mask area of the image are also included in the extracted feature point set, resulting in subsequent mismatching and other situations. Summary of the invention
[0005] In view of the above problems, the embodiments of the present invention provide an ORB feature point extraction method, device, equipment and parking method, which solve the problems of the existing ORB feature point extraction scheme of the DSP module, such as the uneven distribution of the extracted ORB feature points in the image position, the excessive mutation of the descriptor and the failure to consider the mask area.
[0006] According to one aspect of an embodiment of the present invention, there is provided an ORB feature point extraction method, comprising the following steps:
[0007] Use double threshold to extract Fast corner points in the image;
[0008] Filter the Fast corner points based on the mask area to select the determined Fast corner points;
[0009] Use Gaussian kernel to perform Gaussian blur on the original grayscale image of the image;
[0010] The descriptor of each Fast corner point is calculated through the original grayscale image after Gaussian blur to obtain the ORB feature point.
[0011] In some optional implementations, the screening of Fast corner points is determined, specifically including the steps of:
[0012] Grid division is performed on each layer of the image pyramid based on the mask area to screen out the number of valid grids;
[0013] Get the number of Fast corner points and Fast score threshold of each valid grid;
[0014] The determined Fast corner points are screened out according to the number of Fast corner points of each valid grid and the Fast score threshold.
[0015] In some optional implementations, extracting Fast corner points in an image specifically includes the following steps:
[0016] Set the high and low thresholds in the dual threshold;
[0017] Extract Fast corner points in the image through a low threshold;
[0018] The low threshold is the pixel difference threshold of the surrounding neighboring Fast corner points, and the high threshold is the corner point threshold that is primarily obtained during the effective grid screening process.
[0019] In some optional implementations, the screening of valid grids specifically includes the following steps:
[0020] Eliminate the Fast corner points outside the mask area and divide the remaining Fast corner points into several grids;
[0021] Remove the empty grids without Fast corner points to get the number of valid grids.
[0022] In some optional implementations, the calculation of the Fast score threshold of each valid grid specifically includes:
[0023] Traverse the Fast corner point score of each Fast corner point in the valid grid;
[0024] Count the number of Fast corner points in the valid grid whose Fast corner points scores are greater than the high threshold and the low threshold in the double threshold respectively;
[0025] Take the average of the Fast corner scores that are greater than the high threshold and the low threshold respectively;
[0026] The Fast score threshold of the valid grid is calculated by the average score.
[0027] In some optional implementations, the calculation of the Fast score threshold of the valid grid specifically includes:
[0028] When the average score of the Fast corner points greater than the high threshold is greater than or equal to the effective Fast corner point number threshold of the grid, the Fast score threshold of the effective grid is calculated by the following formula:
[0029]
[0030] When the average score of the Fast corner points greater than the low threshold is less than the effective Fast corner point threshold of the grid, the Fast score threshold of the effective grid is calculated by the following formula:
[0031]
[0032] Among them, score grid _th represents the Fast score threshold of the valid grid, score grid _higj represents the average score of Fast corner points greater than the high threshold, core grid _low represents the average score of the Fast corner points that are greater than the low threshold, Q_high represents the high threshold, Q_low represents the low threshold, and grid high _N represents the number of Fast corner points in the valid grid with a Fast corner point score greater than the high threshold, grid low _N represents the number of Fast corner points in the valid grid with a Fast corner point score greater than the low threshold, N grid _th represents the effective Fast corner point count threshold of the grid.
[0033] In some optional implementations, the screening of the determined Fast corner points specifically includes:
[0034] Extract valid grids whose number of Fast corner points is less than or equal to the threshold of the number of valid Fast corner points;
[0035] The Fast corner points in the valid grid whose Fast corner point scores are less than or equal to the Fast score threshold of the valid grid are extracted as the determined Fast corner points.
[0036] In some optional implementations, Gaussian blur is performed on the original grayscale image of the image, specifically including:
[0037] Calculate the angle of the centroid vector of each determined Fast corner point;
[0038] Generate a Gaussian kernel according to the angle of the centroid vector of each determined Fast corner point;
[0039] Two layers of 5×5 Gaussian kernels are used to perform Gaussian blur on the original grayscale image.
[0040] According to another aspect of an embodiment of the present invention, there is provided an ORB feature point extraction device, comprising:
[0041] An acquisition module is used to extract Fast corner points in an image using a double threshold;
[0042] A screening module is used to filter the Fast corner points by gridding on the basis of the mask area to screen out the determined Fast corner points;
[0043] The calculation module is used to perform Gaussian blur on the original grayscale image of the image using a Gaussian kernel according to the determined Fast corner point, and calculate the descriptor of each Fast corner point through the original grayscale image after Gaussian blur to obtain the ORB feature point of the fast corner point.
[0044] According to another aspect of an embodiment of the present invention, there is provided an ORB feature point extraction device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;
[0045] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the ORB feature point extraction method as described above.
[0046] According to another aspect of an embodiment of the present invention, there is provided a parking method, comprising the following steps:
[0047] The ORB feature point extraction method described above is used to obtain the ORB feature points in the current frame image;
[0048] Establishing a parking map based on the ORB feature points;
[0049] During parking, load the established parking map;
[0050] Update the map reference frame corresponding to the vehicle's current position in real time;
[0051] The ORB feature point extraction method described above is used to extract the ORB feature points of the current scene;
[0052] Matching the map reference frame with the extracted ORB feature points of the current scene;
[0053] Positioning within the route based on matching information between the map reference frame and the extracted ORB feature points of the current scene;
[0054] The vehicle is controlled based on the positioning information.
[0055] Compared with the prior art, the present invention has the following advantages:
[0056] The invention provides an ORB feature point extraction method, which selects out the determined Fast corner points through the double threshold method and grid homogenization, solves the problem of uneven distribution of feature points in image positions and failure to consider the mask area; then, the original grayscale image of the image is Gaussian-blurred by using a Gaussian kernel, and the descriptor of each Fast corner point is calculated through the original grayscale image after Gaussian blurring to obtain the ORB feature points, which solves the problem of insufficient smoothness of the descriptor. The ORB feature points obtained by the ORB feature point extraction method provided by the invention have the characteristics of higher homogenization degree and smoother descriptor changes.
[0057] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:
[0059] Figure 1 A schematic diagram of a flow chart of an ORB feature point extraction method provided by an embodiment of the present invention is shown;
[0060] Figure 2 A schematic diagram of a screening process for determining Fast corner points in an ORB feature point extraction method provided by the present invention is shown;
[0061] Figure 3 A schematic diagram of the calculation process of the Fast score threshold of each valid grid in an ORB feature point extraction method provided by the present invention is shown;
[0062] Figure 4 A schematic diagram of the structure of an ORB feature point extraction device provided by an embodiment of the present invention is shown;
[0063] Figure 5A schematic diagram of the structure of an ORB feature point extraction device provided by an embodiment of the present invention is shown;
[0064] Figure 6 A schematic flow chart of a parking method provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0065] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.
[0066] In order to solve the problems of the existing ORB feature point extraction scheme of the DSP module, such as uneven distribution of the extracted ORB feature points in the image position, too large descriptor mutation and failure to consider the mask area, this embodiment provides an ORB feature point extraction method.
[0067] Figure 1 A schematic flow chart of an ORB feature point extraction method provided in this embodiment is shown.
[0068] This embodiment provides an ORB feature point extraction method, such as Figure 1 As shown, the following steps are included:
[0069] S10. Use double threshold to extract Fast corner points in the image.
[0070] In this step, it is assumed that the double threshold has a high threshold and a low threshold. The high threshold and the low threshold are set through multiple experimental verifications or conventional rules, and then the low threshold is used to extract the Fast corner points in the image.
[0071] The low threshold is the pixel difference threshold of the surrounding neighboring Fast corner points, and the high threshold is the corner point threshold that is primarily obtained during the effective grid screening process. Fast corner points with higher scores can be obtained through the high threshold.
[0072] In some optional embodiments, the value range of the high threshold is 15-25, and the value range of the low threshold is 5-8.
[0073] S20. Filter the Fast corner points based on the mask area to select the determined Fast corner points.
[0074] In this step, the mask area needs to be determined first, and then the Fast corner points outside the mask area are removed, and the Fast corner points located in the mask area are filtered to screen out the determined Fast corner points.
[0075] The Fast corner points determined by this step are screened out, and the degree of homogenization is higher.
[0076] In this embodiment, it should be noted that the functions of the mask area mainly include extracting the region of interest, shielding, extracting structural features, and making images of special shapes. Among them, extracting the region of interest can be understood as: the mask can be used to extract a specific area from the image, and the region of interest image can be obtained by multiplying the pre-made region of interest mask with the image to be processed. In this process, the image value in the region of interest remains unchanged, and the image value outside the region is set to 0, thereby realizing the extraction of a specific area.
[0077] In this embodiment, the determination of the mask area is a prior art, which will not be elaborated in detail in this embodiment.
[0078] S30. Use a Gaussian kernel to perform Gaussian blur on the original grayscale image of the image.
[0079] In this step, it can be understood that based on the determined Fast corner points obtained in the above steps, the angles of the centroid vectors of each determined Fast corner point are calculated, and then a Gaussian kernel is generated according to the angles of the centroid vectors of each determined Fast corner point. Finally, a two-layer 5×5 Gaussian kernel is used to perform Gaussian blur on the original grayscale image of the image to smooth the original grayscale image.
[0080] S40. Calculate the descriptor of each Fast corner point through the original grayscale image after Gaussian blur to obtain the ORB feature point.
[0081] This step is specifically as follows: based on the original grayscale image after Gaussian blur, the descriptor of each Fast corner point is calculated, and the obtained descriptor changes more smoothly. The Fast corner point based on the calculated descriptor will constitute an ORB feature point of complete information.
[0082] The ORB feature point extraction method disclosed in this embodiment selects the determined Fast corner points through the double threshold method and grid homogenization, which solves the problem of uneven distribution of feature points in the image position and failure to consider the mask area; then the original grayscale image of the image is Gaussian blurred using a Gaussian kernel, and the descriptor of each Fast corner point is calculated through the original grayscale image after Gaussian blur to obtain the ORB feature point, which solves the problem of insufficient smoothness of the descriptor. The ORB feature points obtained by the ORB feature point extraction method provided by the present invention have the characteristics of higher homogenization and smoother changes in descriptors.
[0083] In some optional embodiments, the screening process of the determined Fast corner points is optimized.
[0084] See also Figure 2 As shown, the screening of the determined Fast corner points specifically includes the following steps:
[0085] S201. Grid each layer of the image pyramid based on the mask area, and select the effective number of grids.
[0086] In this step, it can be understood as: meshing each layer of the image pyramid located in the mask area to obtain multiple grids, and the Fast corner points located in the mask area are distributed in some grids, and the grids containing zero Fast corner points are eliminated to obtain the number of valid grids. The number of Fast corner points contained in each valid grid is not zero, that is, greater than or equal to 1.
[0087] It can also be understood as removing the Fast corner points outside the mask area and meshing each layer of the image pyramid located in the mask area. Thus, the remaining Fast corner points are divided into several grids; the empty grids without Fast corner points are removed to obtain the number of valid grids.
[0088] For example: Assume that each layer of the image pyramid is divided into n*n uniform grids, and the number of grids with zero Fast corner points in the grid is N. empty , then the effective number of grids N = n*nN empty Assuming that the number of Fast corner points required to be extracted by the current pyramid layer is level_N, the expected number of extracted points for each grid is N. grid _hope=level_N / N, at this time, the grid point threshold is also equal to this value, that is, N grid_th =N grid _hope.
[0089] S202. Obtain the number of Fast corner points and the Fast scoring threshold of each valid grid.
[0090] This step can be understood as: counting the number of Fast corner points in each valid grid and calculating the Fast score threshold of the valid grid.
[0091] In some optional embodiments, the calculation process of the Fast score threshold of the valid grid is optimized.
[0092] See also Figure 3 As shown in FIG. 1 , the calculation process of the Fast score threshold of the valid grid includes the following steps:
[0093] S2021. Traverse the Fast corner point score of each Fast corner point in the valid grid.
[0094] This step can be understood as: each Fast corner point has a Fast corner point score, and the Fast corner point score of each Fast corner point in the valid grid is obtained.
[0095] S2022. Count the number of Fast corner points in the valid grid whose Fast corner point scores are respectively greater than the high threshold and the low threshold in the dual threshold.
[0096] This step can be understood as: counting the number of Fast corner points in the valid grid whose Fast corner point score is greater than the high threshold Q_high in the double threshold high _N, and the number of Fast corner points in the valid grid whose Fast corner point score is greater than the low threshold Q_low in the double threshold grid low _N.
[0097] The Fast corner point score of the Fast corner point is greater than the high threshold Q_high, and its Fast corner point score is also greater than the low threshold Q_low.
[0098] S2023. Take the average of the Fast corner point scores that are respectively greater than the high threshold and the low threshold.
[0099] This step can be understood as: taking the average of the Fast corner point scores of multiple Fast corner points whose Fast corner point scores are greater than the high threshold Q_high, and obtaining the first average score score grid _high.
[0100] The average of the Fast corner point scores of multiple Fast corner points with a Fast corner point score greater than the low threshold Q_low is taken to obtain the second average score score grid _low.
[0101] S2024. Calculate the Fast score threshold of the valid grid using the average score value.
[0102] This step is specifically as follows: when the average score of the Fast corner points that are greater than the high threshold is greater than or equal to the effective Fast corner point number threshold of the grid, that is, when the first score average is greater than or equal to the effective Fast corner point number threshold of the grid, the Fast score threshold of the effective grid is calculated by the following formula:
[0103]
[0104] When the average score of the Fast corner points greater than the low threshold is less than the effective Fast corner point threshold of the grid, that is, when the second score average is greater than or equal to the effective Fast corner point threshold of the grid, the Fast score threshold of the effective grid is calculated by the following formula:
[0105]
[0106]
[0107] Among them, scoregrid _th represents the Fast score threshold of the valid grid, score grid _high represents the average score of the Fast corner points that are greater than the high threshold, that is, the first average score, core grid _low represents the average score of the Fast corner points that are greater than the low threshold, that is, the second average score, Q_high represents the high threshold, Q_low represents the low threshold, grid high _N represents the number of Fast corner points in the valid grid with a Fast corner point score greater than the high threshold, grid low _N represents the number of Fast corner points in the valid grid with a Fast corner point score greater than the low threshold, N grid _th represents the effective Fast corner point count threshold of the grid.
[0108] S203. Filter out the determined Fast corner points according to the number of Fast corner points of each valid grid and the Fast score threshold.
[0109] This step specifically includes:
[0110] Extract valid meshes whose number of Fast corner points is less than or equal to the threshold of the number of valid Fast corner points.
[0111] The Fast corner points in the valid grid whose Fast corner point scores are less than or equal to the Fast score threshold of the valid grid are extracted as the determined Fast corner points.
[0112] That is, extract N grid <=N grid _th valid grid, where: N grid Indicates the number of Fast corner points in the effective grid, N grid _th represents the effective Fast corner point count threshold of the grid.
[0113] Then, the Fast corner points in the valid grid that are less than or equal to the Fast score threshold of the valid grid are equally divided and extracted as the determined Fast corner points for subsequent extraction of ORB feature points.
[0114] In addition, in this step, it is also necessary to explain that, in step S201, it is explained that, assuming that the number of Fast corner points required to be extracted by the current layer pyramid is level_N, then when the number of Fast corner points extracted is greater than level_N, no new Fast corner points are included.
[0115] In addition, in the embodiment of the present invention, it should be noted that after the determined Fast corner points are obtained through step S20, the angles of the centroid vectors of each determined Fast corner point are calculated, a Gaussian kernel is generated according to the angles of the centroid vectors of each determined Fast corner point, and a two-layer 5×5 Gaussian kernel is used to perform Gaussian blur on the original grayscale image of the image to smooth the original grayscale image; finally, the descriptor of each Fast corner point is calculated based on the original grayscale image after Gaussian blur, and the obtained descriptor changes more smoothly. The Fast corner points based on the calculated descriptors will constitute ORB feature points of complete information.
[0116] In a preferred embodiment, the generated Gaussian kernel is:
[0117]
[0118] In some optional embodiments, the embodiments of the present invention further provide an ORB feature point extraction device.
[0119] This embodiment provides an ORB feature point extraction device, such as Figure 4 As shown, it includes: an acquisition module 100, a screening module 200 and a calculation module 300.
[0120] The acquisition module 100 is used to extract Fast corner points in an image using a double threshold value.
[0121] The screening module 200 is used to filter the Fast corner points by gridding based on the mask area to screen out the determined Fast corner points.
[0122] The calculation module 300 is used to perform Gaussian blur on the original grayscale image of the image using a Gaussian kernel according to the determined Fast corner point, and calculate the descriptor of each Fast corner point through the original grayscale image after Gaussian blur to obtain the ORB feature point of the fast corner point.
[0123] In this embodiment, the acquisition module 100 uses the lower threshold value of the double threshold value to extract the Fast corner points in the image.
[0124] The specific implementation process of the screening module 200 screening the determined Fast corner points is: gridding each layer of the image pyramid based on the mask area, screening out the number of valid grids, and then obtaining the number of Fast corner points of each valid grid and the Fast score threshold, and finally screening out the determined Fast corner points according to the number of Fast corner points of each valid grid and the Fast score threshold.
[0125] In this embodiment, the specific process of the screening module 200 in realizing the above functions can be found in step S202 in the above ORB feature point extraction method, which will not be repeated in this embodiment.
[0126] The specific implementation process of the calculation module 300 calculating the ORB feature points is as follows: based on the determined Fast corner points obtained by the above-mentioned screening module 200, the angle of the centroid vector of each determined Fast corner point is calculated, and then a Gaussian kernel is generated according to the angle of the centroid vector of each determined Fast corner point, and finally a two-layer 5×5 Gaussian kernel is used to perform Gaussian blur on the original grayscale image of the image to smooth the original grayscale image; based on the original grayscale image after Gaussian blur, the descriptor of each Fast corner point is calculated, and the obtained descriptor changes more smoothly, and the Fast corner points based on the calculated descriptors will constitute ORB feature points with complete information.
[0127] The present embodiment provides an ORB feature point extraction device, which selects the determined Fast corner points through the double threshold method and grid homogenization, and solves the problem of uneven distribution of feature points in the image position and not considering the mask area; then the original grayscale image of the Gaussian kernel is Gaussian blurred, and the descriptor of each Fast corner point is calculated through the original grayscale image after Gaussian blur to obtain the ORB feature point, which solves the problem of insufficient smoothness of the descriptor. The ORB feature points obtained by the ORB feature point extraction method provided by the present invention have the characteristics of higher homogenization and smoother changes of descriptors.
[0128] In some optional embodiments, the present invention further provides an ORB feature point extraction device.
[0129] Figure 5 A schematic structural diagram of an ORB feature point extraction device provided by an embodiment of the present invention is shown.
[0130] like Figure 5 As shown, the ORB feature point extraction device includes: a processor 410 , a memory 420 , a communication interface 430 and a communication bus 440 . The processor 410 , the memory 420 and the communication interface 430 communicate with each other through the communication bus 440 .
[0131] In this embodiment, the memory 420 is used to store at least one executable instruction, and the executable instruction enables the processor 420 to perform the operation of the ORB feature point extraction method as described in any of the above embodiments; the communication interface 430 is used to communicate with other devices such as a client or other server network elements. The processor 410 is used to execute the program 450, and specifically can execute the relevant steps in the above ORB feature point extraction method embodiment.
[0132] Specifically, program 450 may include program code including computer executable instructions.
[0133] The processor 410 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present invention. The one or more processors included in the ORB feature point extraction device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.
[0134] The memory 420 is used to store the program 450. The memory 420 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.
[0135] The program 450 may be specifically called by the processor 410 to enable the ORB feature point extraction device to perform the operation of the ORB feature point extraction method described in any of the above embodiments.
[0136] In some optional embodiments, the embodiments of the present invention further disclose a computer-readable storage medium, wherein at least one executable instruction is stored in the storage medium. When the executable instruction is executed on an ORB feature point extraction device, the ORB feature point extraction device executes the ORB feature point extraction method in any of the above method embodiments.
[0137] In some optional embodiments, the embodiments of the present invention further disclose a parking method.
[0138] See also Figure 6 , the figure shows a schematic flow chart of a parking method provided in this embodiment.
[0139] A parking method disclosed in this embodiment includes the following steps:
[0140] S510. Use the ORB feature point extraction method described in the above method embodiment to obtain the ORB feature points in the current frame image.
[0141] This step can be understood as: when a parking map needs to be created, after the user turns on memory parking map creation, the ORB feature point extraction method described in the above method embodiment extracts the visual features of the current frame scene, namely, ORB feature points, in real time.
[0142] S520. Create a parking map based on the ORB feature points.
[0143] This step specifically includes: tracking the ORB feature points extracted in step S510, retaining valid ORB feature points, and then saving the valid ORB feature points into the map to optimize the map, thereby completing the establishment of the parking map.
[0144] S530. During the parking process, load the established parking map.
[0145] This step can be understood as: when parking is needed, the user turns on memory parking, selects the parking map created last time, and then loads the parking map.
[0146] S540. Update the map reference frame corresponding to the current position of the vehicle in real time.
[0147] S550. Use the ORB feature point extraction method described in the above method embodiment to extract the ORB feature points of the current scene.
[0148] S560. Match the map reference frame with the extracted ORB feature points of the current scene.
[0149] S570. Positioning within the route based on matching information between the map reference frame and the extracted ORB feature points of the current scene.
[0150] S580. Control the vehicle according to the positioning position information to implement the parking process.
[0151] In this embodiment, the parking method includes a mapping process and a parking positioning process, wherein the mapping process includes steps S510-S520; and the parking positioning process includes steps S530-S580.
[0152] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. Similarly, in order to simplify the present invention and help understand one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. Wherein, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself is a separate embodiment of the present invention.
[0153] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive.
[0154] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.
Claims
1. An ORB feature point extraction method, characterized in that: The following steps are involved: Use double threshold to extract Fast corner points in the image; Filter the Fast corner points based on the mask area to select the determined Fast corner points; Use Gaussian kernel to perform Gaussian blur on the original grayscale image of the image; The descriptor of each Fast corner point is calculated through the original grayscale image after Gaussian blur to obtain the ORB feature point.
2. The ORB feature point extraction method according to claim 1, characterized in that: Determine the screening of Fast corner points, including the following steps: Grid division is performed on each layer of the image pyramid based on the mask area to screen out the number of valid grids; Get the number of Fast corner points and Fast score threshold of each valid grid; The determined Fast corner points are screened out according to the number of Fast corner points of each valid grid and the Fast score threshold.
3. The ORB feature point extraction method according to claim 1, characterized in that: Extracting Fast corner points from an image includes the following steps: Set the high and low thresholds in the dual threshold; Extract Fast corner points in the image through a low threshold; The low threshold is the pixel difference threshold of the surrounding neighboring Fast corner points, and the high threshold is the corner point threshold that is primarily obtained during the effective grid screening process.
4. The ORB feature point extraction method according to claim 2, characterized in that: The screening of effective grids includes the following steps: Eliminate the Fast corner points outside the mask area and divide the remaining Fast corner points into several grids; Remove the empty grids without Fast corner points to get the number of valid grids.
5. The ORB feature point extraction method according to claim 2, characterized in that: The calculation of the Fast score threshold for each valid grid includes: Traverse the Fast corner point score of each Fast corner point in the valid grid; Count the number of Fast corner points in the valid grid whose Fast corner points scores are greater than the high threshold and the low threshold in the double threshold respectively; Take the average of the Fast corner scores that are greater than the high threshold and the low threshold respectively; The Fast score threshold of the valid grid is calculated by the average score.
6. The ORB feature point extraction method according to claim 5, characterized in that: The calculation of the Fast score threshold of the valid grid includes: When the average score of the Fast corner points greater than the high threshold is greater than or equal to the effective Fast corner point number threshold of the grid, the Fast score threshold of the effective grid is calculated by the following formula: When the average score of the Fast corner points greater than the low threshold is less than the effective Fast corner point threshold of the grid, the Fast score threshold of the effective grid is calculated by the following formula: Among them, score grid _th represents the Fast score threshold of the valid grid, score grid _high represents the average score of Fast corner points that are greater than the high threshold, core grid _low represents the average score of the Fast corner points that are greater than the low threshold, Q_high represents the high threshold, Q_low represents the low threshold, and grid high _N represents the number of Fast corner points in the valid grid with a Fast corner point score greater than the high threshold, grid low _N represents the number of Fast corner points in the valid grid with a Fast corner point score greater than the low threshold, N grid _th represents the effective Fast corner point count threshold of the grid.
7. The ORB feature point extraction method according to claim 5, characterized in that: The screening of the determined Fast corner points includes: Extract valid grids whose number of Fast corner points is less than or equal to the threshold of the number of valid Fast corner points; The Fast corner points in the valid grid whose Fast corner point scores are less than or equal to the Fast score threshold of the valid grid are extracted as the determined Fast corner points.
8. The ORB feature point extraction method according to claim 1, characterized in that: Gaussian blur is performed on the original grayscale image, including: Calculate the angle of the centroid vector of each determined Fast corner point; Generate a Gaussian kernel according to the angle of the centroid vector of each determined Fast corner point; Two layers of 5×5 Gaussian kernels are used to perform Gaussian blur on the original grayscale image.
9. An ORB feature point extraction device, characterized in that: include: An acquisition module is used to extract Fast corner points in an image using a double threshold; A screening module is used to filter the Fast corner points by gridding on the basis of the mask area to screen out the determined Fast corner points; The calculation module is used to perform Gaussian blur on the original grayscale image of the image using a Gaussian kernel according to the determined Fast corner point, and calculate the descriptor of each Fast corner point through the original grayscale image after Gaussian blur to obtain the ORB feature point of the fast corner point.
10. An ORB feature point extraction device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the ORB feature point extraction method according to any one of claims 1 to 7.
11. A parking method, characterized in that: The following steps are involved: Acquire the ORB feature points in the current frame image using the ORB feature point extraction method described in any one of claims 1 to 8; Establishing a parking map based on the ORB feature points; During parking, load the established parking map; Update the map reference frame corresponding to the vehicle's current position in real time; Extracting ORB feature points of the current scene using the ORB feature point extraction method described in any one of claims 1 to 8; Matching the map reference frame with the extracted ORB feature points of the current scene; Positioning within the route based on matching information between the map reference frame and the extracted ORB feature points of the current scene; The vehicle is controlled based on the positioning information.