Device and method for autonomous driving position recognition using feature points of radar image and free space

Abstract

The present disclosure generally relates to position recognition of an autonomous driving system and, more specifically, to a device and method for recognizing a position of an autonomous driving vehicle by using feature points of a radar image and a free space. An operation method for recognizing a position of an autonomous driving vehicle including a radar sensor may comprise the steps of: extracting feature points from a radar image; dividing angular direction regions on the basis of the extracted feature points; generating a ratio vector by calculating a free space for each angular direction region; and calculating a similarity between a current location and a revisited location by using the ratio vector, wherein the free space refers to a space up to the extracted feature points.

Description

Device and method for autonomous driving position recognition using feature points of radar images and free space

[0001] The present disclosure generally relates to position recognition of an autonomous driving system, and more specifically, to an apparatus and method for recognizing the position of an autonomous vehicle using feature points of a radar image and free space. This research was conducted with funding from the Ministry of Science and ICT (MSIT) and support from the Institute of Information and Communication Technology Planning and Evaluation (IITP) (Project No.: RS-2022-II220448, 40%) and the National Research Foundation of Korea (NRF) (Project No.: RS-2025-02217000, 40%, Project No.: RS-2025-24803365, 20%).

[0002] In autonomous driving systems, location recognition plays a crucial role as a core function. In real-world autonomous driving environments, users encounter various weather conditions (e.g., cloudy, heavy rain, snow), and most sensors that utilize electromagnetic waves in the visible or near-visible light spectrum, such as cameras or LiDAR, are highly vulnerable to these adverse weather conditions, making reliable location recognition difficult.

[0003] On the other hand, radar has the advantage of being virtually unaffected by environmental changes and weather conditions due to the use of long electromagnetic waves. However, existing radar-based location recognition methods focused solely on scene compression and rotation or translation invariance, making it difficult to approach initial orientation estimation to correct accumulated attitude errors for back loops. Furthermore, location recognition on onboard computers was challenging due to the time required to generate descriptors or compare similarities between two descriptors.

[0004] Based on the discussion above, the present disclosure provides an apparatus and method for lightweight location recognition by extracting feature points and free space from radar images.

[0005] In addition, the present disclosure provides an apparatus and method for reducing the effects of multipath and speckle noise by utilizing free space information.

[0006] In addition, the present disclosure provides an apparatus and method for reliable place recognition by utilizing the mutually complementary characteristics of feature points and free space.

[0007] In addition, the present disclosure provides an apparatus and method for correcting attitude errors through initial orientation estimation and combining them with a SLAM pipeline.

[0008] In addition, the present disclosure provides an apparatus and method for real-time processing on an onboard computer through a lightweight descriptor with low computational cost.

[0009] According to various embodiments of the present disclosure, a method for operating to recognize the location of an autonomous vehicle including a radar sensor comprises the steps of: extracting feature points from a radar image; distinguishing angular direction regions based on the extracted feature points; generating a ratio vector by calculating free space for each angular direction region; and calculating a similarity between a current location and a revisited location using the ratio vector, wherein the free space may refer to the space up to the extracted feature points.

[0010] According to various embodiments of the present disclosure, an apparatus for recognizing the position of an autonomous vehicle including a radar sensor comprises a memory and a processor operably connected to the memory, wherein the processor is configured to extract feature points from a radar image, distinguish angular direction regions based on the extracted feature points, generate ratio vectors by calculating free space for each angular direction region, and calculate similarity between a current location and a revisited location using the ratio vectors, and the free space may refer to the space up to the extracted feature points.

[0011] The apparatus and method according to various embodiments of the present disclosure enable robust position recognition even under adverse weather conditions by generating a descriptor by simultaneously utilizing feature points and free space from a radar image.

[0012] In addition, the apparatus and method according to various embodiments of the present disclosure enable real-time location recognition even in environments with limited computing resources by using a lightweight descriptor and an efficient search method.

[0013] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below.

[0014] FIG. 1 is a drawing illustrating the overall concept of a position recognition system for an autonomous vehicle according to one embodiment of the present disclosure and the differences from existing technology.

[0015] FIG. 2 illustrates the overall system configuration and data processing pipeline of the present invention according to one embodiment of the present disclosure.

[0016] FIG. 3 is a diagram showing the initial direction estimation and matching processes sequentially in a position recognition system of the present disclosure according to one embodiment of the present disclosure.

[0017] FIG. 4 is a drawing showing the actual driving trajectory results to which the position recognition system of the present disclosure is applied, according to one embodiment of the present disclosure.

[0018] FIG. 5 is an overall flowchart of a position recognition method for an autonomous vehicle according to one embodiment of the present disclosure.

[0019] FIG. 6 is a diagram showing the configuration of a device according to various embodiments of the present disclosure.

[0020] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of other embodiments. A singular expression may include a plural expression unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art described in this disclosure. Terms used in this disclosure that are defined in a general dictionary may be interpreted as having the same or similar meaning as they have in the context of the relevant technology, and are not to be interpreted in an ideal or overly formal sense unless explicitly defined in this disclosure. In some cases, even terms defined in this disclosure are not to be interpreted to exclude the embodiments of this disclosure.

[0021] In the various embodiments of the present disclosure described below, a hardware-based approach is described as an example. However, since the various embodiments of the present disclosure include techniques using both hardware and software, the various embodiments of the present disclosure do not exclude a software-based approach.

[0022] Additionally, in the detailed description and claims of the present disclosure, “at least one of A, B, and C” may mean “only A,” “only B,” “only C,” or “any combination of A, B, and C.” Additionally, “at least one of A, B, or C” or “at least one of A, B, and / or C” may mean “at least one of A, B, and C.”

[0023] The present disclosure relates to an apparatus and method for utilizing feature points of a radar image and free space in position recognition of an autonomous driving system. Specifically, the present disclosure describes a technology for performing robust and lightweight position recognition by utilizing the mutually complementary characteristics of feature points and free space.

[0024] FIG. 1 is a drawing illustrating the overall concept of a position recognition system for an autonomous vehicle according to one embodiment of the present disclosure and the differences from existing technology.

[0025] Referring to FIG. 1, the overall map (101) shown on the left side of FIG. 1 is generated by accumulating point cloud data acquired from radar sensors while the autonomous vehicle is driving. In this map, the dots marked in white represent the reflection points of objects detected by the radar, which include location information of fixed structures such as building corners, trees, pillars, and signs. The parts marked in purple represent the current location of the autonomous vehicle, and the green lines represent the trajectory of the vehicle's movement.

[0026] Radar images used for location recognition are displayed in the center (103) of FIG. 1. The Query image at the top is an image acquired from the current location, and the Candidate image at the bottom is an image acquired from a previously visited location. The radar images are expressed in a polar coordinate system containing distance and angle information, where the horizontal axis represents the angle (from -180 degrees to 180 degrees) and the vertical axis represents the distance (from 0m to the maximum measurement distance). In the image, the brightness value represents the reflection intensity of the radar signal, and the brighter the part, the stronger the reflection.

[0027] In particular, the areas highlighted in yellow represent false signals caused by the Multipath phenomenon. Multipath is a phenomenon where radar signals are reflected sequentially by multiple objects and received via a detour, causing reflection points to be generated at incorrect locations rather than the actual positions of the objects. This Multipath phenomenon occurs particularly frequently in urban environments and is a major cause of significantly reduced location recognition accuracy.

[0028] The right side (103) of FIG. 1 visualizes the results of a position recognition algorithm. The top side shows the results of applying the method of the present disclosure (Ours), and the bottom side shows the results of applying the conventional Radar SC method. The conventional Radar SC method uses a method of generating descriptors by directly resizing radar images. This method has the advantage of being simple to implement, but it reflects radar-specific noise, such as multipath or speckle noise, directly into the descriptors. As a result, position recognition fails (Loop Failed) as shown in the bottom result.

[0029] In contrast, the method of the present disclosure proposes a new descriptor that simultaneously utilizes feature points and free space information. Feature points represent the locations where actual objects exist, while free space refers to empty spaces where no objects exist. Since free space information has characteristics opposite to multipath noise, including it in the descriptor can effectively cancel out the influence of noise. In this way, successful location recognition (Loop Success) is possible, as shown in the results above.

[0030] Figure 1 exemplarily uses a lightweight descriptor of size 42X1. This is significantly smaller than existing methods, and can reduce memory usage by at least 4 to up to 600 times compared to existing methods such as Radar SC (20X60=1,616 bytes), RaPlace (3,008 bytes), and RadVLAD (32768X1=262,272 bytes). Due to these lightweight characteristics, real-time (4-5Hz) processing is possible even in embedded systems with limited computing power (quad-core ARM CPU) and memory (4GB RAM), such as the Jetson Nano shown in the center of Figure 1.

[0031] As such, the present disclosure provides a new location recognition method that overcomes the limitations of existing methods and simultaneously satisfies the robustness and real-time capabilities required in actual autonomous driving environments.

[0032] FIG. 2 illustrates the overall system configuration and data processing pipeline of the present invention according to one embodiment of the present disclosure.

[0033] Referring to FIG. 2, the system (or device) of the present disclosure may largely include an Input & Feature Extraction module (201), a Frontend ReFeree module (203), and a Backend (205).

[0034] The Input & Feature Extraction module (201) processes the original image acquired from the radar sensor. The radar image has a polar coordinate system of size HXW, and the value of each pixel represents the radar cross-section (RCS) value. The RCS value is the reflection coefficient of the target, representing the ratio of reflected power to incident power density. H represents the height associated with the angle (from -180 degrees to 180 degrees), and W represents the width associated with the distance (from 0m to the maximum measurement distance). This radar image I It is defined as.

[0035] In the feature point extraction process, the free space identifier function Ψ(p) is first defined.

[0036] [Mathematical Formula 1]

[0037]

[0038] In the signal processing stage, a linear uniform filter suitable for onboard computing is used instead of the conventional non-linear Gaussian filter. The filter size is adaptively adjusted according to distance; a 3x3 filter is applied at close ranges, and a 5x5 filter at long ranges, reflecting changes in signal characteristics with distance. Radar image with feature points extracted through this linear filter processing Creates.

[0039] The ReFeree module is a core part of the system and generates two descriptors called R-ReFeree and A-ReFeree. R-ReFeree is a descriptor that provides rotational invariance and is defined as in Equation 2. represents the ratio of free space in each block divided in the distance direction, and is the size of the distance direction block, is a descriptor index vector with the range [1, 2, ..., Nw]. Here, Nw is It is calculated to create a total of 42 indexes.

[0040] [Mathematical Formula 2]

[0041]

[0042] A-ReFeree is a descriptor for direction estimation and is defined as Equation 3. represents free space information in blocks divided in the angular direction, and is the size of the angle direction block, is an angle direction descriptor index vector. In particular is an index representing the distance to the farthest feature point from the i-th row, utilizing the transmission characteristics of radar to consider the space behind the object.

[0043] [Mathematical Formula 3]

[0044]

[0045] Location recognition in the backend (205) is performed by L2-distance-based matching using a KD-Tree, which is expressed as Equation 4. Here represents the L2-norm distance between the Query descriptor Rq and the Candidate descriptor Rc. The optimal candidate location c * As shown in mathematical formula 5, the location having the smallest L2 distance within the distance threshold τ is selected.

[0046] [Mathematical Formula 4]

[0047]

[0048] [Mathematical Formula 5]

[0049]

[0050] Initial direction estimation is performed through A-ReFeree. In Equation 6 represents the optimal angular direction index calculated using the cosine distance function d(·,·). This estimated direction is the rotation matrix according to Equation 7. It is converted to, where 360 / N h represents the angle per block in the angle direction.

[0051] [Mathematical Formula 6]

[0052]

[0053] [Mathematical Formula 7]

[0054]

[0055] During the registration process, the initially sorted Query point cloud is obtained through mathematical formula 8. Calculate . Precise alignment is performed using the Nano-GICP (Generalized ICP) algorithm ξ as shown in Equations 9 and 10, which includes an efficient KD-Tree construction utilizing the NanoFLANN library.

[0056] [Mathematical Formula 8]

[0057]

[0058] [Mathematical Formula 9]

[0059]

[0060] [Mathematical Formula 10]

[0061]

[0062] Finally, Graph Optimization is performed through Equations 11 and 12. Here is the optimization function, the odometry factor and loop combination factors Consider all of them. is the radar odometry estimate, and They represent the uncertainty for the odometry factor and the loop coupling factor, respectively.

[0063] [Mathematical Formula 11]

[0064]

[0065] [Mathematical Formula 12]

[0066]

[0067] The experimental results show that the position recognition performance of this system is as follows. Recall@1 represents the proportion of times the system found the correct position on the first estimation, achieving 0.723 in the DCC sequence. This indicates that the system recognized the correct position on the first attempt in 72.3% of cases. AUC (Area Under the Curve) is the area under the Precision-Recall curve, representing the system's overall position recognition accuracy, and achieved 0.936. Here, Precision refers to the proportion of actual correct positions among those found by the system, and Recall refers to the proportion of positions found by the system among all actual positions.

[0068] In terms of processing time, descriptor generation took an average of 0.069 seconds, demonstrating that real-time processing of over 4Hz is possible on the Jetson Nano. This represents superior performance compared to existing methods such as RadVLAD (0.081 seconds), FFT-RadVLAD (0.088 seconds), Radar SC (0.368 seconds), and RaPlace (0.359 seconds). Furthermore, stable direction estimation was possible with an average rotation error of 23.32 degrees in reverse driving situations, and the final Absolute Pose Error (APE) showed an improvement of approximately 15 times, decreasing from an average odometry value of 53.336m to 3.493m.

[0069] FIG. 3 is a diagram showing the initial direction estimation and matching processes sequentially in a position recognition system of the present disclosure according to one embodiment of the present disclosure. In particular, it shows the matching process between the 1492nd frame and the 1041st frame of the DCC 01 sequence.

[0070] Figure 3(a) shows the initial state. The red dots represent the Query point cloud acquired at the current location (frame 1492), and the green dots represent the Candidate point cloud acquired at the previously visited location (frame 1041). In this state, direct alignment is difficult because the two point clouds are facing in different directions.

[0071] Figure 3(b) shows the result of applying the initial direction calculated through the A-ReFeree descriptor to the Query point cloud. The initial direction estimation is performed using Equations 6 and 7. Specifically, the optimal rotation angle is estimated by calculating the cosine distance between the A-ReFeree descriptors, and this is converted into a rotation matrix and applied to the Query point cloud. As a result, the Query point cloud (red) is aligned in a direction similar to the Candidate point cloud (green).

[0072] FIG. 3(c) shows the final result of precise alignment using the Nano-GICP (generalized iterative closest point) algorithm with the initial direction applied. This process is performed according to Equations 9 and 10 and is processed at high speed through an efficient KD-Tree construction using the NanoFLANN library. As a result of the precise alignment, it can be confirmed that the two point clouds overlap accurately, which demonstrates that the position recognition system of the present disclosure is capable of accurate position matching even in reverse driving situations.

[0073] In particular, the present disclosure was able to estimate the initial direction with an average rotation error of 23.32 degrees in the DCC 01 sequence, which is a performance capability that existing methods cannot provide. Such accurate initial direction estimation is a key factor in increasing the success rate of the subsequent precision matching process and reducing computation time. Furthermore, stable matching is possible despite noise and multipath phenomena in the radar sensor, which demonstrates the effectiveness of the method proposed in the present disclosure that simultaneously utilizes feature points and free space.

[0074] FIG. 4 is a diagram showing actual driving trajectory results to which the position recognition system of the present disclosure is applied, according to one embodiment of the present disclosure. Specifically, the results obtained from the KAIST sequence of the Mulran dataset show a comparison of odometry (Odom), the method of the present invention (Ours), and the actual position (GT, Ground Truth).

[0075] The X and Y axes represent positions on a two-dimensional plane in meters (m), and the vehicle traveled within an area of ​​approximately 600m x 600m. The odometry trajectory, indicated by the red line, shows that the position estimation error accumulates over time, causing a significant deviation from the actual path (gray line). In particular, a rapid increase in error is observed in the long straight section at the bottom left and the curved section on the right.

[0076] On the other hand, the method of the present invention, indicated by the blue line, shows a trajectory very similar to the actual path. This means that the position recognition system of the present invention effectively corrects cumulative errors. In particular, position errors are successfully corrected at points where loop closure occurs, so it can be confirmed that the overall trajectory matches the actual path well.

[0077] The superiority of the present invention was also demonstrated in quantitative evaluations. While the average position error (APE) of odometry was 53.336 m, the method of the present disclosure showed an accuracy improved by approximately 15 times to 3.493 m. The maximum position error also decreased significantly from 95.059 m of odometry to 9.005 m. This performance improvement demonstrates that the feature point and free-space-based position recognition method proposed in the present disclosure operates effectively in actual autonomous driving environments.

[0078] FIG. 5 is an overall flowchart of a position recognition method for an autonomous vehicle according to one embodiment of the present disclosure.

[0079] Referring to FIG. 5, the device for position recognition of an autonomous vehicle of the present disclosure (hereinafter "device") may first perform a process (501) of extracting feature points from a radar image. In the feature point extraction process, a radar image in the form of a polar coordinate system of size H / XW acquired from a radar sensor is processed. At this time, each pixel of the radar image has a Radar Cross Section (RCS) value, which represents the reflection characteristics of an object. In the signal processing process, a linear uniform filter is applied to remove radar-specific noise such as abnormal return values, speckle noise, and multipath reflections. The size of the filter is adaptively adjusted according to distance, and a size of 3 / X3 is used for short distances and 5 / X5 for long distances.

[0080] Next, the device can perform a process (503) of distinguishing angular direction regions based on extracted feature points. This is a process of dividing the angular direction (-180 degrees to 180 degrees) of the radar image into multiple regions, each region reflecting the distribution characteristics of the feature points. The size of each region is determined by the size α of the angular direction block, which is set considering the accuracy of direction estimation and computational efficiency.

[0081] Next, the device may perform a process (505) of generating a ratio vector by calculating free space for each angular direction region. Here, free space refers to the space up to the feature point, and as in claim 2, the ratio of free space to the distance to the feature point is calculated for each angular direction region. This ratio is used as a descriptor that reflects the geometric characteristics of the corresponding region.

[0082] Finally, the device can perform a process (507) of calculating the similarity between the current location and the revisited location using a ratio vector. According to one embodiment, the operation (507) is efficiently performed through a KD-tree search. A KD-tree is a data structure capable of rapidly performing a nearest neighbor search of high-dimensional data, and identifies the most similar location through L2-distance-based similarity calculation.

[0083] In addition, according to one embodiment, a process of estimating the initial direction between the current location and the revisited location based on a ratio vector can be additionally performed. The initial direction estimation uses an A-ReFeree descriptor and calculates the optimal rotation angle through a cosine distance function. The initial direction estimated in this way serves to increase the success rate of the subsequent precision matching process and reduce computation time.

[0084] Through this series of processes, the method of the present disclosure enables accurate position recognition while being robust against noise in radar sensors, and provides stable performance, especially in reverse driving situations. Experimental results demonstrated excellent performance with a Recall@1 performance of 72.3% and an average rotation error of 23.32 degrees in the DCC sequence.

[0085] FIG. 6 is a diagram showing the configuration of a device according to various embodiments of the present disclosure.

[0086] Referring to FIG. 6, the device (600) may include at least one processor (610), a memory (620), and a communication device (630) that is connected to a network to perform communication. Additionally, the device (600) may further include an input interface device (640), an output interface device (650), a storage device (660), etc. Each component included in the device (600) may be connected by a bus (670) to communicate with one another.

[0087] However, each component included in the device (600) may be connected via individual interfaces or individual buses centered around the processor (610), rather than via a common bus (670). For example, the processor (610) may be connected via a dedicated interface to at least one of a memory (620), a communication device (630), an input interface device (640), an output interface device (650), and a storage device (660).

[0088] The processor (610) can execute a program command stored in at least one of the memory (620) and the storage device (660). The processor (610) may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. Each of the memory (620) and the storage device (660) may be composed of at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory (620) may be composed of at least one of read-only memory (ROM) and random access memory (RAM).

[0089] Methods according to the embodiments described in the claims or specification of the present disclosure may be implemented in the form of hardware, software, or a combination of hardware and software.

[0090] When implemented in software, a computer-readable storage medium may be provided for storing one or more programs (software modules). One or more programs stored in the computer-readable storage medium are configured for execution by one or more processors within an electronic device. One or more programs include instructions that cause the electronic device to execute methods according to the embodiments described in the claims or specification of this disclosure.

[0091] Such programs (software modules, software) may be stored in random access memory, non-volatile memory including flash memory, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic disc storage devices, compact disc-ROM (CD-ROM), digital versatile discs (DVDs), or other forms of optical storage devices, magnetic cassettes. Alternatively, they may be stored in memory composed of some or all of these. Additionally, each constituent memory may include multiple units.

[0092] Additionally, the program may be stored on an attachable storage device that can be accessed via a communication network such as the Internet, Intranet, LAN (local area network), WAN (wide area network), or SAN (storage area network), or a combination thereof. Such a storage device may be connected to a device performing an embodiment of the present disclosure through an external port. Additionally, a separate storage device on a communication network may be connected to a device performing an embodiment of the present disclosure.

[0093] In the specific embodiments of the present disclosure described above, the components included in the disclosure are expressed in a singular or plural form according to the specific embodiments presented. However, the singular or plural expression is selected to suit the situation presented for convenience of explanation, and the present disclosure is not limited to singular or plural components; even if a component is expressed in the plural form, it may be composed of a singular form, and even if a component is expressed in the singular form, it may be composed of a plural form.

[0094] Meanwhile, although specific embodiments have been described in the detailed description of the present disclosure, it is understood that various modifications are possible within the scope of the present disclosure. Therefore, the scope of the present disclosure should not be limited to the described embodiments, but should be defined by the claims set forth below as well as equivalents thereof.

Claims

1. A method of operation for recognizing the position of an autonomous vehicle including a radar sensor, The process of extracting feature points from radar images, and A process of distinguishing angular direction regions based on the above-mentioned extracted feature points, and The process of generating a ratio vector by calculating free space for each of the above-mentioned angle direction regions, and It includes a process of calculating the similarity between the current location and the revisited location using the above ratio vector, and The above free space refers to the space up to the above extracted feature points, method 2. The method of claim 1, wherein the ratio vector is calculated as the ratio of free space to the distance to the feature point for each angular direction region.

3. A method according to claim 1, wherein the calculation of similarity between the current location and the revisited location is performed through KD-tree search.

4. A method according to Claim 1, further comprising a process of estimating the initial direction between the current location and the revisited location based on the ratio vector.

5. A device for recognizing the position of an autonomous vehicle including a radar sensor, It includes a memory and a processor operably connected to the memory, The above processor is, A device configured to extract feature points from a radar image, distinguish angular direction regions based on the extracted feature points, generate ratio vectors by calculating free space for each angular direction region, and calculate similarity between a current location and a revisited location using the ratio vectors, wherein the free space refers to the space up to the extracted feature points.

6. An apparatus according to claim 5, wherein the ratio vector is calculated as the ratio of free space to the distance to the feature point for each angular direction region.

7. An apparatus according to claim 5, wherein the calculation of similarity between the current location and the revisited location is performed through KD-tree search.

8. The apparatus of claim 5, wherein the processor is further configured to estimate the initial direction between the current location and the revisited location based on the ratio vector.

Images