Device and method for position recognition of lidar sensor with limited viewing angle

Abstract

The present disclosure generally relates to a position recognition technology of an autonomous mobile robot and, more specifically, to a device and method for effective position recognition in a lidar sensor with a limited viewing angle. An operation method for position recognition of a lidar sensor with a limited viewing angle may comprise the steps of: dividing a point cloud into a distance-elevation region and an azimuth-elevation region; generating a first feature vector for the distance-elevation region and a second feature vector for the azimuth-elevation region; calculating the number of points corresponding to each region index of the first feature vector and the second feature vector; generating a weight vector by summing along an elevation axis and performing normalization; and generating a final feature vector for position recognition by inner product of the first feature vector, the second feature vector, and the weight vector.

Description

Device and method for position recognition of a LiDAR sensor with a limited field of view

[0001] The present disclosure generally relates to position recognition technology for autonomous robots, and more specifically to an apparatus and method for effective position recognition in a LiDAR sensor with a limited field of view. This research was conducted with funding from the Ministry of Science and ICT (MSIT) and support from the Institute of Information and Communication Technology Planning and Evaluation (IITP) (Project No.: RS-2022-II220448, 40%) and the National Research Foundation of Korea (NRF) (Project No.: RS-2025-02217000, 40%, Project No.: RS-2025-24803365, 20%).

[0002] LiDAR sensors are widely used for position recognition in autonomous robots and vehicles. In particular, they are attracting attention for their robustness against changes in lighting and visual conditions, unlike vision-based position recognition methods.

[0003] However, current LiDAR-based positioning algorithms are developed based on 360-degree omnidirectional data. These algorithms suffer from significant performance degradation in real-world situations where the field of view is limited due to various factors such as sensor fusion or sensor mounting.

[0004] Furthermore, due to the limited field of view, sufficient feature points necessary for location recognition cannot be secured, which presents a limitation that makes accurate map construction and location estimation difficult. This poses a significant constraint, particularly in correcting accumulated errors that occur when the robot revisits previously visited locations.

[0005] Based on the discussion above, the present disclosure provides an apparatus and method for effective position recognition even in a LiDAR sensor environment with a limited field of view.

[0006] In addition, the present disclosure provides an apparatus and method for representing a place as a distance-altitude region and an azimuth-altitude region through spatial organization.

[0007] In addition, the present disclosure provides an apparatus and method for representing a robust place through weighting based on vertical direction information.

[0008] In addition, the present disclosure provides an apparatus and method for lightweight position recognition capable of responding to rotational changes and determining an initial direction.

[0009] In addition, the present disclosure provides an apparatus and method for robust position recognition even in occlusion situations caused by physical shielding of the sensor.

[0010] According to various embodiments of the present disclosure, a method for operating to recognize the position of a LiDAR sensor having a limited field of view may include the steps of: dividing a point cloud into a distance-elevation region and an azimuth-elevation region; generating a first feature vector for the distance-elevation region and a second feature vector for the azimuth-elevation region; calculating the number of points corresponding to each region index of the first feature vector and the second feature vector; generating a weight vector by adding along an elevation axis and performing normalization; and generating a final feature vector for position recognition by taking the inner product of the first feature vector, the second feature vector, and the weight vector.

[0011] According to various embodiments of the present disclosure, an apparatus for position recognition of a LiDAR sensor having a limited field of view comprises: a LiDAR sensor receiving a point cloud; a processor; and a memory operably connected to the LiDAR sensor and the processor. The processor may divide the point cloud into a distance-elevation region and an azimuth-elevation region, generate a first feature vector for the distance-elevation region and a second feature vector for the azimuth-elevation region, calculate the number of points corresponding to each region index of the first feature vector and the second feature vector, generate a weight vector by adding along an elevation axis and performing normalization, and generate a final feature vector for position recognition by taking the inner product of the first feature vector, the second feature vector and the weight vector.

[0012] The apparatus and method according to various embodiments of the present disclosure enable accurate position estimation and map construction of an autonomous driving robot or vehicle by performing effective position recognition even at a limited field of view through spatially organized representation and weighting based on vertical direction information.

[0013] In addition, the apparatus and method according to various embodiments of the present disclosure provide a lightweight structure and a fast processing speed, thereby enabling real-time location recognition even on an onboard computer.

[0014] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below.

[0015] FIG. 1 shows a configuration diagram of a device for position recognition of a lidar sensor having a limited field of view according to one embodiment of the present disclosure.

[0016] FIG. 2 illustrates a LiDAR point cloud in various field-of-view limiting situations according to one embodiment of the present disclosure.

[0017] FIG. 3 illustrates a spatial configuration method for position recognition flexible to the viewing angle of the present disclosure and a feature vector generation process according to one embodiment of the present disclosure.

[0018] FIG. 4 illustrates the entire process of generating R-SOLiD and A-SOLiD from three-dimensional scan data of the present disclosure according to one embodiment of the present disclosure.

[0019] FIG. 5 illustrates a flowchart of a method for position recognition of a lidar sensor having a limited field of view according to one embodiment of the present disclosure.

[0020] FIG. 6 is a diagram showing the configuration of a device according to various embodiments of the present disclosure.

[0021] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of other embodiments. A singular expression may include a plural expression unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art described in this disclosure. Terms used in this disclosure that are defined in a general dictionary may be interpreted as having the same or similar meaning as they have in the context of the relevant technology, and are not to be interpreted in an ideal or overly formal sense unless explicitly defined in this disclosure. In some cases, even terms defined in this disclosure are not to be interpreted to exclude the embodiments of this disclosure.

[0022] In the various embodiments of the present disclosure described below, a hardware-based approach is described as an example. However, since the various embodiments of the present disclosure include techniques using both hardware and software, the various embodiments of the present disclosure do not exclude a software-based approach.

[0023] Additionally, in the detailed description and claims of the present disclosure, “at least one of A, B, and C” may mean “only A,” “only B,” “only C,” or “any combination of A, B, and C.” Additionally, “at least one of A, B, or C” or “at least one of A, B, and / or C” may mean “at least one of A, B, and C.”

[0024] The present disclosure relates to an apparatus and method for effective position recognition in a LiDAR sensor environment with a limited field of view. Specifically, the present disclosure describes a technique for performing robust position recognition even with a limited field of view through spatial organization and weighting based on vertical direction information.

[0025] FIG. 1 shows a configuration diagram of a device for position recognition of a lidar sensor having a limited field of view according to one embodiment of the present disclosure.

[0026] The device of the present invention may include a data acquisition and preprocessing unit (101), a spatially organized feature vector generation unit (Spatially Organized Descriptor) (103), and a position recognition and initial direction estimation unit (PR & Initial Heading) (105).

[0027] The data acquisition and preprocessing unit (101) acquires point cloud data from various types of LiDAR sensors. The acquired data undergoes a two-stage preprocessing process. First, distant points that are outside the maximum observation distance (Lmax) and near points that are very close to the sensor are removed. This is to remove unreliable data and increase processing efficiency. Second, downsampling is performed using a voxel grid of size 0.5m. This has the effect of equalizing the density of the point cloud and reducing computational complexity.

[0028] The spatial configuration-based feature vector generation unit generates two main feature vectors from the preprocessed point cloud. The first is REC (Radial-Elevational Points Counter), which represents the distance-elevation domain N r ×N e It is divided into bins, and the number of points included in each bin is calculated. The second is the AEC (Azimuthal-Elevational Points Counter), which defines the azimuth-elevation area as N a ×N e Calculate by dividing into bins. Here, N r =40, N a It is set to =60, and N e It is determined by the vertical resolution of the LiDAR sensor.

[0029] REC and AEC are expressed by the following mathematical formula 1.

[0030] [Mathematical Formula 1]

[0031]

[0032] In mathematical formula 1, REC and AEC are matrices storing the number of points in the distance-altitude / azimuth-altitude directions, Rik is the number of points in the bin corresponding to the distance direction index i and the altitude direction index k, Ajk is the number of points in the bin corresponding to the azimuth direction index j and the altitude direction index k, and [Nr] and [Ne] can be sets {1, 2, ..., Nr} and {1, 2, ..., Ne}, respectively.

[0033] Each index i, j, and k is determined according to the following mathematical formulas 2, 3, and 4.

[0034] [Mathematical Formula 2]

[0035]

[0036] [Mathematical Formula 3]

[0037]

[0038] [Mathematical Formula 4]

[0039]

[0040] In Equations 2, 3, and 4, Nr is the number of distance direction bins (set to 40), Na is the number of azimuth direction bins (set to 60), Ne is the number of elevation direction bins, Lmax is the maximum observable distance, Fu and Fd are the upper / lower vertical FOV angles of the lidar sensor, and R(·) may be a rounding function.

[0041] Here is calculated through mathematical formula 5.

[0042] [Mathematical Formula 5]

[0043]

[0044] In mathematical equation 5, x, y, and z are coordinates in the 3D Cartesian coordinate system of the point cloud, r is the distance from the origin, is the azimuth angle in the horizontal plane, can be the vertical elevation angle.

[0045] Subsequently, sum in the direction of altitude It generates, which is expressed by mathematical formula 6:

[0046] [Mathematical Formula 6]

[0047]

[0048] Here, is the sum vector in the elevation direction, : Sum of points in the k-th altitude bin, : It can be a sum in the direction of distance.

[0049] is calculated through mathematical formula 7.

[0050] [Mathematical Formula 7]

[0051]

[0052] In mathematical formula 7, is the Implicit Elevation Vector (IEV), , : Can be the minimum and maximum values ​​of the vector E.

[0053] In the weight rebalancing step, R-SOLiD and A-SOLiD are generated through Equation 8.

[0054] [Mathematical Formula 8]

[0055]

[0056] In mathematical equation 8, S is the final generated SOLiD feature vector, SR is R-SOLiD for Loop Detection, SA is A-SOLiD for Initial Heading, and D(·,·) can be a matrix multiplication operation.

[0057] The cosine distance for position recognition is calculated using mathematical formula 9.

[0058] [Mathematical Formula 9]

[0059]

[0060] In mathematical formula 9, is the R-SOLiD of the query frame, is the R-SOLiD, ∥·∥: L2 norm of the candidate frame.

[0061] The location recognition and initial direction estimation unit (105) utilizes the generated feature vectors. R-SOLiD performs fast location recognition using a kd-tree based database and Euclidean distance. Euclidean distance was chosen because it accurately reflects the overall similarity between feature vectors. A-SOLiD estimates the initial direction using Manhattan distance. Manhattan distance is more suitable for direction estimation because it compares direction differences element by element.

[0062] Through this configuration, the present disclosure enables real-time processing of 80Hz even in a limited field of view environment, and allows for effective position recognition and orientation estimation with feature vectors that are more than twice as light as existing methods.

[0063] FIG. 2 illustrates a LiDAR point cloud in various field-of-view limiting situations according to one embodiment of the present disclosure. Specifically, FIG. 2 shows various field-of-view limiting scenarios that may occur in a real environment.

[0064] The leftmost 360-degree omnidirectional scan data (201) represents the maximum amount of information that a LiDAR sensor can acquire in an ideal situation. This data includes structures and feature points in all directions centered on the sensor, providing the richest information for position recognition.

[0065] The second occlusion situation (203) refers to a case where a portion of the area is obscured by physical obstacles around the sensor or by the robot's mechanism. Such occlusion inevitably occurs depending on the sensor mounting position or the structural characteristics of the robot, resulting in the complete loss of data in a specific direction.

[0066] The third through fifth images (205) show situations where the field of view is progressively limited (180 degrees, 90 degrees, 60 degrees). Such limitations may occur in real-world situations such as the following:

[0067] - Limited field of view assigned to a specific sensor in a multi-sensor fusion system

[0068] - Limited sensor scan range due to system power consumption limitations

[0069] - Limitations of the sensor's physical scanning mechanism

[0070] - Intentional field of view limitation in special application environments

[0071]

[0072] As the field of view becomes more limited, the following technical problems arise:

[0073] - Difficulty in extracting place features due to partial loss of environmental information

[0074] - Decreased location recognition accuracy due to limited information

[0075] - Lack of reference points for direction estimation

[0076] - Performance degradation of existing 360-degree based algorithms

[0077]

[0078] The present disclosure provides a position recognition method that operates effectively even in such a limited field of view environment. In particular, it demonstrates superior performance compared to existing methods even in situations with a very limited field of view of 60 degrees or irregular occlusion. This is achieved through a spatial configuration method that effectively utilizes vertical direction information and an adaptive weighting method.

[0079] FIG. 3 illustrates a spatial configuration method for position recognition flexible to the viewing angle of the present disclosure and a feature vector generation process according to one embodiment of the present disclosure.

[0080] The spherical coordinate system diagram on the left shows 3D point cloud data in terms of distance (r) and azimuth ( ), altitude( It shows the process of breaking down into ) components.

[0081] Each point of the point cloud is in the spherical coordinate system (r, according to Equation 5) from the orthogonal coordinate system (x,y,z). , It is converted to ). In the spherical coordinate system, r is the distance from the sensor origin, is the rotation angle (azimuth) in the horizontal plane, represents the elevation angle in the vertical direction.

[0082] Points in the transformed spherical coordinate system are represented by dividing the space in two ways. The first is the generation of a Radial-Elevation Counter (REC), which is distance (r) - elevation ( ) Divide the plane into a grid and count the number of points included in each grid (bin). This is represented by the REC matrix in the upper right corner, where the rows are in the distance direction (r) and the columns are in the elevation direction ( It represents ) and the number in each cell means the number of points in the corresponding area.

[0083] The second is the generation of the AEC (Azimuth-Elevation Counter), which is the azimuth ( )-altitude( ) Divide the plane into a grid and count the number of points included in each grid. This is represented by the AEC matrix at the bottom right, where the rows are in the azimuth direction ( ), heat is in the altitude direction ( It represents ) and the number in each cell means the number of points in the corresponding area.

[0084] The generated REC and AEC produce the final feature vector through matrix multiplication with the IEV (Implicit Elevation Vector). R-SOLiD is generated through the matrix multiplication of REC and IEV and is a vector that compresses distance-direction features, used for Loop Detection. A-SOLiD is generated through the matrix multiplication of AEC and IEV and is a vector that compresses azimuth-direction features, used for Initial Heading estimation.

[0085] This dual-structure feature vector generation method enables stable feature extraction even with limited viewing angles, effectively utilizes vertical structure information, and allows for feature representation robust to rotational changes. Furthermore, the generation of lightweight final feature vectors provides computational efficiency capable of real-time processing.

[0086] FIG. 4 illustrates the entire process of generating R-SOLiD and A-SOLiD from three-dimensional scan data of the present disclosure according to one embodiment of the present disclosure.

[0087] Referring to FIG. 4, when 3D LiDAR scan data is input, it is first processed by projecting it in two directions. The first is Radial(r) Projection (401), which projects the data in the distance direction to generate a Radial-Elevation Counter (REC). In this process, each point of the point cloud is assigned to an appropriate bin according to distance and elevation, and the value of each bin represents the number of points existing in that area.

[0088] The second is Azimuthal( ) Projection (403) generates an Azimuth-Elevation Counter (AEC) by projecting data in the azimuth direction. In this process as well, each point is assigned to a bin according to its azimuth and elevation. In both projection processes, elevation ( ) Information is preserved and can maintain the characteristics of the vertical structure.

[0089] The generated REC data is summed in the distance (r) direction (405). This is a process of compressing the distribution of all points existing at each elevation into a single value. Subsequently, an Implicit Elevation Vector (IEV) is generated through Min-Max normalization. Normalization is performed according to Equation 7, and as a result, each element of the IEV has a value between 0 and 1. The IEV is a vector that effectively compresses and represents vertical structural features, and it preserves the characteristics of vertical structures such as buildings or trees.

[0090] Finally, two feature vectors are generated through a matrix product operation (407). An R-SOLiD (SR) of size 40×1 is generated through the matrix product of REC and IEV. This vector compressively represents information in the distance direction and is used as a feature vector for position recognition. Additionally, an A-SOLiD (SA) of size 60×1 is generated through the matrix product of AEC and IEV. This vector compressively represents information in the azimuth direction and is used as a feature vector for initial direction estimation.

[0091] This dual-structure feature vector generation process offers the following advantages. First, it can efficiently compress 3D spatial information while preserving key features necessary for localization. Second, by explicitly utilizing vertical structure information, stable feature extraction is possible even with a limited field of view. Third, high computational efficiency enables real-time processing. Fourth, the small size of the generated feature vectors minimizes memory usage. Fifth, localization and orientation estimation can be performed simultaneously through R-SOLiD and A-SOLiD.

[0092] FIG. 5 illustrates a flowchart of a method for position recognition of a lidar sensor having a limited field of view according to one embodiment of the present disclosure.

[0093] Referring to FIG. 5, in operation (501), a device (hereinafter “device”) for position recognition of a lidar sensor having a limited field of view can divide a point cloud into a distance-elevation region and an azimuth-elevation region. According to one embodiment, the point cloud can be downsampled using a voxel grid. The preprocessed points are converted into a spherical coordinate system using Equation 5, using the arctan2 function for numerical stability. Points that are outside the maximum observation distance or are too close to the sensor may be removed due to low reliability.

[0094] In operation (503), the device can generate a first feature vector (REC) for the distance-elevation region and a second feature vector (AEC) for the azimuth-elevation region. REC is a matrix of size NrXNe, where the distance direction is divided into Nr equal bins. AEC is a matrix of size NaXNe, where the azimuth is divided into equal intervals. Ne is determined by the number of vertical channels of the lidar and divides the elevation angle according to each channel interval.

[0095] In operation (505), the device can perform adaptive processing based on the field of view limitation. When the field of view is limited, the number of valid points decreases, so the size of the bin can be dynamically adjusted. Additionally, areas where occlusion occurs invalidate the corresponding bins to prevent incorrect matching.

[0096] In operation (507), the device can calculate the number of points included in each bin. At this time, for memory efficiency, it can be stored in the form of a sparse matrix, and bins without points may not be stored. Points at bin boundaries are assigned to only one bin to prevent double calculation.

[0097] In operation (509), the device can generate a weight vector (IEV) by adding along the elevation axis and performing normalization. A logarithmic scale may be used to prevent numerical overflow during the summing process according to Equation 6. Equation 7 is used during the normalization process, and the normalized values ​​may be passed through a softmax function to mitigate the influence of extreme values.

[0098] In operation (511), the device can generate a final feature vector according to Equation 8 by taking the inner product of the feature vectors and the weight vector. The generated feature vector is used for location recognition and direction estimation, and the cosine distance of Equation 9 can be used when calculating similarity with a database.

[0099] Through this series of processing steps, effective location recognition and direction estimation are possible even in environments with a limited field of view.

[0100] FIG. 6 is a diagram showing the configuration of a device according to various embodiments of the present disclosure.

[0101] Referring to FIG. 6, the device (600) may include at least one processor (610), a memory (620), and a communication device (630) that is connected to a network to perform communication. Additionally, the device (600) may further include an input interface device (640), an output interface device (650), a storage device (660), etc. Each component included in the device (600) may be connected by a bus (670) to communicate with one another.

[0102] However, each component included in the device (600) may be connected via individual interfaces or individual buses centered around the processor (610), rather than via a common bus (670). For example, the processor (610) may be connected via a dedicated interface to at least one of a memory (620), a communication device (630), an input interface device (640), an output interface device (650), and a storage device (660).

[0103] The processor (610) can execute a program command stored in at least one of the memory (620) and the storage device (660). The processor (610) may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. Each of the memory (620) and the storage device (660) may be composed of at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory (120) may be composed of at least one of read-only memory (ROM) and random access memory (RAM).

[0104] Methods according to the embodiments described in the claims or specification of the present disclosure may be implemented in the form of hardware, software, or a combination of hardware and software.

[0105] When implemented in software, a computer-readable storage medium may be provided for storing one or more programs (software modules). One or more programs stored in the computer-readable storage medium are configured for execution by one or more processors within an electronic device. One or more programs include instructions that cause the electronic device to execute methods according to the embodiments described in the claims or specification of this disclosure.

[0106] Such programs (software modules, software) may be stored in random access memory, non-volatile memory including flash memory, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic disc storage devices, compact disc-ROM (CD-ROM), digital versatile discs (DVDs), or other forms of optical storage devices, magnetic cassettes. Alternatively, they may be stored in memory composed of some or all of these. Additionally, each constituent memory may include multiple units.

[0107] Additionally, the program may be stored on an attachable storage device that can be accessed via a communication network such as the Internet, Intranet, LAN (local area network), WAN (wide area network), or SAN (storage area network), or a combination thereof. Such a storage device may be connected to a device performing an embodiment of the present disclosure through an external port. Additionally, a separate storage device on a communication network may be connected to a device performing an embodiment of the present disclosure.

[0108] In the specific embodiments of the present disclosure described above, the components included in the disclosure are expressed in a singular or plural form according to the specific embodiments presented. However, the singular or plural expression is selected to suit the situation presented for convenience of explanation, and the present disclosure is not limited to singular or plural components; even if a component is expressed in the plural form, it may be composed of a singular form, and even if a component is expressed in the singular form, it may be composed of a plural form.

[0109] Meanwhile, although specific embodiments have been described in the detailed description of the present disclosure, it is understood that various modifications are possible within the scope of the present disclosure. Therefore, the scope of the present disclosure should not be limited to the described embodiments, but should be defined by the claims set forth below as well as equivalents thereof.

Claims

1. A method of operation for position recognition of a LiDAR sensor having a limited field of view, The process of separating the point cloud into distance-altitude and azimuth-altitude regions, and A process of generating a first feature vector for the distance-altitude region and a second feature vector for the azimuth-altitude region, and A process of calculating the number of points corresponding to each region index of the first feature vector and the second feature vector, and The process of generating a weight vector by adding along the above elevation axis and performing normalization, and A method comprising the process of generating a final feature vector for position recognition by taking the inner product of the first feature vector, the second feature vector, and the weight vector.

2. The method of claim 1, further comprising the process of calculating similarity with a previously visited place using the generated final feature vector for location recognition.

3. The method of claim 1, wherein the process of distinguishing the area comprises calculating the distance, azimuth, and elevation for each point of the point cloud and assigning each point to the corresponding area based on the calculated values.

4. A method according to claim 1, wherein the weight vector is used to determine the importance of each region by reflecting spatial characteristics in the direction of the elevation axis.

5. The method of claim 2, further comprising the process of recognizing a place having a similarity greater than or equal to a threshold value as a revisited place during the similarity calculation process, and correcting the current position of the robot based on the revisited place recognition result.

6. In a device for position recognition of a LiDAR sensor having a limited field of view, LiDAR sensor receiving point clouds; processor; and The above-mentioned LiDAR sensor and memory operably connected to the above-mentioned processor, and the processor, The point cloud is divided into distance-altitude and azimuth-altitude regions, and Generate a first feature vector for the distance-altitude region and a second feature vector for the azimuth-altitude region, and Calculate the number of points corresponding to each region index of the first feature vector and the second feature vector, and A weight vector is generated by adding along the above elevation axis and performing normalization, and A device for generating a final feature vector for position recognition by taking the inner product of the first feature vector, the second feature vector, and the weight vector.

7. In Claim 6, The above processor is a device that calculates similarity with a previously visited place using the generated final feature vector for location recognition.

8. In Claim 6, A device in which the processor calculates distance, azimuth, and elevation for each point of the point cloud and assigns each point to a corresponding area based on the calculated values.

9. In Claim 6, The above weight vector is a device used to determine the importance of each region by reflecting spatial characteristics in the direction of the elevation axis.

10. In Claim 7, A device in which the processor recognizes a place having a similarity greater than or equal to a threshold value in the similarity calculation as a revisited place, and corrects the current position of the robot based on the revisited place recognition result.

Images