Method and apparatus for aligning target point cloud data and source point cloud data by using alignment algorithm selected on basis of number of matching points between target point cloud data and source point cloud data

Abstract

Provided is a data processing method that involves: acquiring source point cloud data and target point cloud data for a space; selecting an alignment algorithm for aligning two point cloud data sets on the basis of the number of matching feature pairs according to feature matching between the source point cloud data and the target point cloud data; and performing data alignment by using the selected alignment algorithm.

Description

A method and device for matching target point cloud data and source point cloud data using a matching algorithm selected based on the number of matching points between the target point cloud data and the source point cloud data.

[0001] Embodiments relate to a method and device for matching target point cloud data and source point cloud data, and to a method and device for matching target point cloud data and source point cloud data using a matching algorithm selected based on the number of matching points between the target point cloud data and the source point cloud data.

[0002] Estimating the pose of a mapping device, such as a mapping robot that includes a sensor unit such as a lidar sensor (i.e., the pose of the lidar sensor or 'lidar pose'), with respect to a space is very important in implementing autonomous driving of a mobile device such as a robot or implementation of augmented reality content for a space. For example, a mobile device including a lidar sensor drives while recognizing the surrounding environment within a space using the lidar sensor. At this time, if the lidar pose is not accurate, the surrounding environment cannot be accurately recognized, which may cause problems in the driving of the mobile device. If the lidar pose is accurately estimated, the performance (accuracy, range, etc.) of the lidar sensor can be improved.

[0003] In order to estimate such a lidar pose, it is required to align target point cloud data generated based on sensor data acquired from a moving object (mapping device) including a lidar sensor with source point cloud data that serves as a reference for space.

[0004] However, when there are few matching feature pairs (i.e., matching points) between target point cloud data sets to be aligned due to environmental characteristics of the space, etc., it is difficult to accurately match these target point cloud data sets, which leads to inaccurate estimation of the lidar pose.

[0005] Korean Patent Publication No. 10-2016-0116862 (published on October 10, 2016) discloses a device and method for automatically aligning building polygons on multiple maps.

[0006] The information described above is for the purpose of understanding only and may contain contents that do not form part of the prior art.

[0007] A data processing method can be provided that acquires source point cloud data and target point cloud data for a space, selects a matching algorithm for matching two point cloud data sets based on the number of matching feature pairs according to feature matching between the source point cloud data and the target point cloud data, and performs data matching using the selected matching algorithm.

[0008] In a case where the number of matching feature pairs according to feature matching between source point cloud data and target point cloud data is small, a first matching algorithm that matches point cloud data at a first degree of freedom level is selected from among a plurality of matching algorithms, but in a case where the number of matching feature pairs according to feature matching is large, a second matching algorithm that matches point cloud data at a second degree of freedom level higher than the first degree of freedom level is selected, thereby providing a data processing method that performs matching.

[0009] In one aspect, a data processing method performed by a computer system is provided, comprising: acquiring first point cloud data for a space in which a moving object travels; acquiring second point cloud data generated based on sensing data from the moving object for the space; performing feature matching between the first point cloud data and the second point cloud data; selecting a matching algorithm for matching the second point cloud data to the first point cloud data based on the number of matching feature pairs between the first point cloud data and the second point cloud data acquired according to the feature matching; and matching the second point cloud data to the first point cloud data using the selected matching algorithm.

[0010] The above data processing method may further include a step of estimating a pose of the moving object based on alignment of the second point cloud data with the first point cloud data.

[0011] The first point cloud data may be at least a portion of source point cloud data for the space that serves as a reference for matching with the second point cloud data, and the second point cloud data may be at least a portion of target point cloud data for the space generated based on the sensing data acquired by a sensor unit of the mobile body while the mobile body moves through the space.

[0012] The above selecting step may include, if the number of the matching feature pairs obtained through the feature matching is less than a first number, selecting a first matching algorithm that matches the second point cloud data to the first point cloud data at a first degree of freedom level among a plurality of matching algorithms, and if the number of the matching feature pairs obtained through the feature matching is greater than or equal to the first number, selecting a second matching algorithm that matches the second point cloud data to the first point cloud data at a second degree of freedom level higher than the first degree of freedom level among the plurality of matching algorithms.

[0013] The first matching algorithm may be configured to match the second point cloud data to the first point cloud data by estimating the first degree of freedom level, which is 4 degrees of freedom (DoF), and the second matching algorithm may be configured to match the second point cloud data to the first point cloud data by estimating the second degree of freedom level, which is 6 degrees of freedom (6 DoF).

[0014] When the second point cloud data is aligned with the first point cloud data by the first alignment algorithm, a 4-degree-of-freedom pose can be estimated as the pose of the moving object, and when the second point cloud data is aligned with the first point cloud data by the second alignment algorithm, a 6-degree-of-freedom pose can be estimated as the pose of the moving object.

[0015] The first matching algorithm may be configured to match the second point cloud data to the first point cloud data by estimating a first degree of freedom variable representing a rotation in the x-axis direction, a second degree of freedom variable representing a rotation in the y-axis direction, a third degree of freedom variable representing a rotation in the z-axis direction, and a fourth degree of freedom variable representing a rotation in the yaw direction as the four degrees of freedom, and the second matching algorithm may be configured to match the second point cloud data to the first point cloud data by further estimating a fifth degree of freedom variable representing a rotation in the roll direction and a sixth degree of freedom variable representing a rotation in the pitch direction in addition to the first to fourth degree of freedom variables as the six degrees of freedom.

[0016] The first matching algorithm may be configured to match the second point cloud data to the first point cloud data under a first environmental condition that assumes that the first point cloud data and the second point cloud data are on the same plane.

[0017] At least one of the first matching algorithm and the second matching algorithm may be configured to match the second point cloud data to the first point cloud data under a second environmental condition that the space is composed of at least one rectangular parallelepiped object.

[0018] The above-described selecting step selects a matching algorithm that matches the second point cloud data to the first point cloud data by estimating a greater number of degree-of-freedom variables among a plurality of degree-of-freedom variables as the number of matching feature pairs obtained according to the feature matching increases, and the plurality of degree-of-freedom variables may include a first degree-of-freedom variable representing a rotation in the x-axis direction, a second degree-of-freedom variable representing a rotation in the y-axis direction, a third degree-of-freedom variable representing a rotation in the z-axis direction, a fourth degree-of-freedom variable representing a rotation in the yaw direction, a fifth degree-of-freedom variable representing a rotation in the roll direction, and a sixth degree-of-freedom variable representing a rotation in the pitch direction.

[0019] The step of performing the feature matching may include: extracting a plurality of feature points from each of the first point cloud data and the second point cloud data; and determining matching feature pairs between the first point cloud data and the second point cloud data by matching feature points extracted from the first point cloud data with feature points extracted from the second point cloud data.

[0020] The selecting step may include, if the number of the matching feature pairs is less than a first number, selecting a first matching algorithm that matches the second point cloud data to the first point cloud data at a first degree of freedom level from among a plurality of matching algorithms, and if the number of the matching feature pairs obtained through the feature matching is greater than or equal to the first number, selecting a second matching algorithm that matches the second point cloud data to the first point cloud data at a second degree of freedom level higher than the first degree of freedom level from among the plurality of matching algorithms, wherein the first number may be a value determined based on at least one of a ratio of first valid feature points among first feature points extracted from the first point cloud data and a ratio of second valid feature points among second feature points extracted from the second point cloud data, and a ratio of first matching points used to determine the matching feature pairs among the first valid feature points and a ratio of second matching points used to determine the matching feature pairs among the second valid feature points.

[0021] In another aspect, a computer system for performing a data processing method is provided, comprising at least one processor configured to execute computer-readable instructions contained in a memory, wherein the at least one processor obtains first point cloud data for a space in which a moving object travels, obtains second point cloud data generated based on sensing data for the space from the moving object, performs feature matching between the first point cloud data and the second point cloud data, and selects a matching algorithm for matching the second point cloud data to the first point cloud data based on the number of matching feature pairs between the first point cloud data and the second point cloud data obtained according to the feature matching, and matches the second point cloud data to the first point cloud data using the selected matching algorithm.

[0022] According to the data processing method of the embodiment, when matching target point cloud data for a space to source point cloud data for the space, a matching algorithm having a level of freedom suitable for performing the matching can be selected according to the number of matching feature pairs between two target point cloud data sets.

[0023] When the number of matching feature pairs between two target point cloud data sets is relatively small, a matching algorithm that matches point cloud data with a lower degree of freedom can be selected, and thus, the occurrence of extreme errors in matching can be prevented.

[0024] By improving the accuracy of alignment between point cloud data sets, the accuracy of the estimated lidar pose can be improved accordingly.

[0025] FIG. 1 illustrates a data processing method for matching target point cloud data (second point cloud data) to source point cloud data (first point cloud data) performed by a computer system according to one embodiment.

[0026] FIG. 2 illustrates a computer system performing a data processing method according to one embodiment.

[0027] FIG. 3 is a flowchart illustrating a data processing method for aligning second point cloud data with first point cloud data and estimating a pose of a moving object, according to one embodiment.

[0028] FIG. 4 is a flowchart illustrating a method for selecting an appropriate matching algorithm based on the number of matching feature pairs between first point cloud data and second point cloud data, according to an example.

[0029] FIG. 5 illustrates a method for selecting an appropriate matching algorithm and estimating the pose of a moving object based on the number of matching feature pairs between first point cloud data and second point cloud data, according to an example.

[0030] Figure 6 illustrates source point cloud data and target point cloud data according to an example.

[0031] FIG. 7 illustrates a method for extracting feature points (i.e., performing feature extraction) from each of source point cloud data and target point cloud data, according to an example.

[0032] Figures 8 and 9 illustrate a method for selecting an appropriate matching algorithm based on the result of feature matching, according to an example.

[0033] Hereinafter, various embodiments of this document are described with reference to the attached drawings.

[0034]

[0035] FIG. 1 illustrates a data processing method for matching target point cloud data (second point cloud data) to source point cloud data (first point cloud data) performed by a computer system according to one embodiment.

[0036] Referring to FIG. 1, a method for matching target point cloud data (30) (hereinafter also referred to as second point cloud data (30)) generated based on sensing data (i.e., sensor data) for the corresponding space from a moving object, not shown, with respect to source point cloud data (20) (hereinafter also referred to as first point cloud data (20)) that serves as a reference for the space will be described.

[0037] The space represented by the source point cloud data (20) and the target point cloud data (30) may represent an outdoor or indoor environment, or may represent a composite space combining indoors and outdoors.

[0038] The source point cloud data (20) may be point cloud data for a space that serves as a reference for alignment. The source point cloud data (20) may be a pre-built 3D model for the space or data constituting such a 3D model. Alternatively, the source point cloud data (20) may be a pre-built 3D map for the space or data constituting such a 3D map.

[0039] Meanwhile, target point cloud data (30) may be input point cloud data that is the target of matching with source point cloud data (20). Such target point cloud data (30) may be generated based on sensor data acquired as space is sensed by a moving object (e.g., mapping equipment, etc.) including a sensor unit (e.g., lidar sensor, etc.).

[0040] The source point cloud data (20) and the target point cloud data (30) can each represent a data set of point clouds.

[0041] In an embodiment, target point cloud data (30) can be aligned with source point cloud data (20) corresponding to a map of a space in which the target point cloud data (30) has been generated, and accordingly, the pose of the moving object (i.e., the lidar sensor included in the moving object) can be estimated.

[0042] The alignment of the above data and estimation of the pose can be performed by a computer system (100).

[0043] The computer system (100) may be a server or a computer system constituting a server. The computer system (100) may obtain target point cloud data (30) from a mobile object by communicating with the mobile object that acquires sensor data while traveling through space, or may generate target point cloud data (30) based on sensor data acquired from the mobile object. The computer system (100) may align the target point cloud data (30) with the source point cloud data (20) and estimate the pose of the mobile object.

[0044] Alternatively, the computer system (100) may be a computer system included in a mobile body (e.g., a robot). In this case, the robot may have source point cloud data (20) and may be configured to generate target point cloud data (30) based on acquired sensor data, align the target point cloud data (30) with the source point cloud data (20), and estimate a pose.

[0045] Alternatively, the computer system (100) may be a computer system included in a robot control system for controlling a mobile object (e.g., a robot). In this case, the robot control system can, similarly to the case of the server described above, align target point cloud data (30) with source point cloud data (20) and estimate the pose of the mobile object.

[0046] In an embodiment, in order to align target point cloud data (30) to source point cloud data (20), feature matching may first be performed between the target point cloud data (30) and the source point cloud data (20). As a result of the feature matching, the number of matching feature pairs (i.e., matching feature points) between the target point cloud data (30) and the source point cloud data (20) may be determined. The computer system (100) may select a matching algorithm to be used for aligning the target point cloud data (30) and the source point cloud data (20) based on the number of such matching feature pairs (i.e., matching feature points). The computer system (100) may align the target point cloud data (30) to the source point cloud data (20) using the selected matching algorithm.

[0047] A matching algorithm may refer to a matching method, matching method, or matching process performed to match point cloud data sets. The computer system (100) can use this matching algorithm to estimate a certain degree of freedom to match point cloud data sets. In other words, feature points included in each matching feature pair between target point cloud data (30) and source point cloud data (20) can be estimated along with their degrees of freedom. These estimated degrees of freedom may vary depending on the type of matching algorithm selected by the computer system (100).

[0048] The computer system (100) may select a matching algorithm for matching the target point cloud data (30) to the source point cloud data (20) based on the number of matching feature pairs between the source point cloud data (20) and the target point cloud data (30) obtained through feature matching (for example, based on a value based on the number). For example, the computer system (100) may select a matching algorithm that estimates a relatively lower degree of freedom when the number of matching feature pairs is less than a certain level, and may select a matching algorithm that estimates a relatively higher degree of freedom when the number of matching feature pairs is greater than a certain level. Thus, in the embodiment, even when the number of matching feature pairs between the source point cloud data (20) and the target point cloud data (30) is small, occurrence of extreme errors in matching can be prevented, thereby increasing the accuracy of data matching.

[0049] That is, in the embodiment, based on the result of feature matching between the source point cloud data (20) and the target point cloud data (30), the cloud point data sets can be aligned with an optimal degree of freedom, and accordingly, the pose of the moving object can be estimated with an optimal degree of freedom.

[0050] A specific method for performing feature matching between source point cloud data (20) and target point cloud data (30), a specific method for selecting an appropriate matching algorithm based on the result of feature matching, and a specific method for performing matching using the selected matching algorithm and estimating the pose of a moving object will be described in more detail with reference to FIGS. 2 to 9, which will be described later.

[0051]

[0052] FIG. 2 illustrates a computer system performing a data processing method according to one embodiment.

[0053] The computer system (100) may be an electronic device configured to perform a data processing method for performing data alignment and pose estimation. The computer system (100) may execute an application, software, or program for performing such data processing method.

[0054] As described above, the computer system (100) may be a server or a computer system constituting a server, or may be a separate computer system for performing the data processing method described above. For example, the computer system (100) may be a computer system included in a mobile device (e.g., a robot), or a computer system included in a robot control system for controlling a mobile device (e.g., a robot).

[0055] As illustrated in FIG. 2, the computer system (100) may include, as components, a memory (110), a processor (120), a communication interface (130), and an input / output interface (140). The input / output interface (140) may communicate with an input / output device (150) within the computer system (100) or separate from the computer system (100).

[0056] The memory (110) is a computer-readable recording medium, and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Here, the ROM and the permanent mass storage device such as the disk drive may be included in the computer system (100) as a separate permanent storage device distinct from the memory (110). In addition, the memory (110) may store an operating system and at least one program code. These software components may be loaded into the memory (110) from a computer-readable recording medium separate from the memory (110). This separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. In another embodiment, the software components may be loaded into the memory (110) through a communication interface (130) rather than a computer-readable recording medium. For example, software components may be loaded into the memory (110) of the computer system (100) based on a computer program that is installed by files received over a network (160).

[0057] The processor (120) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (120) via the memory (110) or the communication interface (130). For example, the processor (120) may be configured to execute instructions received according to program code stored in a storage device such as the memory (110).

[0058] The communication interface (130) may provide a function for the computer system (100) to communicate with other devices via a network (160). For example, requests, commands, data, files, etc. generated by the processor (120) of the computer system (100) according to program codes stored in a recording device such as a memory (110) may be transmitted to other devices via the network (160) under the control of the communication interface (130). Conversely, signals, commands, data, files, etc. from other devices may be received by the computer system (100) via the communication interface (130) of the computer system (100) via the network (160). The signals, commands, data, etc. received via the communication interface (130) may be transmitted to the processor (120) or the memory (110), and the files, etc. may be stored in a storage medium (the aforementioned permanent storage device) that the computer system (100) may further include.

[0059] The communication method through the communication interface (130) is not limited, and may include not only a communication method utilizing a communication network (e.g., a mobile communication network, wired Internet, wireless Internet, broadcasting network) that the network (160) may include, but also short-range wired / wireless communication between devices. For example, the network (160) may include any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, etc. In addition, the network (160) may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree, or a hierarchical network.

[0060] The input / output interface (140) may be a means for interfacing with an input / output device (150). For example, the input device may include a device such as a microphone, a keyboard, a camera, or a mouse, and the output device may include a device such as a display or a speaker. As another example, the input / output interface (140) may be a means for interfacing with a device that integrates input and output functions, such as a touchscreen. The input / output device (150) may also be configured as a single device with the computer system (100).

[0061] Additionally, in other embodiments, the computer system (100) may include fewer or more components than those illustrated in FIG. 2. However, most conventional components need not be explicitly depicted. For example, the computer system (100) may be implemented to include at least some of the input / output devices (150) described above, or may further include other components such as a transceiver, a camera, various sensors, a database, and the like.

[0062] The processor (120) of the computer system (100) may be configured to perform steps for performing a data processing method for acquiring target point cloud data (30) (i.e., second point cloud data (30)) and source point cloud data (20) (i.e., first point cloud data (20)), matching the second point cloud data (30) to the first point cloud data (20), and estimating the pose of a moving object, as described below.

[0063] In the detailed description to be provided below, for convenience of explanation, the above steps are described as being performed by the computer system (100), and operations performed by the processor (120) or other components of the computer system (100), or operations performed by an application / program executed by the processor (120), may be described as operations performed by the computer system (100) for convenience of explanation.

[0064] Above, the technical features described above with reference to Fig. 1 can also be applied to Fig. 2, so redundant descriptions are omitted.

[0065]

[0066] FIG. 3 is a flowchart illustrating a data processing method for aligning second point cloud data with first point cloud data and estimating a pose of a moving object, according to one embodiment.

[0067] Referring to FIG. 3, steps for performing a data processing method for matching second point cloud data (30) to first point cloud data (20) and estimating the pose of a moving object by a computer system (100) are described.

[0068] In step (310), the computer system (100) can acquire first point cloud data (20) for the space in which the mobile device travels. The first point cloud data (20) may be pre-stored in the computer system (100), or may be acquired from another computer system or database with which the computer system (100) can communicate.

[0069] In step (320), the computer system (100) can obtain second point cloud data (30) generated based on sensing data about space from a moving object. The moving object may be, for example, a robot that moves through space. The moving object may be mapping equipment that generates mapping information (e.g., 3D mapping information) about the space while moving through space. Such mapping equipment may be a mapping robot. The moving object may include a sensor unit for sensing data about the space while moving through space. The sensing data may be sensor data acquired by such a sensor unit. The sensor unit may include, for example, a lidar sensor for sensing the space. Accordingly, the computer system (100) can obtain second point cloud data (30) generated based on sensing data about the space acquired by the lidar sensor of the moving object. The generation of the second point cloud data (30) may be performed in the computer system (100), or may be performed within the moving object or in a control system (robot control system) that controls the moving object.

[0070] As previously discussed, the first point cloud data (20) may be at least a portion of the source point cloud data (20) for a space that serves as a reference for matching with the second point cloud data (30). Meanwhile, the second point cloud data (30) may be at least a portion of the target point cloud data (30) for a space generated based on sensing data acquired by a sensor unit (such as a lidar sensor) of a moving object while the moving object is moving in the space.

[0071] Additionally, each of the first point cloud data (20) and the second point cloud data (30) may represent a map generated based on a point cloud for space. Accordingly, the alignment between the first point cloud data (20) and the second point cloud data (30) may also mean the alignment between the maps represented by the first point cloud data (20) and the second point cloud data (30).

[0072] In step (330), the computer system (100) can perform feature matching between the first point cloud data (20) and the second point cloud data (30).

[0073] Feature matching between the first point cloud data (20) and the second point cloud data (30) may be a matching of feature points between point cloud data sets (or maps). In other words, feature matching may be a matching of feature points of the first point cloud data (20) with corresponding feature points of the second point cloud data (30).

[0074] For this feature matching, the computer system (100) can extract a plurality of characteristic points (i.e., extract features) from each of the first point cloud data (20) and the second point cloud data (30) in step (332).

[0075] The feature points of the first point cloud data (20) may be, for example, vertices (corners) of an outline in the first point cloud data (20) (map), points on an edge, etc. Meanwhile, the feature points of the second point cloud data (30) (map) may be, for example, vertices of an outline in the second point cloud data (30), points on an edge, etc. The feature points may also be pixels (pixel coordinates) on an image (map image) composed of the first point cloud data (20) or the second point cloud data (30).

[0076] In step (334), the computer system (100) can determine matching feature pairs between the first point cloud data (20) and the second point cloud data (30) by matching feature points extracted from the first point cloud data (20) with feature points extracted from the second point cloud data (30). A 'matching feature pair' can represent a pair of matching feature points between the first point cloud data (20) and the second point cloud data (30).

[0077] Feature matching between feature points may be performed based on a descriptor associated with each feature point obtained when extracting the feature points, or any algorithm may be used for such feature matching that identifies the feature point on the second point cloud data (30) that is most similar (most corresponding) to each feature point on the other first point cloud data (20).

[0078] The computer system (100) can determine the number of matching feature pairs determined as a result of such feature matching.

[0079] In step (340), the computer system (100) may select a matching algorithm for matching the second point cloud data (30) to the first point cloud data (20) based on the number of matching feature pairs between the first point cloud data (20) and the second point cloud data (30) obtained through feature matching.

[0080] "Selecting a matching algorithm based on the number of matching feature pairs" may mean using the number of matching feature pairs itself to select a matching algorithm, or using a value based on the number of matching feature pairs to select a matching algorithm. The matching feature pairs may be feature inliers for selecting a matching algorithm, and the number of matching feature pairs may be the number of feature inliers.

[0081] Selecting a matching algorithm may involve selecting an appropriate one from among multiple matching algorithms.

[0082] Each matching algorithm may represent a matching method, matching method, or matching process performed to match point cloud data sets. The computer system (100) may use these matching algorithms to estimate a certain degree of freedom to match point cloud data sets. In other words, feature points included in each matching feature pair between the second point cloud data (30) and the first point cloud data (20) may be estimated along with their degrees of freedom. These estimated degrees of freedom may vary depending on the type of matching algorithm selected by the computer system (100) and used for matching.

[0083] For example, the computer system (100) may select a matching algorithm that estimates a relatively lower degree of freedom when the number of matching feature pairs is less than a certain level, and may select a matching algorithm that estimates a relatively higher degree of freedom when the number of matching feature pairs is greater than a certain level. Thus, the computer system (100) of the embodiment may select a suitable matching algorithm that can prevent the occurrence of extreme errors in matching even when the number of matching feature pairs between the first point cloud data (20) and the second point cloud data (30) is small.

[0084] A specific method for selecting an appropriate one among multiple matching algorithms is described in more detail with reference to Fig. 4, which will be described later.

[0085] In step (350), the computer system (100) can align the second point cloud data (30) to the first point cloud data (20) using the alignment algorithm selected by step (340).

[0086] In step (360), the computer system (100) can estimate the pose of the moving object based on the alignment of the second point cloud data (30) with the first point cloud data (20). That is, this pose is the pose of the moving object that drove to generate the second point cloud data (30), and may be the pose of the sensor unit that acquired the sensor data that serves as the basis of the second point cloud data (30) (i.e., the lidar pose).

[0087] The estimated pose of the moving object may have a degree of freedom corresponding to the degree of freedom estimated by the alignment algorithm selected in the above step (340). For example, if the selected alignment algorithm estimates 4 degrees of freedom (DoF) to align data, the estimated pose may be a 4-degree-of-freedom pose, and if the selected alignment algorithm estimates 6 degrees of freedom (6 DoF) to align data, the estimated pose may be a 6-degree-of-freedom pose.

[0088] The specific details of estimating the pose of a moving object according to the selected alignment algorithm will be described in more detail with reference to Fig. 5, which will be described later.

[0089] Above, the technical features described above with reference to FIGS. 1 and 2 can also be applied to FIG. 3, so redundant descriptions are omitted.

[0090]

[0091] FIG. 4 is a flowchart illustrating a method for selecting an appropriate matching algorithm based on the number of matching feature pairs between first point cloud data and second point cloud data, according to an example.

[0092] Referring to FIG. 4, a method for selecting an appropriate matching algorithm based on the number of matching feature pairs between the first point cloud data (20) and the second point cloud data (30) by the computer system (100) is described in more detail.

[0093] In step (410), the computer system (100) can determine whether the number of matching feature pairs obtained through feature matching between the first point cloud data (20) and the second point cloud data (30) is greater than or equal to a first number (i.e., a certain level). The first number may be, for example, 10, and may be set to various values ​​as needed.

[0094] In step (420), if the number of matching feature pairs obtained according to the feature matching is less than the first number, the computer system (100) may select a first matching algorithm that matches the second point cloud data (30) to the first point cloud data (20) with a first degree of freedom level among a plurality of matching algorithms.

[0095] Additionally, in step (430), if the number of matching feature pairs obtained through the feature matching is greater than or equal to a first number, the computer system (100) may select a second matching algorithm that matches the second point cloud data (30) to the first point cloud data (20) at a second degree of freedom level higher than the first degree of freedom level among the plurality of matching algorithms.

[0096] In this way, the computer system (100) can select a matching algorithm having different degrees of freedom to match the second point cloud data (30) to the first point cloud data (20) depending on whether the number of matching feature pairs is above a certain level.

[0097] Here, a higher degree of freedom level may mean estimating more degrees of freedom variables during matching.

[0098] For example, the first alignment algorithm may be a alignment algorithm configured to align second point cloud data (30) to first point cloud data (20) by estimating four degrees of freedom (4 DoF), which is a first degree of freedom level.

[0099] Meanwhile, the second alignment algorithm may be a alignment algorithm configured to align the second point cloud data (30) to the first point cloud data (20) by estimating the second degree of freedom level, which is 6 degrees of freedom (6 DoF).

[0100] The first matching algorithm can perform matching by estimating four degree-of-freedom variables. For example, the first matching algorithm can be a matching algorithm configured to match the second point cloud data (30) to the first point cloud data (20) by estimating a first degree-of-freedom variable representing rotation in the x-axis direction, a second degree-of-freedom variable representing rotation in the y-axis direction, a third degree-of-freedom variable representing rotation in the z-axis direction, and a fourth degree-of-freedom variable representing rotation in the yaw direction as the four degrees of freedom.

[0101] Meanwhile, the second matching algorithm can perform matching by estimating six degree-of-freedom variables. For example, the second matching algorithm may be a matching algorithm configured to match the second point cloud data (30) to the first point cloud data (20) by further estimating a fifth degree-of-freedom variable representing a rotation in the roll direction and a sixth degree-of-freedom variable representing a rotation in the pitch direction in addition to the first to fourth degree-of-freedom variables estimated by the first matching algorithm, which are the six degrees of freedom.

[0102] In this way, in the embodiment, the computer system (100) can select the first matching algorithm or the second matching algorithm as a matching algorithm having different degrees of freedom depending on whether the number of matching feature pairs is above a certain level.

[0103] The computer system (100) can perform the matching by selecting the first matching algorithm that performs the matching by estimating only 4 degrees of freedom when the number of matching feature pairs is less than a certain level, and thus, even when the number of matching feature pairs is small, a relatively accurate matching result can be obtained. When the number of matching feature pairs is small, the degree of freedom variables representing the rotation in the roll direction and the degree of freedom variables representing the rotation in the pitch direction cannot be estimated accurately and may cause extreme errors, and therefore, these degree of freedom variables may not be estimated. In this way, when the number of matching feature pairs is small, performing the matching by estimating only 4 degrees of freedom can obtain a more accurate matching result than performing the matching by estimating 6 degrees of freedom.

[0104] Meanwhile, the computer system (100) can perform the matching by selecting the second matching algorithm that estimates all six degrees of freedom and performs the matching when the number of matching feature pairs is above a certain level, and thus, when the number of matching feature pairs is large, an accurate matching result can be obtained. When the number of matching feature pairs is large, the degree of freedom variable indicating the rotation in the roll direction and the degree of freedom variable indicating the rotation in the pitch direction can also be estimated relatively accurately, and thus, these degree of freedom variables can be estimated. In this way, when the number of matching feature pairs is large, a more accurate matching result can be obtained by estimating six degrees of freedom.

[0105] Meanwhile, according to an embodiment, the computer system (100) may select a matching algorithm that matches the second point cloud data to the first point cloud data by estimating a greater number of degree-of-freedom variables among a plurality of degree-of-freedom variables as the number of matching feature pairs obtained through feature matching between the first point cloud data (20) and the second point cloud data (30) increases.

[0106] At this time, the plurality of degree of freedom variables may include a first degree of freedom variable representing rotation in the x-axis direction, a second degree of freedom variable representing rotation in the y-axis direction, a third degree of freedom variable representing rotation in the z-axis direction, a fourth degree of freedom variable representing rotation in the yaw direction, a fifth degree of freedom variable representing rotation in the roll direction, and a sixth degree of freedom variable representing rotation in the pitch direction.

[0107] For example, the computer system (100) can consider a plurality of reference numbers as well as the first number, and can select an appropriate matching algorithm from among three or more matching algorithms configured to perform matching by estimating different numbers of degree of freedom variables depending on whether the matching feature pair is greater than or equal to each reference number.

[0108] Alternatively, the computer system (100) may select a suitable matching algorithm from among matching algorithms configured to perform matching by estimating different types and / or numbers of degree-of-freedom variables depending on whether the number of matching feature pairs is greater than or equal to a predetermined reference number.

[0109] Meanwhile, FIG. 5 illustrates a method for selecting an appropriate matching algorithm and estimating the pose of a moving object based on the number of matching feature pairs between first point cloud data and second point cloud data according to an example.

[0110] As described above, the computer system (100) can select a suitable matching algorithm according to the number of matching feature pairs obtained through feature matching (step (510)). Here, if the number of matching feature pairs obtained is less than a certain level, the computer system (100) can select a first matching algorithm that performs matching by estimating 4 degrees of freedom, and at this time, the estimated degree of freedom variable may be the first to fourth degree of freedom variable described above (step (520)). Meanwhile, if the number of matching feature pairs obtained is greater than a certain level, the computer system (100) can select a second matching algorithm that performs matching by estimating 6 degrees of freedom, and at this time, the estimated degree of freedom variable may be the first to sixth degree of freedom variable described above (step (540)). When the computer system (100) performs the alignment of the second point cloud data (30) to the first point cloud data (20) by the first alignment algorithm, the computer system (100) can estimate a 4-degree-of-freedom pose as the pose of the moving object (step (530)). Meanwhile, when the computer system (100) performs the alignment of the second point cloud data (30) to the first point cloud data (20) by the second alignment algorithm, the computer system (100) can estimate a 6-degree-of-freedom pose as the pose of the moving object (step (550)).

[0111] In this way, the computer system (100) can estimate the pose of a moving object by selecting different degrees of freedom to estimate depending on the number of matching feature pairs (i.e., selecting matching algorithms with different degrees of freedom), and thus, the accuracy of pose estimation can be improved even when the number of matching feature pairs is small.

[0112] Below, specific examples of the first matching algorithm and the second matching algorithm described above are described in more detail.

[0113] The first alignment algorithm and / or the second alignment algorithm may be algorithms that align the first point cloud data (20) and the second point cloud data (30) under certain environmental conditions. These environmental conditions may be assumptions (preconditions) regarding the space represented by the first point cloud data (20) and the second point cloud data (30).

[0114] For example, the first alignment algorithm may be configured to align the second point cloud data (30) to the first point cloud data (20) under a first environmental condition that assumes that the first point cloud data (20) and the second point cloud data (30) are on the same plane.

[0115] That is, the first alignment algorithm can be configured to perform alignment under environmental conditions that assume that the two environments (the first point cloud data (20) and the second point cloud data (30)) are vertically identical. In this first alignment algorithm, it can be assumed that there is no rotation in the roll direction and no rotation in the pitch direction in the first point cloud data (20) and the second point cloud data (30) (i.e., the rotation in the roll direction and the rotation in the pitch direction are identical in the two environments).

[0116] Accordingly, when performing alignment using the first alignment algorithm, i.e., when the matching feature pairs are below a certain level, four degrees of freedom can be estimated that ignore the roll direction rotation and pitch direction rotation that cause larger errors or errors.

[0117] In the embodiment, considering that it is difficult to determine whether there is a height difference between two environments (i.e., whether there is a difference in roll and pitch rotations) when there are insufficient matching feature points (i.e., when there is little correspondence between feature points), roll and pitch rotations that generate large errors can be ignored. This prevents extreme errors from occurring during matching, and errors can be evenly distributed across multiple factors rather than being limited to a specific factor.

[0118] Additionally, at least one of the first matching algorithm and / or the second matching algorithm may be configured to match the second point cloud data (30) to the first point cloud data (20) under a second environmental condition that the space is composed of at least one rectangular parallelepiped object.

[0119] That is, the second environmental condition may represent an assumption that the floor is flat in both environments (the first point cloud data (20) and the second point cloud data (30)) and that objects (buildings, structures, facilities, etc.) within the environment are composed of rectangular parallelepiped(s). Under the second environmental condition, all columns, etc. contained within the space may also be assumed to be vertical. The second environmental condition may be applied to the first matching algorithm or may be a precondition applied to both the first and second matching algorithms.

[0120] The first matching algorithm for estimating four degrees of freedom may be, for example, the Quatro matching algorithm. The environmental conditions applied to the first algorithm may be the Atlanta world assumption. The Quatro matching algorithm may be, for example, described in Reference 1 below.

[0121] Document 1: “A Single Correspondence Is Enough: Robust Global Registration to Avoid Degeneracy in Urban Environments”, Hyungtae Lim, Suyong Yeon, Soohyun Ryu, Yonghan Lee, Youngji Kim, Jaeseong Yun, Euigon Jung, Donghwan Lee, Hyun Myung, in Proc. IEEE Int'l Conf. on Robotics and Automation (ICRA), pp. 8010-8017, Philadelphia, USA, May 2022.

[0122] A second algorithm for estimating six degrees of freedom may be, for example, the TEASER matching algorithm. The TEASER matching algorithm may be, for example, described in Reference 2 below.

[0123] Literature 2: “TEASER: Fast and Certifiable Point Cloud Registration”, Yang, H., Shi, J., & Carlone, L., IEEE Trans. Robotics, 37(2), 314-333.

[0124] Below, a method for determining a first number (or a certain level) of matching feature pairs that serve as a criterion for selecting the first matching algorithm or the second matching algorithm described above is described in more detail.

[0125] For example, the first number of matching feature pairs, which is a criterion for selecting the first matching algorithm or the second matching algorithm, may be a value determined based on at least one of a ratio of first valid feature points among first feature points extracted from the first point cloud data (20) and a ratio of second valid feature points among second feature points extracted from the second point cloud data (30), and at least one of a ratio of first matching points used to determine matching feature pairs among the first valid feature points and a ratio of second matching points used to determine matching feature pairs among the second valid feature points.

[0126] The first valid feature points may represent valid feature points (e.g., those excluding feature points corresponding to noise or specific points indicating points that do not correspond to an outline) among the first feature points, which are all feature points extracted from the first point cloud data (20). Meanwhile, the second valid feature points may represent valid feature points (e.g., those excluding feature points corresponding to noise or specific points indicating points that do not correspond to an outline) among the second feature points, which are all feature points extracted from the second point cloud data (30).

[0127] In an embodiment, the ratio of first valid feature points to these first feature points of the first point cloud data (20) and / or the ratio of second valid feature points to these second feature points of the second point cloud data (30) (i.e., the first ratio), the ratio of first matching points included in matching point pairs for the first valid feature points and / or the ratio of second matching points included in matching point pairs for the first valid feature points (i.e., the second ratio) may be used to determine the first value.

[0128] For example, in an embodiment, 2000 feature points can be extracted from each of source point cloud data (20) and target point cloud data (30), and a matching algorithm can be selected so that if there are 200 or more matching point pairs as a result of feature matching, a 6-degree-of-freedom pose can be estimated, and if there are less than 200, a 4-degree-of-freedom pose can be estimated.

[0129] At this time, the first number (certain level), which is the number of matching point pairs that serve as a reference, may be determined as a value based on 'a predetermined minimum ratio of extracted feature points X matching ratio of feature points' by feature extraction. For example, the first number may be determined based on "a predetermined minimum value of the aforementioned first ratio X a predetermined minimum value of the aforementioned second ratio." 'A predetermined minimum ratio of extracted feature points X matching ratio of feature points' may be a formula for determining whether there are sufficient valid feature points for the two point cloud data (20, 30).

[0130] For example, if there are matching feature pairs greater than or equal to a value based on 'a predetermined minimum ratio of extracted feature points X matching ratio of feature points', a matching algorithm for estimating a 6-degree-of-freedom pose may be selected, and otherwise, a matching algorithm for estimating a 4-degree-of-freedom pose may be selected.

[0131] Alternatively, the first number, which is the number of matching point pairs that serve as a reference, may be a value determined by experiment.

[0132] Alternatively, the number of matching point pairs that serve as a reference may correspond to a certain ratio. For example, if point cloud data has 15,000 to 20,000 feature points, the size of data corresponding to approximately 10% may correspond to 2,000 point pairs, and if the number of matched points exceeds 1% (i.e., 200), it may be determined that there is a sufficient ratio of matching points to estimate 6 degrees of freedom, and thus, a matching algorithm for estimating a 6-degree-of-freedom pose may be selected. On the other hand, if the number of matched points is 1% or less, it may be determined that there is an insufficient ratio of matching points to estimate 6 degrees of freedom, and thus, a matching algorithm for estimating a 4-degree-of-freedom pose may be selected. Since the technical features described above with reference to FIGS. 1 to 3 can also be applied to FIGS. 4 and 5, a redundant description will be omitted.

[0133]

[0134] Figure 6 illustrates source point cloud data and target point cloud data according to an example.

[0135] In Fig. 6, specific examples of source point cloud data (610) and target point cloud data (620), which are the first point cloud data (20) and second point cloud data (30) described above, are illustrated. These point cloud data (610, 620) may be referred to as raw sensor data.

[0136] Meanwhile, FIG. 7 illustrates a method for extracting feature points (i.e., performing feature extraction) from each of source point cloud data and target point cloud data according to an example.

[0137] In the data sets (700) of FIG. 7, the extraction results (710) of feature points for source point cloud data (610) and the extraction results (720) of feature points for target point cloud data (620) are illustrated. As illustrated, each of the feature points may be a vertex (corner point) of an outline, a point on an edge, etc.

[0138] For example, deep learning-based feature extraction methods can be used to extract these feature points.

[0139] Meanwhile, FIGS. 8 and 9 illustrate a method for selecting an appropriate matching algorithm based on the result of feature matching, according to an example.

[0140] In Fig. 8, a case is exemplified where the first matching algorithm that estimates 4 degrees of freedom (x, y, z, yaw) is selected when the number of matching feature point pairs (i.e., inlier feature correspondence) as a result of feature matching is below a certain level (800).

[0141] In contrast, in FIG. 9, a case is exemplified where the second matching algorithm that estimates 6 degrees of freedom (x, y, z, yaw, pitch, roll) is selected when the number of matching feature point pairs (i.e., inlier feature correspondence) as a result of feature matching is above a certain level (900).

[0142] In this way, in the embodiment, a matching algorithm that considers the degree of freedom that can increase the accuracy of data matching and pose estimation can be selected by considering the number of matching feature point pairs according to the result of feature matching, and thus, the accuracy of data matching and pose estimation can be increased as a result.

[0143] Above, the technical features described above with reference to FIGS. 1 to 5 can also be applied to FIGS. 6 to 9, so redundant descriptions are omitted.

[0144] The devices described above may be implemented as hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0145] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0146] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program commands, including ROM, RAM, and flash memory. In addition, examples of other media may include recording media or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0147] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0148] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. In a data processing method performed by a computer system, A step of acquiring first point cloud data for a space in which a mobile body moves; A step of acquiring second point cloud data generated based on sensing data for the space from the moving object; A step of performing feature matching between the first point cloud data and the second point cloud data; A step of selecting a matching algorithm for matching the second point cloud data to the first point cloud data based on the number of matching feature pairs between the first point cloud data and the second point cloud data obtained according to the feature matching; and A step of aligning the second point cloud data to the first point cloud data using the selected alignment algorithm. A data processing method comprising:

2. In paragraph 1, A step of estimating the pose of the moving object based on the alignment of the second point cloud data with the first point cloud data. A method of processing data, further comprising 3. In paragraph 1, The above first point cloud data is at least a portion of the source point cloud data for the space that serves as a basis for alignment with the above second point cloud data, A data processing method, wherein the second point cloud data is at least a portion of target point cloud data for the space generated based on the sensing data acquired by the sensor unit of the mobile body while the mobile body moves through the space.

4. In paragraph 1, The above selection steps are: If the number of the matching feature pairs obtained through the feature matching is less than the first number, a first matching algorithm that matches the second point cloud data to the first point cloud data at the first degree of freedom level is selected from among a plurality of matching algorithms, A data processing method, wherein, if the number of the matching feature pairs obtained through the feature matching is greater than or equal to a first number, a second matching algorithm that matches the second point cloud data to the first point cloud data at a second degree of freedom level higher than the first degree of freedom level is selected from among the plurality of matching algorithms.

5. In paragraph 4, The above first matching algorithm is configured to match the second point cloud data to the first point cloud data by estimating the first degree of freedom level, which is 4 degrees of freedom (DoF). A data processing method, wherein the second alignment algorithm is configured to align the second point cloud data to the first point cloud data by estimating the second degree of freedom level, which is 6 degrees of freedom (6 DoF).

6. In paragraph 5, When the second point cloud data is aligned with the first point cloud data by the first alignment algorithm, a four-degree-of-freedom pose is estimated as the pose of the moving object. A data processing method in which a 6-degree-of-freedom pose is estimated as the pose of the moving body when the second point cloud data is aligned with the first point cloud data by the second alignment algorithm.

7. In paragraph 5, The first matching algorithm is configured to match the second point cloud data to the first point cloud data by estimating a first degree of freedom variable representing rotation in the x-axis direction, a second degree of freedom variable representing rotation in the y-axis direction, a third degree of freedom variable representing rotation in the z-axis direction, and a fourth degree of freedom variable representing rotation in the yaw direction as the four degrees of freedom. A data processing method, wherein the second matching algorithm is configured to match the second point cloud data to the first point cloud data by further estimating a fifth degree of freedom variable representing a rotation in the roll direction and a sixth degree of freedom variable representing a rotation in the pitch direction in addition to the first to fourth degree of freedom variables as the six degrees of freedom.

8. In paragraph 5, A data processing method, wherein the first matching algorithm is configured to match the second point cloud data to the first point cloud data under a first environmental condition that assumes that the first point cloud data and the second point cloud data are on the same plane.

9. In paragraph 5, A data processing method, wherein at least one of the first matching algorithm and the second matching algorithm is configured to match the second point cloud data to the first point cloud data under a second environmental condition that the space is composed of at least one rectangular parallelepiped object.

10. In paragraph 1, The above selection steps are: The more the number of matching feature pairs obtained through the feature matching, the more degree of freedom variables are estimated among the plurality of degree of freedom variables, and a matching algorithm is selected to match the second point cloud data to the first point cloud data. A data processing method, wherein the plurality of degree-of-freedom variables include a first degree-of-freedom variable representing rotation in the x-axis direction, a second degree-of-freedom variable representing rotation in the y-axis direction, a third degree-of-freedom variable representing rotation in the z-axis direction, a fourth degree-of-freedom variable representing rotation in the yaw direction, a fifth degree-of-freedom variable representing rotation in the roll direction, and a sixth degree-of-freedom variable representing rotation in the pitch direction.

11. In paragraph 1, The step of performing the above feature matching is: A step of extracting a plurality of feature points from each of the first point cloud data and the second point cloud data; and A step of determining matching feature pairs between the first point cloud data and the second point cloud data by matching feature points extracted from the first point cloud data and feature points extracted from the second point cloud data. A data processing method comprising:

12. In paragraph 1, The above selection steps are: If the number of the above matching feature pairs is less than the first number, a first matching algorithm that matches the second point cloud data to the first point cloud data at the first degree of freedom level is selected from among a plurality of matching algorithms, If the number of the matching feature pairs obtained through the feature matching is greater than or equal to the first number, a second matching algorithm is selected from among the plurality of matching algorithms for matching the second point cloud data to the first point cloud data at the second degree of freedom level higher than the first degree of freedom level. The above first number is, At least one of a ratio of first valid feature points among first feature points extracted from the first point cloud data and a ratio of second valid feature points among second feature points extracted from the second point cloud data, A data processing method, wherein the value is determined based on at least one of a ratio of first matching points used to determine the matching feature pairs among the first valid feature points and a ratio of second matching points used to determine the matching feature pairs among the second valid feature points.

13. A non-transitory computer-readable recording medium storing a computer program for executing the method of paragraph 1 on the computer system.

14. In a computer system that performs a data processing method, At least one processor configured to execute computer-readable instructions contained in memory Including, At least one processor, Acquire the first point cloud data for the space in which the mobile object is moving, Obtain second point cloud data generated based on sensing data for the space from the above moving object, Perform feature matching between the first point cloud data and the second point cloud data, Based on the number of matching feature pairs between the first point cloud data and the second point cloud data obtained through the feature matching, a matching algorithm for matching the second point cloud data to the first point cloud data is selected, A computer system that aligns the second point cloud data to the first point cloud data using the selected alignment algorithm.

Images