Computing device for recognizing relative pose through point cloud comparison, and operating method thereof

Abstract

The present disclosure relates to a computing device using a sensor module so as to acquire a plurality of point clouds collected at different points, and an operating method thereof, which are capable of: matching a second point in a second point cloud to a first point while iteratively changing a reference distance for each first point in a first point cloud; and determining, on the basis of the first point and the second point, a relative pose between points at which each of the first point cloud and the second point cloud is collected. In the disclosure, the reference distance can be changed at each iteration according to a function of the sensing distance from the sensor module to the first point and the pose change amount estimated in the previous iteration.

Description

Computing device for relative pose recognition through point cloud comparison and its operation method

[0001] The present disclosure relates to a computing device for map production and an operating method thereof, and more particularly, to a computing device for relative pose recognition through point cloud comparison and an operating method thereof.

[0002] There is a technology that allows a device to wander through an unknown environment, collect multiple point clouds of that environment, and then create a map by matching these point clouds. Typically, the device detects points within a certain distance of each point cloud and matches these points to estimate the relative pose between the points where the point clouds were collected. This allows the device to align the point clouds based on the relative pose.

[0003] However, such a device relies on preset parameter values ​​for matching points within point clouds. For example, the distance as a parameter value for matching points must be preset according to the scale of the environment. For another example, the device performs multiple steps by estimating the distance between relative positions based on a pair of point clouds at each step. In this case, the distance as a parameter value for matching points must be preset, including the distance between relative positions estimated in the previous step. As such, the device does not adaptively operate according to changes in the environment or the progress of multiple steps for matching points within point clouds. This leads to a decrease in the accuracy of estimating the relative pose between points where point clouds are collected by the device. Furthermore, such a device requires a relatively long computation time to estimate the relative pose between points where point clouds are collected.

[0004] The present disclosure provides a computing device and its operating method for relative pose recognition through point cloud comparison.

[0005] The present disclosure provides a computing device and an operating method thereof for estimating a relative pose between points collected from point clouds with high accuracy and a relatively short computation time.

[0006] The present disclosure provides a method of operating a computing device that acquires a plurality of point clouds collected at different points using a sensor module, the method comprising the steps of: matching a second point in a second point cloud to each first point in a first point cloud while repeatedly changing a reference distance for each first point in the first point cloud; and determining a relative pose between points where the first point cloud and the second point cloud were collected, respectively, wherein the reference distance may be changed at each repetition according to a function of a sensing distance from the sensor module to the first point and a pose change amount estimated at a previous repetition.

[0007] The present disclosure provides a computing device that acquires a plurality of point clouds collected at different points using a sensor module, the computing device including a memory, and a processor connected to the memory and configured to execute at least one command stored in the memory, wherein the processor is configured to repeatedly change a reference distance for each first point in a first point cloud, match a second point in a second point cloud to the first point, and determine a relative pose between points where the first point cloud and the second point cloud are collected, respectively, based on the first point and the second point, wherein the reference distance may be changed at each repetition according to a function of a sensing distance from the sensor module to the first point and a pose change amount estimated at a previous repetition.

[0008] According to the present disclosure, it is possible to estimate relative poses between points from which point clouds are collected with high accuracy and relatively short computation times. Specifically, the computing device can estimate relative positions from which point clouds are collected by repeatedly changing the reference distance for each point in the point clouds and matching points within the reference distance. That is, the reference distance varies for each point and may also vary for each point depending on the repetition. In this way, the computing device can adaptively operate for each point without requiring prior information about the scale of the environment or previous steps. At this time, the computing device can achieve accurate matching of points by considering the normal vector for each point. Here, the computing device can obtain more accurate matching results for point clouds by detecting the normal vector based on the position of a sensor module or a sensor module within the device, rather than the floor or the center of the computing device or device. Therefore, the computing device can estimate the relative positions from which point clouds are collected with high accuracy.

[0009] FIG. 1 is a block diagram illustrating a computing device for relative pose recognition through point cloud comparison according to various embodiments.

[0010] Figure 2 is an example diagram for explaining the reference distance that changes according to various embodiments.

[0011] Figure 3 is an example diagram for explaining normal vectors considered according to various embodiments.

[0012] Figures 4, 5, and 6 are exemplary diagrams illustrating effects achieved by considering normal vectors according to various embodiments.

[0013] FIG. 7 is a flowchart illustrating an operation method of a computing device for relative pose recognition through point cloud comparison according to various embodiments.

[0014] Figure 8 is a flowchart detailing the steps of matching the second point to the first point of Figure 7.

[0015] Figure 9 is a flowchart illustrating in detail the steps for determining the validity of the matching of each first point and second point of Figure 8.

[0016] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions of widely known functions or configurations will be omitted if they may unnecessarily obscure the gist of the present disclosure.

[0017] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Furthermore, in the description of the embodiments below, duplicate descriptions of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0018] The advantages and features of the disclosed embodiments, and methods for achieving them, will become clearer with reference to the embodiments described below, along with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure the completeness of the disclosure and to fully inform those skilled in the art of the scope of the invention.

[0019] The terms used in this disclosure will be briefly described, and the disclosed embodiments will be described in detail. The terms used in this disclosure have been selected from widely used and common terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of engineers working in the relevant field, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant invention. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0020] In this disclosure, singular expressions include plural expressions unless the context clearly dictates otherwise. Furthermore, plural expressions include singular expressions unless the context clearly dictates otherwise. Throughout the specification, when a part is said to include a certain component, this does not exclude other components, but rather implies that other components may be included, unless otherwise specifically stated.

[0021] In addition, the term 'module' or 'part' used in the present disclosure means a software or hardware component, and the 'module' or 'part' performs certain roles. However, the 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to be on an addressable storage medium and may be configured to play one or more processors. Thus, as an example, the 'module' or 'part' may include at least one of components such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and 'modules' or 'parts' may be combined into a smaller number of components and 'modules' or 'parts', or may be further separated into additional components and 'modules' or 'parts'.

[0022] According to the present disclosure, a 'module' or 'unit' may be implemented as a processor and a memory. 'Processor' should be broadly construed to include a general purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, and the like. In some circumstances, a 'processor' may also refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), and the like. A 'processor' may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other such combination of configurations. In addition, 'memory' should be broadly construed to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc. Memory is said to be in electronic communication with the processor if the processor can read information from, and / or write information to, the memory. Memory integrated in a processor is in electronic communication with the processor.

[0023]

[0024] Hereinafter, the present disclosure provides a computing device (100) for relative position recognition through point cloud comparison and an operating method thereof. Specifically, the computing device (100) can acquire a plurality of point clouds for an unknown environment and create a map by matching such point clouds. In some embodiments, the computing device (100) may be implemented in a device, such as a device that is portable or wearable by a user and movable by the user, or a robot capable of autonomous navigation. In such a case, the computing device (100) can directly collect point clouds while moving within a wide area. In other embodiments, the computing device (100) may be implemented in a server that communicates with the device. In such a case, the device can collect point clouds while moving within a wide area, and the computing device (100) can receive the point clouds from the device. In this case, the point clouds can be collected at different points within the wide area. Accordingly, the computing device (100) can compare point clouds and estimate the relative pose between the points where each point cloud was collected. Here, the pose may include position and orientation. Accordingly, the computing device can align the point clouds based on the relative pose.

[0025] In general, there are techniques for estimating the relative positions of collected point clouds by detecting points within a certain distance within the point cloud and matching those points. For example, according to GICP (Segal, Aleksandr, Dirk Haehnel, and Sebastian Thrun. "Generalized-icp." Robotics: science and systems. Vol. 2, No. 4, 2009), the distance as a parameter value for matching points is preset according to the scale of the environment. As another example, according to KISS-ICP (Vizzo, Ignacio, et al. "Kiss-icp: In defense of point-to-point icp - simple, accurate, and robust registration if done the right way." IEEE Robotics and Automation Letters 8.2 (2023): 1029-1036.), multiple steps are performed in such a way that the distance between relative positions is estimated based on a pair of point clouds at each step, and in this case, the distance as a parameter value for matching points is preset including the distance between relative positions estimated in the previous step. As such, general techniques depend on preset parameter values, which leads to a decrease in accuracy.

[0026] In contrast, in the present disclosure, the computing device (100) can estimate the relative pose between points from which point clouds are collected by repeatedly changing the reference distance for each point in the point clouds and matching points within the reference distance. According to the present disclosure, the computing device (100) can adaptively operate for each point without requiring prior information about the scale of the environment or information from previous steps. At this time, the computing device (100) can achieve accurate matching of points by considering the normal vector for each point. Various embodiments of the present disclosure will be described in more detail below.

[0027] FIG. 1 is a block diagram illustrating a computing device (100) for relative pose recognition through point cloud comparison according to various embodiments. FIG. 2 is an exemplary diagram explaining a reference distance that changes according to various embodiments. FIG. 3 is an exemplary diagram explaining normal vectors considered according to various embodiments. FIG. 4, FIG. 5, and FIG. 6 are exemplary diagrams explaining effects achieved by considering normal vectors according to various embodiments.

[0028] Referring to FIG. 1, the computing device (100) may include at least one of a camera module (110), a sensor module (120), a communication module (130), an input module (140), an output module (150), a memory (160), or a processor (170). In some embodiments, at least one of the components of the computing device (100) may be omitted. In some embodiments, at least one other component may be added to the computing device (100). In some embodiments, at least two of the components of the computing device (100) may be implemented as a single integrated circuit.

[0029] The camera module (110) can capture images of the surrounding environment of the computing device (100). For example, the images can include moving images and still images. According to one embodiment, the camera module (110) can include at least one lens, at least one image sensor, an image signal processor, or a flash.

[0030] The sensor module (120) can detect the state of the surrounding environment of the computing device (100) and generate an electrical signal or data value corresponding thereto. The sensor module (120) may include, for example, at least one of a LiDAR sensor, a radar sensor, an infrared (IR) sensor, or a distance sensor. Additionally, the sensor module (120) may detect the operating state of the computing device (100) and generate an electrical signal or data value corresponding thereto. In this case, the sensor module (120) may further include at least one of a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0031] The communication module (130) can communicate with an external device in the computing device (100). For example, the external device may include at least one of a device, a base station, a server, or a satellite. The communication module (130) may include at least one of a wired communication module and a wireless communication module. The wired communication module may be wired and connected to the external device via a connection terminal (not shown) to communicate with the external device via a wire. The wireless communication module may include at least one of a short-range communication module and a long-range communication module. The short-range communication module may communicate with the external device via a short-range communication method. For example, the short-range communication method may include at least one of Bluetooth, Wi-Fi Direct, or infrared communication. The long-range communication module may communicate with the external device via a long-range communication method. Here, the long-range communication module may communicate with the external device via a network. For example, the network may include at least one of a cellular network, the Internet, or a computer network such as a LAN or WAN. In some embodiments, the communication module (130) may receive global navigation satellite system (GNSS) information. For example, the GNSS information may include global positioning system (GPS) information.

[0032] The input module (140) can input a signal to be used in at least one component of the computing device (100). In some embodiments, the input module (140) can include at least one of a microphone, a mouse, or a keyboard. In other embodiments, the input module (140) can include at least one of touch circuitry configured to detect a touch, or a sensor circuitry configured to measure the intensity of a force generated by a touch.

[0033] The output module (150) can output information from the computing device (100). At this time, the output module (150) can include at least one of a display module for visually displaying information or an audio module for audibly reproducing information. For example, the display module can include at least one of a display, a holographic device, or a projector. As an example, the display module can be implemented as a touch screen by being assembled with at least one of a touch circuit or a sensor circuit of the input module (140). For example, the audio module can include at least one of a speaker, a receiver, an earphone, or a headphone.

[0034] The memory (160) can store various data used by at least one component of the computing device (100). For example, the memory (160) can include at least one of volatile memory and non-volatile memory. The data can include at least one program and input data or output data related thereto. The program can be stored in the memory (160) as software including at least one command.

[0035] The processor (170) can execute a program in the memory (160) to control at least one component of the computing device (100). Through this, the processor (170) can perform data processing or calculations. Here, the processor (170) can execute commands stored in the memory (160). In various embodiments, the processor (170) can estimate a relative pose between points where point clouds are collected.

[0036] In various embodiments, the processor (170) may acquire multiple point clouds collected at different points. In some embodiments, the processor (170) may directly collect point clouds while the computing device (100) moves within a wide area. In such an embodiment, the processor (170) may collect point clouds via the sensor module (120). For example, the processor (170) may collect point clouds via a lidar sensor. In other embodiments, a separate device may move within a wide area and collect point clouds via a sensor module within the device. In such an embodiment, the processor (170) may receive point clouds from the separate device via the communication module (130). In some embodiments, the multiple point clouds may be collected in a single scan of the wide area. In other embodiments, multiple point clouds may be collected in a single scan, and then multiple point clouds may be generated over multiple scans, such that the point clouds are aligned into a single point cloud.

[0037] In various embodiments, the processor (170) may compare a pair of point clouds to estimate a relative pose between points where the point clouds were respectively collected. For example, the processor (170) may compare a first point cloud (PC1) and a second point cloud (PC2) to estimate a relative pose between points where the point clouds (PC1, PC2) were respectively collected, and may compare a second point cloud (PC2) and a third point cloud (PC3) to estimate a relative pose between points where the point clouds (PC2, PC3) were respectively collected. Based on this, the processor (170) may estimate a relative pose between points where the three point clouds (PC1, PC2, PC3) were respectively collected. At this time, the pair of point clouds may include a first point cloud and a second point cloud. Here, the first point cloud may be formed by a combination of a plurality of first points, and the second point cloud may be formed by a combination of a plurality of second points.

[0038] The processor (170) repeatedly calculates a threshold distance for each first point (i) within the first point cloud. k , where k represents the number of repetitions), the second point (j) in the second point cloud can be matched to the first point (i), i.e., the threshold distance k ) can be changed at each iteration, i.e., according to the iteration round. The threshold distance k ) is the sensing distance (r) for the first point (i). i , where i represents the index of the first point within the first point cloud) and the estimated pose change (ΔT) at the previous iteration, i.e., at the previous iteration (k-1). k -1) can be changed according to the function. At this time, in various embodiments, in the next iteration (k), the pose change amount (ΔT) estimated in the previous iteration (k-1) is at most k - 1) It is assumed that a change proportional to the threshold distance can occur. k ) is different for each point, and for each point, it may also be different depending on the number of repetitions (k).

[0039] Referring to Figure 2, the threshold distance k ) can be defined as in [Mathematical Formula 1] below. The sensing distance (r) for the first point (i) i ) may represent the distance from the sensor module (110) or the sensor module of the device to the first point (i). Similarly, the sensing distance for the second point (j) may represent the distance from the sensor module (110) or the sensor module of the device to the second point (j). The pose change amount (ΔT k ), a source position (SP) and a target position (TP) can be defined for the first point (i). The source position corresponds to the first point cloud and can represent any point from which the first point (i) was collected. The target position corresponds to the second point cloud and can represent any point from which the second point (j) was collected. The pose variation (ΔT) k ) represents the change in the relative pose of the source point to the target point, or in other words, the previous source point (SP k - 1) Relative pose (T) k - 1) from the next source point (SP) k ) in the relative pose (T k ) can be expressed as the amount of change (ΔT k =func(T k-1 , T k )). For example, the pose change (ΔT k ) is the rotational change of the source point (δR k ) and displacement change (δt k ) may be included. In this case, the threshold distance k ) is the position (p) of the first point (i) i ) and the estimated rotation change (δR) in the previous iteration (k-1). k - 1) The product of the displacement change (δt) estimated in the previous iteration (k-1) k-1 ) can be determined as the added value.

[0040]

[0041] Specifically, the processor (170) can match the second point within an initial reference distance (threshold0) to the first point (i). The initial reference distance (threshold0) is the sensing distance (r) to the first point (i). i ) can be determined as a function of the relative pose (T0) of the source point to the target point. To determine the initial reference distance (threshold0), the relative pose (T0) can be determined according to the uncertainty of the predetermined pose, or can be preset to an arbitrarily large value. Then, the processor (170) determines the reference distance (threshold) from the initial reference distance (threshold0) at the first iteration (k=1). k ) is calculated, and the previous reference distance (threshold) is calculated in the next iteration (k>2). k - 1) from the threshold distance k ), while calculating the threshold distance k ) can match the second point (j). The threshold distance k ) is repeatedly changed, the pose change amount (ΔT k) the relative pose of the source point to the target point becomes closer to the actual pose, and the pose change amount (ΔT k - 1) As the threshold distance decreases, k ) can also be reduced.

[0042] The processor (170) can estimate a relative pose between points where the first point cloud and the second point cloud are collected, respectively. To this end, the processor (170) can verify the validity of the matching between the first point (i) and the second point (j). As a result, if the matching is valid, the processor (170) can estimate a relative pose between the target point and the final source point. In this case, if there are multiple valid matchings, the processor (170) can estimate a relative pose based on each matching, estimate relative poses corresponding to all matchings, and estimate a final relative pose by combining such relative poses.

[0043] In some embodiments, the processor (170) verifies the validity of the matching of the first point (i) and the second point (j) by calculating the position (p) of the first point (i) relative to the target point. i ') and the position of the second point (j) based on the target point (p j ) can be compared. Specifically, the processor (170) determines the position (p) of the first point (i) based on the target point, as in [Mathematical Formula 2] below. i ') and the position of the second point (j) based on the target point (p j ) is the threshold distance for the first point k ) or less, it can be determined to be valid. For example, the position (p) of the first point (i) based on the target point i ') is the relative pose of the source point to the target point and the position (p) of the first point (i) relative to the source point. i ) can be calculated from.

[0044]

[0045] In other embodiments, the processor (170) may compare the normal vector of the first point (i) to the target point and the normal vector of the second point (j) to the target point to verify the validity of the matching of the first point (i) and the second point (j). Here, the normal vector of any point may represent a vector perpendicular to the plane to which the point belongs. In addition, the plane to which the point belongs may be calculated from a plurality of points, for example, 20 points, which are composed of the point and adjacent points to the point. For example, the plane to which the point belongs may be calculated using the principal component analysis (PCA) technique, and since the point cloud is three-dimensional, the third vector with the smallest singular value of the analysis result may become the normal vector. The processor (170) may detect normal vectors based on the position of the sensor module (110) or the sensor module within the device, rather than the center of the computing device (100) or the device, as illustrated in FIG. 3. Specifically, the processor (170) may determine that the normal vector is valid if the angle between the normal vector of the first point (i) with respect to the target point and the normal vector of the second point (j) with respect to the target point is less than or equal to a threshold angle. At this time, the processor (170) may detect the normal vector of the first point (i) and the normal vector of the second point (j) with respect to the target point.

[0046] In this way, based on the validity of the matching of the first point (i) and the second point (j), more accurate matching of the point clouds can be achieved by estimating the relative positions where the first point cloud and the second point cloud were collected, respectively. For example, when normal vectors are not considered, as a result of matching the point clouds, as shown in Fig. 4(a), points corresponding to different walls (distinguished by red and blue in the green circle) may be considered valid matches and may be combined into a single wall. In contrast, when normal vectors are considered, as a result of matching the point clouds, as shown in Fig. 4(b), points corresponding to different walls (distinguished by red and blue in the green circle) may be considered invalid matches and may be separated into a single, different wall.

[0047] In addition, normal vectors may be detected based on the position of the sensor module (110) or the sensor module within the device, rather than the floor or the center of the computing device (100) or the device. For example, when normal vectors are detected based on the center of the computing device (100) or the device, as a result of matching point clouds, normal vectors of points corresponding to one floor may exhibit different directions (distinguished in pink and red within a yellow circle), as shown in (a) of FIG. 5. In contrast, when normal vectors are detected based on the position of the sensor module (110) or the sensor module within the device, as a result of matching point clouds, normal vectors of points corresponding to one floor may exhibit generally the same direction (distinguished in red within a yellow circle), as shown in (b) of FIG.

[0048] Meanwhile, multiple point clouds can be collected in a single scan, and these point clouds can be aligned to form a single point cloud, thereby generating multiple point clouds according to multiple scans. In this case, if normal vectors are not detected based on the position of the sensor module (110) or the sensor module within the device, as a result of the alignment of the point clouds, points corresponding to different walls may be considered valid matches and combined into a single wall (expressed in green within a red circle), as shown in (a) of FIG. 6 . In contrast, if normal vectors are detected based on the position of the sensor module (110) or the sensor module within the device, as a result of the alignment of the point clouds, points corresponding to different walls may be considered invalid matches and distinguished as single different walls (expressed in green and blue within a red circle), as shown in (b) of FIG.

[0049] FIG. 7 is a flowchart illustrating an operation method of a computing device (100) for relative pose recognition through point cloud comparison according to various embodiments.

[0050] Referring to FIG. 7, the computing device (100) may acquire a plurality of point clouds collected at different locations in step 710. In some embodiments, the processor (170) may directly collect the point clouds while the computing device (100) moves within a wide area. In this case, the processor (170) may collect the point clouds through the sensor module (120). For example, the processor (170) may collect the point clouds through a lidar sensor. In other embodiments, a separate device may move within a wide area and collect the point clouds through a sensor module within the device. In this case, the processor (170) may receive the point clouds from the separate device through the communication module (130). In some embodiments, the multiple point clouds may be collected in a single scan of the wide area. In other embodiments, multiple point clouds may be collected in a single scan, and then multiple point clouds may be generated over multiple scans, such that the point clouds are aligned into a single point cloud.

[0051] At this time, a pair of point clouds may be defined. At this time, the pair of point clouds may include a first point cloud and a second point cloud. Here, the first point cloud may be formed by a combination of a plurality of first points, and the second point cloud may be formed by a combination of a plurality of second points. For example, in the case of a first pair of point clouds (PC1, PC2) including a first point cloud (PC1) and a second point cloud (PC2), the first point cloud may be the first point cloud (PC1), and the second point cloud may be the second point cloud (PC2). Meanwhile, in the case of a second pair of point clouds (PC2, PC3) including a second point cloud (PC2) and a third point cloud (PC3), the first point cloud may be the second point cloud (PC2), and the second point cloud may be the third point cloud (P3).

[0052] Next, the computing device (100) repeatedly calculates a threshold distance for each first point (i) within the first point cloud in step 720. k ) can be matched to the second point (j) in the second point cloud for the first point (i). That is, the processor (170) can change the reference distance for each repetition, i.e., according to the repetition number (k). The reference distance is the sensing distance (r) for the first point (i). i ) and the estimated pose change (ΔT) at the previous iteration, i.e., at the previous iteration (k-1). k - 1) can be changed according to the function. At this time, in various embodiments, in the next iteration, the maximum pose change amount (ΔT) estimated in the previous iteration k -1) It is assumed that a change proportional to the amount can occur. This will be described in more detail later with reference to Fig. 8.

[0053] Figure 8 is a flowchart illustrating in detail the step (step 720) of matching the second point to the first point of Figure 7.

[0054] Referring to FIG. 8, the computing device (100) determines a relative pose (T) between a source point for the first point cloud and a target point for the second point cloud at step 810. k -1 ) and pose change (ΔT k - 1) can be initially set (k=1). The source point corresponds to the first point cloud and represents an arbitrary point from which the first point cloud was collected. The target point corresponds to the second point cloud and represents an arbitrary point from which the second point cloud was collected. Specifically, the processor (170) can set an initial relative pose (T0) and an initial pose variation (ΔT0). At this time, the initial relative pose (T0) may be set according to the uncertainty of a predetermined pose or may be set to an arbitrary large value. Meanwhile, the initial pose variation (ΔT0) may be 0.

[0055] Next, the computing device (100) generates a relative pose (T) at step 820. k - 1) The first point cloud can be transformed from the source point reference to the target point reference. Specifically, the processor (170) converts the relative pose (T k - 1) Based on the source point, the positions of the first points (i) (p) i ) the positions of the first points (i) based on the target point (p) i ') can be converted into relative poses (T). The processor (170) can be converted into relative poses (T k -1) and the positions of the first points (i) based on the source point (p i ) from the position (p) of the first point (i) based on the target point i ') can be calculated. For example, the processor (170) can calculate the relative pose (T k - 1) and the positions of the first points (i) based on the source point (p i ) are the product of each of the positions (p) of the first points (i) based on the target point. i ') can be calculated for each of (p i '=T k-1 *p i ).

[0056] Next, the computing device (100) can match the second point (j) closest to each first point (i) among the second point clouds in step 830. Specifically, the processor (170) can match the positions (p) of the first points (i) with respect to the target point. i '), the closest second point (j) among the second point clouds can be matched to each first point (i).

[0057] Next, the computing device (100) can determine the validity of the matching of each first point (i) and the second point (j) in step 840. At this time, the processor (170) can detect matching information for each matching, wherein the matching information is the position (p) of the first point (i) based on the source point. i ), the position (p) of the first point (i) relative to the target point i '), and the position of the second point (j) (p j ) may be included. The processor (170) may determine validity based on the matching information. As a result, the computing device (100) may obtain at least one valid match. This will be described in more detail with reference to FIG. 9.

[0058] FIG. 9 is a flowchart illustrating in detail the step (step 840) of determining the validity of the matching of each first point (i) and second point (j) of FIG. 8.

[0059] Referring to FIG. 9, the computing device (100) determines the pose change amount (ΔT) at step 910. k - 1) Using the threshold distance for the first point (i) k ) can be calculated. Specifically, the processor (170) calculates the position (p) of the first point (i) based on the source point. i ) and pose change (ΔT k-1 ) as a function of the threshold distance k ) can be calculated (threshold k =func(p i , ΔT k-1 )). For example, the pose change (ΔT k - 1) is the rotational change of the source point (δR k - 1) and displacement change (δt k-1 ) may be included. In this case, the threshold distance k ) is the position (p) of the first point (i) i ) and rotation change (δR k - 1) The product of the displacement change (δt) k - 1) can be determined by adding the value.

[0060] Next, the computing device (100) can calculate the distance between the first point (i) and the second point (j) based on the target point in step 920. ) Specifically, the processor (170) determines the position (p) of the first point (i) relative to the target point. i ') and the position of the second point (j) based on the target point (p j) can be calculated. Then, the computing device (100) calculates the distance calculated in step 930 as a threshold distance k ) can be compared. Specifically, the processor (170) calculates the calculated distance as a threshold distance k ) can be used to determine whether or not it is below.

[0061] The distance calculated at step 930 is the threshold distance k ) if it is judged to be below ( ≤ threshold k ), the computing device (100) can detect the angle between the normal vectors of the first point (i) and the second point (j) with respect to the target point in step 940. Specifically, the processor (170) can compare the normal vector of the first point (i) with respect to the target point and the normal vector of the second point (j) with respect to the target point. Here, the normal vector of any point can represent a vector perpendicular to the plane to which the point belongs. In addition, the plane to which the point belongs can be calculated from a plurality of points, for example, 20 points, which are composed of the point and adjacent points to the point. For example, the plane to which the point belongs can be calculated using the principal component analysis (PCA) technique, and since the point cloud is three-dimensional, the third vector with the smallest singular value of the analysis result can be the normal vector. The processor (170) may detect normal vectors, as illustrated in FIG. 3, based on the position of the sensor module (110) or a sensor module within the device, rather than the center of the computing device (100) or the device. Then, the computing device (100) may compare the detected angle with a threshold angle in step 950. Specifically, the processor (170) may determine whether the detected angle is less than or equal to the threshold angle.

[0062] If the angle detected in step 950 is determined to be less than or equal to the threshold angle, the computing device (100) can confirm the validity of the matching between the first point (i) and the second point (j) in step 960. That is, the processor (170) can confirm the validity of the matching information. Thereafter, the computing device (100) can proceed to step 850 of FIG. 8.

[0063] Meanwhile, the distance calculated at step 930 is the threshold distance k ) or if the angle detected in step 950 is determined to exceed the threshold angle, the computing device (100) can confirm that the matching of the first point (i) and the second point (j) is invalid in step 970. That is, the processor (170) can confirm that the matching information is invalid. Thereafter, the computing device (100) can proceed to step 850 of FIG. 8.

[0064] Referring again to FIG. 8, the computing device (100) determines a relative pose (T) between a source point and a target point based on at least one valid match at step 850. k ) and pose change (ΔT k ) can be updated. Specifically, the processor (170) generates a new pose change amount (ΔT) as a function of the position (pi') of the first point (i) and the position (pj) of the second point (j) based on the target point. k ), by estimating the pose change (ΔT k ) can be updated (ΔT k =func(p i ', p j )). Then, the processor (170) calculates the pose change amount (ΔT k ) using the relative pose (T k ) can be updated. At this time, the processor (170) updates the previous relative pose (T k ) and pose change (ΔT k) as the product of the new relative pose (T k ) can be estimated (ΔT k *T k -1 =T k ) At this time, if there are multiple valid matchings, the processor (170) can estimate a relative pose based on each matching, estimate relative poses corresponding to all matchings, and estimate a final relative pose by combining such relative poses.

[0065] Next, the computing device (100) generates a relative pose (T) at step 860. k ) and pose change (ΔT k ) can determine whether to terminate the update. If it is determined that it should not terminate at step 860, the computing device (100) can return to step 820. At this time, the processor (170) can increase the number of repetitions (k). Then, the computing device (100) can repeat steps 820 to 850. After this, the computing device (100) can proceed to step 860 again. Meanwhile, if it is determined that it can terminate at step 860, the computing device (100) can return to step 730 of FIG. 7.

[0066] Specifically, the processor (170) repeats the relative pose (T) a predetermined number of times in step 860. k ) and pose change (ΔT k) can be determined whether the update has been made. In other words, the processor (170) can determine whether the current repetition number (k) has reached a predetermined number of repetitions. If the repetition number (k) has not reached the predetermined number of repetitions in step 860, the processor (170) can increase the repetition number (k). At this time, if the update in the previous step 850 was made from the relative pose (T0) set in step 810, the processor (170) can set the repetition number (k) to 1. Meanwhile, if the update in the previous step 850 was made from the relative pose (T0) updated in the previous step 850, k ), the processor (170) can increase the number of repetitions (k) by 1. Meanwhile, if the number of repetitions (k) reaches a predetermined number of repetitions in step 860, the processor (170) can return to step 730 of FIG. 7 without increasing the number of repetitions (k).

[0067] In this way, the computing device (100) repeats the relative pose (T) a predetermined number of times. k ) and pose change (ΔT k ) can be updated. This is the relative pose (T k ) can also be repeatedly changed to update the threshold. At this time, the threshold distance is the previously estimated pose change amount (ΔT k ) can be calculated using.

[0068] Referring back to FIG. 7, the computing device (100) can determine the relative pose between the points where the first point cloud and the second point cloud were collected in step 730. That is, the processor (170) determines the relative pose (T) at which the update is finished, i.e., updated by the number of repetitions. k ) to the final relative pose (T k ) can be determined.

[0069] According to the present disclosure, it is possible to estimate relative positions of collected point clouds with high accuracy and relatively short computation time. Specifically, the computing device (100) can estimate relative positions of collected point clouds by repeatedly changing the reference distance for each point in the point clouds and matching points within the reference distance. That is, the reference distance varies for each point, and may also vary for each point depending on the repetition. In this way, the computing device (100) can adaptively operate for each point without requiring prior information about the scale of the environment or information from previous steps. At this time, the computing device (100) can achieve accurate matching of points by considering the normal vector for each point. Here, the computing device (100) can obtain more accurate matching results of point clouds by detecting the normal vector based on the position of the sensor module (110) or the sensor module within the device, rather than the floor, the center of the computing device (100), or the device. Therefore, the computing device (100) can estimate the relative positions of the collected point clouds with high accuracy.

[0070]

[0071] In summary, the present disclosure provides a computing device (100) for relative position recognition through point cloud comparison and an operating method thereof.

[0072] The method of operating a computing device (100) that acquires a plurality of point clouds collected at different points using a sensor module (110) of the present disclosure comprises repeatedly determining a reference distance (threshold) for each first point (i) in the first point cloud. k), a step (step 720) of matching the second point (j) in the second point cloud to the first point (i), and a relative pose (T) between the points where the first point cloud and the second point cloud were collected, respectively. k ) may include a step of determining (step 730).

[0073] In this disclosure, a threshold distance k ) is the sensing distance (r) from the sensor module (110) to the first point (i) for each repetition. i ) and the estimated pose change (ΔT) in the previous iteration k - 1) It can be changed depending on the function.

[0074] In the present disclosure, the step (step 720) of matching the second point (j) to the first point (i) is to determine the relative pose (T) between the source point from which the first point cloud was collected and the target point from which the second point cloud was collected. k - 1) Step 810 of initializing (k=1), relative pose (T k - 1) Using the source point reference, the positions (p) of the first points (i) of the first point cloud are determined based on the target point reference. i -> p i ') a step of transforming the first point (i) (step 820), a step of matching the second point (j) closest to the second point cloud (step 830) for each first point (i), and a step of generating a relative pose (T) between the source point and the target point based on the matching of at least one valid first point (i) and the second point (j). k ) may include a step of updating (step 850).

[0075] In the present disclosure, after the step of updating the relative pose (step 850), the positions (p) of the first points (i) of the first point cloud i -> pi ') and return to the step (step 820) of converting the first points (i) of the first point cloud (k+1=k), and the positions (p) of the first points (i) of the first point cloud i -> p i ') to the step of converting the relative pose (step 820) to the relative pose (T k ) can be repeated (step 850).

[0076] In this disclosure, the relative pose (T k ) is determined (step 730), the last updated relative pose (T) at the end of the iteration k ) to the final relative pose (T k ) can be determined.

[0077] In the present disclosure, the operating method of the computing device (100) may further include a step (step 840) of determining the validity of the matching of each of the first point (i) and the second point (j).

[0078] In the present disclosure, the step of determining validity (step 840) is a threshold distance from the first point (i) k ) (step 910), the distance between the first point (i) and the second point (j) is calculated as a threshold distance k ) below (steps 920 and 930), a step of determining that it is valid (step 960) may be included.

[0079] In the present disclosure, the step of updating the relative pose (step 850) is to update the positions (p) of the first point (i) and the second point (j). i ', p j ) as a function of pose change (ΔT k ) and the pose change amount (ΔT k ) using the relative pose (T k ) may include a step of updating.

[0080] In this disclosure, a threshold distance k) is calculated (step 910), the position (p) of the first point (i) relative to the source point from the source point i ) sensing distance (r) i ) and the previously estimated pose change (ΔT k - 1) as a function of threshold distance k ) may include a step of calculating.

[0081] In this disclosure, the pose change amount (ΔT k - 1) is the rotational change of the source point (δR k - 1) and displacement change (δt k-1 ) and includes a threshold distance k ) is the sensing distance (r i ), rotation change (δR k-1 ), and displacement change (δt k-1 ) can be calculated from.

[0082] In the present disclosure, the step of determining validity (step 840) may further include a step of determining validity (step 960) if the angle between the normal vector of the first point (i) to the target point and the normal vector of the second point (j) to the target point is less than or equal to a threshold angle (steps 940 and 950).

[0083] In the present disclosure, the normal vector of the first point (i) and the normal vector of the second point (j) can be detected based on the position of the sensor module (110) at the target position.

[0084] In the present disclosure, the sensor module (110) may include a lidar sensor.

[0085] A computing device (100) for acquiring a plurality of point clouds collected at different points using a sensor module (110) of the present disclosure includes a memory (160), and a processor (170) connected to the memory (160) and configured to execute at least one command stored in the memory (160), wherein the processor (170) repeatedly determines a threshold distance for each first point (i) in the first point cloud. k ), and can be configured to match the second point (j) within the second point cloud to the first point (i), and estimate the relative pose between the points where the first point cloud and the second point cloud were collected, respectively.

[0086] In this disclosure, a threshold distance k ) is the sensing distance (r) from the sensor module (110) to the first point (i) for each repetition. i ) and the estimated pose change (ΔT) in the previous iteration k - 1) It can be changed depending on the function.

[0087] In the present disclosure, the processor (170) provides a relative pose (T) between a source point from which a first point cloud is collected and a target point from which a second point cloud is collected. k - 1) Initialize (k=1) and set the relative pose (T k - 1) Using the source point reference, the positions (p) of the first points (i) of the first point cloud are determined based on the target point reference. i -> p i ') are transformed, and for each first point (i), the closest second point (j) among the second point clouds is matched, and the relative pose (T) between the source point and the target point is determined based on the matching of at least one valid first point (i) and second point (j). k ) can be configured to update.

[0088] In the present disclosure, the processor (170) updates the relative pose and then determines the positions (p) of the first points (i) of the first point cloud. i -> p i ') to return to the transformation, and the positions (p) of the first points (i) of the first point cloud i -> p i ') to transform the action or relative pose (T k ) is repeated, and when the repetition ends, the last updated relative pose (T k ) to the final relative pose (T k ) can be configured to determine.

[0089] In the present disclosure, the processor (170) determines a threshold distance from the first point (i) k ) is calculated, and the distance between the first point (i) and the second point (j) is the threshold distance k ) or less, it can be configured to determine that it is valid.

[0090] In the present disclosure, the processor (170) updates the pose change amount as a function of the positions of the first point (i) and the second point (j) of the matching, and the pose change amount (ΔT k ) can be configured to update the relative pose.

[0091] In the present disclosure, the processor (170) determines the position (p) of the first point (i) relative to the source point from the source point i ) sensing distance (r) i ) and the previously estimated pose change (ΔT k - 1) as a function of threshold distance k ) can be configured to calculate.

[0092] In this disclosure, the pose change amount (ΔT k - 1) is the rotational change of the source point (δR k- 1) and displacement change (δt k-1 ) and includes a threshold distance k ) is the sensing distance (r i ), rotation change (δR k-1 ), and displacement change (δt k-1 ) can be calculated from.

[0093] In the present disclosure, the processor (170) may be configured to determine that the target point is valid if the angle between the normal vector of the first point (i) to the target point and the normal vector of the second point (j) to the target point is less than or equal to a threshold angle.

[0094] In the present disclosure, the normal vector of the first point (i) and the normal vector of the second point (j) can be detected based on the position of the sensor module (110) at the target position.

[0095] In the present disclosure, the sensor module (110) may include a lidar sensor.

[0096] The above-described method may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may be one that continuously stores a computer-executable program, or one that temporarily stores it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program instructions, including ROM, RAM, and flash memory. In addition, examples of other media may include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0097] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will appreciate that the various exemplary logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software will depend on the particular application and design requirements imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementations should not be construed as causing a departure from the scope of the present disclosure.

[0098] In a hardware implementation, the processing units used to perform the techniques may be implemented within one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, a computer, or a combination thereof.

[0099] Accordingly, the various exemplary logical blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0100] In a firmware and / or software implementation, the techniques may be implemented as instructions stored on a computer-readable medium, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, a compact disc (CD), a magnetic or optical data storage device, etc. The instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described herein.

[0101] While the embodiments described above have been described as utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or distributed computing environment. Furthermore, aspects of the subject matter in the present disclosure may be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include personal computers, network servers, and portable devices.

[0102] While this disclosure has been described with reference to certain embodiments, various modifications and variations can be made without departing from the scope of the present disclosure, which would be apparent to those skilled in the art. Furthermore, such modifications and variations should be considered to fall within the scope of the claims appended hereto.

Claims

1. In a method of operating a computing device that acquires multiple point clouds collected from different points using a sensor module, A step of matching a second point in a second point cloud to each first point while repeatedly changing the reference distance for each first point in the first point cloud; and A step of determining a relative pose between points where the first point cloud and the second point cloud are collected, respectively. Including, The above reference distance changes at each repetition according to a function of the sensing distance from the sensor module to the first point and the pose change amount estimated at the previous repetition. How a computing device operates.

2. In paragraph 1, The step of matching the second point to the first point is: A step of initially setting a relative pose between a source point from which the first point cloud is collected and a target point from which the second point cloud is collected; A step of transforming the positions of the first points of the first point cloud from the source point reference to the target point reference using the relative pose; A step of matching the second point closest to each first point among the second point clouds; and A step of updating the relative pose between the source point and the target point based on a matching of at least one valid first point and second point. including, How a computing device operates.

3. In paragraph 2, After the step of updating the relative pose, return to the step of transforming the positions of the first points of the first point cloud, The step of converting the positions of the first points of the first point cloud and the step of updating the relative pose are repeated, The step of determining the above relative pose is: At the end of the iteration, the last updated relative pose is determined as the final relative pose. How a computing device operates.

4. In paragraph 2, A step for judging the validity of the matching of each first point and second point Including more, The steps for judging the above validity are: a step of calculating a reference distance from the first point; and A step of determining that the validity is present if the distance between the first point and the second point is less than or equal to the calculated reference distance. including, How a computing device operates.

5. In paragraph 4, The step of updating the above relative pose is: A step of updating the pose change amount as a function of the positions of the first point and the second point; and A step of updating the relative pose using the above pose change amount. including, How a computing device operates.

6. In paragraph 5, The step of calculating the above reference distance is: A step of calculating the reference distance as a function of the sensing distance from the source point to the position of the first point based on the source point and the previously estimated pose change amount. including, How a computing device operates.

7. In paragraph 6, The above pose change amount includes the rotation change amount and the displacement change amount of the source point, The above reference distance is calculated from the sensing distance, the rotation change amount, and the displacement change amount. How a computing device operates.

8. In paragraph 4, The steps for judging the above validity are: A step of determining that the method is valid if the angle between the normal vector of the first point to the target point and the normal vector of the second point to the target point is less than or equal to a critical angle. including more, How a computing device operates.

9. In paragraph 8, The normal vector of the first point and the normal vector of the second point are detected based on the position of the sensor module at the target position. How a computing device operates.

10. In paragraph 1, The above sensor module includes a lidar sensor, How a computing device operates.

11. A computer program stored in a non-transitory computer-readable recording medium for executing the method of any one of claims 1 to 10 on the computing device.

12. In a computing device that acquires multiple point clouds collected from different points using a sensor module, memory; and A processor connected to the memory and configured to execute at least one instruction stored in the memory, The above processor, Matching a second point in a second point cloud to each first point by repeatedly changing the reference distance for each first point in the first point cloud, It is configured to estimate the relative pose between the points where the first point cloud and the second point cloud are collected, respectively, The above reference distance changes at each repetition according to a function of the sensing distance from the sensor module to the first point and the pose change amount estimated at the previous repetition. Computing device.

13. In paragraph 12, The above processor, Initialize the relative pose between the source point from which the first point cloud was collected and the target point from which the second point cloud was collected, Using the above relative pose, the positions of the first points of the first point cloud are transformed from the source point reference to the target point reference, For each first point, match the second point closest to the second point cloud, configured to update the relative pose between the source point and the target point based on a matching of at least one valid first point and a second point, Computing device.

14. In paragraph 13, The above processor, After updating the relative pose, return to transform the positions of the first points of the first point cloud, Repeating the operation of transforming the positions of the first points of the first point cloud or the operation of updating the relative pose, It is configured to determine the last updated relative pose as the final relative pose at the end of the iteration. Computing device.

15. In paragraph 13, The above processor, Calculate the reference distance from the first point above, If the distance between the first point and the second point is less than or equal to the reference distance, it is determined to be valid. Computing device.

16. In paragraph 15, The above processor, Update the pose change amount as a function of the positions of the first point and the second point of the above matching, configured to update the relative pose using the above pose change amount, Computing device.

17. In paragraph 16, The above processor, configured to calculate the reference distance as a function of the sensing distance from the source point to the position of the first point relative to the source point and the previously estimated pose change amount, Computing device.

18. In paragraph 17, The above pose change amount includes the rotation change amount and the displacement change amount of the source point, The above reference distance is calculated from the sensing distance, the rotation change amount, and the displacement change amount. Computing device.

19. In paragraph 15, The above processor, If the angle between the normal vector of the first point to the target point and the normal vector of the second point to the target point is less than or equal to a threshold angle, it is determined to be valid. Computing device.

20. In paragraph 19, The normal vector of the first point and the normal vector of the second point are detected based on the position of the sensor module at the target position. Computing device.

Images