Electronic device and method for processing point cloud data

Abstract

Disclosed is an electronic device comprising a processor and a memory operatively connected to the processor. The memory stores instructions that, when executed, cause the processor to: receive a plurality of pieces of actual point cloud data acquired by sensing an actual space a designated number of times by using an actual sensor; receive a plurality of pieces of virtual point cloud data acquired by sensing a virtual space, which is obtained by simulating the actual space, a designated number of times by means of a virtual sensor that simulates the actual sensor; and determine the level of simulation between the actual sensor and the virtual sensor on the basis of the plurality of pieces of actual point cloud data and the plurality of pieces of virtual point cloud data.

Description

Electronic device and method for processing point cloud data

[0001] The present disclosure relates to an electronic device and method for processing point cloud data.

[0002]

[0003] Virtual spaces that simulate real spaces (or real spaces) can serve as testbeds, predicting potential real-world events through events occurring in the virtual space and processing these events to appropriately respond to them. These virtual spaces are being utilized in industries that require more sophisticated technological implementation through simulation, such as simulated driving and autonomous driving.

[0004] Just as in real space, virtual space can support sensors capable of detecting objects within the space. These sensors are virtual sensors that mimic real sensors (or physical sensors), and are required to implement the same functions and operations as the real sensors. For example, a virtual sensor must have a certain level of accuracy in replicating the real sensor to be considered reliable. Accordingly, evaluating the accuracy of a virtual sensor relative to a real sensor is essential for the development of virtual sensor technology and is becoming a crucial element in industries utilizing virtual space.

[0005] The recent surge in popularity of virtual data stems from the ability to secure large amounts of corner case data, which are difficult to obtain with real-world data alone, at low cost. This access to large amounts of corner case data can improve the performance of AI models. However, even with the largest data volume, if the virtual data lacks the ability to replicate actual sensors, the resulting virtual data will differ from the real data, ultimately failing to positively impact AI model performance.

[0006]

[0007] The present disclosure provides an electronic device and method for processing point cloud data to solve the above problems.

[0008]

[0009] The present disclosure can be implemented in various ways, including methods, devices (systems), and / or computer programs stored on a computer-readable storage medium.

[0010] According to one embodiment of the present disclosure, an electronic device includes a processor and a memory operatively connected to the processor, wherein the memory may store instructions that, when executed, cause the processor to receive a plurality of real point cloud data acquired by sensing a real space a specified number of times using a real sensor, receive a plurality of virtual point cloud data acquired by sensing a virtual space simulating the real space a specified number of times using a virtual sensor simulating the real sensor, and determine a degree of similarity between the real sensor and the virtual sensor based on the plurality of real point cloud data and the plurality of virtual point cloud data.

[0011] According to one embodiment, determining a similarity may include identifying a first voxel in a three-dimensional space represented by a plurality of real point cloud data, identifying a second voxel corresponding to the first voxel in the three-dimensional space represented by the plurality of virtual point cloud data, calculating a representative value associated with the first voxel for each of the plurality of real point cloud data to determine a first set of representative values, calculating a representative value associated with the second voxel for each of the plurality of virtual point cloud data to determine a second set of representative values, and calculating a similarity between the plurality of real point cloud data and the plurality of virtual point cloud data based on the first set of representative values ​​associated with the first voxel and the second set of representative values ​​associated with the second voxel.

[0012] In one embodiment, identifying the first voxel may include selecting a first voxel associated with a location of a real object disposed in real space, and identifying the second voxel may include selecting a second voxel associated with a virtual object disposed in virtual space and corresponding to the real object.

[0013] According to one embodiment, the plurality of real point cloud data may include first real point cloud data and second real point cloud data, and determining a representative value of the first set may include identifying a first set of points associated with a first voxel among points included in the first real point cloud data, identifying a second set of points associated with the first voxel among points included in the second real point cloud data, calculating an average coordinate of the identified first set of points, calculating an average coordinate of the identified second set of points, determining the average coordinate of the calculated first set of points as a first representative value associated with the first voxel of the first real point cloud data, and determining the average coordinate of the calculated second set of points as a second representative value associated with the first voxel of the second real point cloud data.

[0014] In one embodiment, calculating the similarity may include calculating the similarity between the plurality of real point cloud data and the plurality of virtual point cloud data based on a number of representative values ​​of a first set associated with the first voxel and a number of representative values ​​of a second set associated with the second voxel.

[0015] In one embodiment, calculating the similarity may include calculating a distance between each of a first set of representative values ​​associated with a first voxel and an actual sensor to determine a first set of distance values, calculating a distance between each of a second set of representative values ​​associated with a second voxel and a virtual sensor to determine a second set of distance values, and calculating a similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on the first set of distance values ​​and the second set of distance values.

[0016] In one embodiment, calculating the similarity may include generating a first probability variable for a first voxel through probability distribution fitting using a first set of representative values ​​associated with a plurality of real point cloud data, calculating a first covariance matrix associated with the first voxel based on the first probability variable, generating a second probability variable for a second voxel through probability distribution fitting using a second set of representative values ​​associated with the plurality of virtual point cloud data, calculating a second covariance matrix associated with the second voxel based on the second probability variable, and calculating a similarity between the plurality of real point cloud data and the plurality of virtual point cloud data based on the first covariance matrix and the second covariance matrix.

[0017] According to one embodiment, calculating the similarity further includes calculating a first mean associated with the first voxel based on the first random variable, and calculating a second mean associated with the second voxel based on the second random variable, wherein the similarity may be calculated further based on the first mean and the second mean.

[0018] According to one embodiment, determining a similarity may include identifying a first voxel in a three-dimensional space represented by a plurality of real point cloud data, identifying a second voxel corresponding to the first voxel in the three-dimensional space represented by the plurality of virtual point cloud data, calculating a mean and a covariance matrix associated with the first voxel for each of the plurality of real point cloud data to determine a first set of representative values, calculating a mean and a covariance matrix associated with the second voxel for each of the plurality of virtual point cloud data to determine a second set of representative values, and calculating a similarity between the plurality of real point cloud data and the plurality of virtual point cloud data based on the first set of representative values ​​associated with the first voxel and the second set of representative values ​​associated with the second voxel.

[0019] According to one embodiment of the present disclosure, a method for processing point cloud data may include a step of receiving a plurality of real point cloud data acquired by sensing a real space a specified number of times using a real sensor, a step of receiving a plurality of virtual point cloud data acquired by sensing a virtual space simulating a real space a specified number of times using a virtual sensor simulating the real sensor, and a step of determining a degree of similarity between a real sensor and a virtual sensor based on the plurality of real point cloud data and the plurality of virtual point cloud data.

[0020]

[0021] According to some embodiments of the present disclosure, the degree of similarity between a real sensor and a virtual sensor can be improved by measuring / evaluating the degree of similarity between a plurality of real point cloud data and a plurality of virtual point cloud data.

[0022] In addition, according to some embodiments of the present disclosure, by dividing a three-dimensional space represented by point cloud data into a plurality of voxels and calculating a similarity between point cloud data based on a representative value associated with each voxel, it is possible to support measuring a similarity with respect to noise elements of a sensor.

[0023] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary knowledge in the technical field to which the present disclosure belongs (referred to as “ordinary skilled person”) from the description of the claims.

[0024]

[0025] Embodiments of the present disclosure will be described below with reference to the accompanying drawings, wherein like reference numerals represent similar elements, but are not limited thereto.

[0026] FIG. 1 is a diagram for explaining the configuration of an electronic device for processing point cloud data according to one embodiment of the present disclosure.

[0027] FIG. 2 is a schematic diagram showing a configuration in which an information processing system is connected to enable communication with a plurality of user terminals in relation to data processing according to one embodiment of the present disclosure.

[0028] FIG. 3 is a block diagram showing the internal configuration of a user terminal and an information processing system according to one embodiment of the present disclosure.

[0029] FIG. 4 is a drawing for explaining a virtual space that simulates an actual space of an outdoor environment according to one embodiment of the present disclosure.

[0030] FIG. 5 is a drawing for explaining a virtual space simulating an actual space of an indoor environment according to one embodiment of the present disclosure.

[0031] FIG. 6 is a diagram for explaining point cloud data acquired through a sensor according to one embodiment of the present disclosure.

[0032] FIG. 7 is a diagram for explaining a method for selecting data of a specified area from point cloud data according to one embodiment of the present disclosure.

[0033] FIG. 8 is a diagram illustrating a method for voxelizing point cloud data according to one embodiment of the present disclosure.

[0034] FIG. 9 is a diagram illustrating a method for calculating similarity between point cloud data based on a set of representative values ​​associated with voxels according to one embodiment of the present disclosure.

[0035] FIG. 10 is a diagram illustrating a method for processing point cloud data according to one embodiment of the present disclosure.

[0036]

[0037] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions of widely known functions or configurations will be omitted if they may unnecessarily obscure the gist of the present disclosure.

[0038] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Furthermore, in the description of the embodiments below, duplicate descriptions of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0039] The advantages and features of the disclosed embodiments, and methods for achieving them, will become clearer with reference to the embodiments described below, along with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure the completeness of the disclosure and to fully inform those skilled in the art of the scope of the invention.

[0040] The terms used in this specification will be briefly explained, followed by a detailed description of the disclosed embodiments. The terms used in this specification have been selected from widely used, current terms, taking into account the functions of the present disclosure. However, these terms may vary depending on the intentions of engineers working in the relevant field, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on their meanings and the overall content of the present disclosure.

[0041] In this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, plural expressions include singular expressions unless the context clearly indicates otherwise. When a part of the specification is said to include a component, this does not exclude other components, but rather implies that other components may be included, unless otherwise specifically stated.

[0042] Also, the term 'module' or 'part' used in the specification means a software or hardware component, and the 'module' or 'part' performs certain roles. However, the 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. Thus, as an example, the 'module' or 'part' may include at least one of components such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, or variables. The functionality provided within the components and 'modules' or 'parts' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0043] According to one embodiment of the present disclosure, a 'module' or 'unit' may be implemented as a processor and a memory. 'Processor' should be broadly construed to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, and the like. In some circumstances, a 'processor' may also refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field-programmable gate array (FPGA), and the like. A 'processor' may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other such combination of configurations. In addition, 'memory' should be broadly construed to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or marking data storage, registers, etc. Memory is said to be in electronic communication with the processor if the processor can read information from, and / or write information to, the memory. Memory integrated in a processor is in electronic communication with the processor.

[0044] In addition, terms such as first, second, A, B, (a), (b), etc. used in the following embodiments are only used to distinguish certain components from other components, and the nature, order, or sequence of the components are not limited by the terms.

[0045] Additionally, in the embodiments below, when it is described that a component is 'connected', 'coupled' or 'connected' to another component, it should be understood that the component may be directly connected or connected to the other component, but another component may also be 'connected', 'coupled' or 'connected' between each component.

[0046] Additionally, the words 'comprises' and / or 'comprising' used in the following embodiments do not exclude the presence or addition of one or more other components, steps, operations and / or elements.

[0047] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0048] Sensors that detect objects in space (real and virtual sensors) can acquire point cloud data. For example, a LiDAR (Light Detection And Ranging) sensor can be used as a sensor that detects objects. A point cloud is a collection of points expressed as coordinate information in three-dimensional space, and each point can represent location information in space. This point cloud data is data acquired by real and virtual sensors, and because it is the result of the sensors sensing space, the similarity between the point cloud data acquired by the real sensor and the point cloud data acquired by the virtual sensor can be measured to measure the degree of imitation of the virtual sensor with respect to the real sensor.

[0049] FIG. 1 is a diagram illustrating a configuration of an electronic device (100) for processing point cloud data according to an embodiment of the present disclosure. Referring to FIG. 1, the electronic device (100) for processing point cloud data may include a processor (110) and a memory (150) operatively connected to the processor (110). However, the configuration of the electronic device (100) is not limited thereto. According to various embodiments, the electronic device (100) may further include at least one other component in addition to the above-described components. As an example, the electronic device (100) may further include a communication circuit (130) for communicating with an external electronic device. The communication circuit (130) may, for example, support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (100) and an external electronic device (e.g., a first external electronic device (102) or a second external electronic device (104)) and the performance of communication through the established communication channel. According to one embodiment, the electronic device (100) can receive point cloud data acquired using a sensor (e.g., a first sensor (102a) or a second sensor (104a)) from an external electronic device via a communication circuit (130).

[0050] The processor (110) may execute software (or a program) to control at least one other component (e.g., a hardware or software component) of the electronic device (100) connected to the processor (110) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (110) may load a command or data received from another component (e.g., a communication circuit (130)) into a volatile memory, process the command or data stored in the volatile memory, and store the resulting data in a non-volatile memory.

[0051] The memory (150) can store various data used by at least one component (e.g., the processor (110)) of the electronic device (100). The data can include, for example, input data or output data for software (or a program) and commands related thereto. The memory (150) can include volatile memory or non-volatile memory. According to one embodiment, the memory (150) can store point cloud data received via the communication circuit (130).

[0052] The memory (150) may include at least one instruction related to processing of point cloud data. The at least one instruction may include, for example, instructions related to acquisition, preprocessing, processing, and analysis of point cloud data. The memory (150) may include a data acquisition module (152), a data preprocessing module (154), a data processing module (156), and a data analysis module (158). However, the types of modules included in the memory (150) are classified according to the function of the instructions, and the types and numbers thereof are not limited thereto. In addition, the modules (or instructions included in the modules) included in the memory (150) are executed by the processor (110), and may be implemented by the processor (110) itself.

[0053] The data acquisition module (152) can acquire point cloud data. According to one embodiment, the data acquisition module (152) can receive actual point cloud data acquired by sensing an actual space using a first sensor (102a) from a first external electronic device (102) via a communication circuit (130). Here, the first sensor (102a) may represent an actual sensor capable of detecting at least one object placed in an actual space. In addition, the actual point cloud data acquired by using the first sensor (102a) may be referred to as point cloud data of an actual space, actual point cloud data, first point cloud data, etc. According to one embodiment, the first sensor (102a) may include a lidar sensor. According to one embodiment, the data acquisition module (152) can receive virtual point cloud data acquired by sensing a virtual space using a second sensor (104a) from a second external electronic device (104) via a communication circuit (130). Here, the virtual space may represent a space that simulates an actual space, and the second sensor (104a) may represent a virtual sensor that simulates the first sensor (102a). In addition, the virtual point cloud data acquired using the second sensor (104a) may be referred to as point cloud data of the virtual space or second point cloud data, etc. According to one embodiment, the second sensor (104a) may include a virtual lidar sensor that can detect at least one object placed in the virtual space. However, the type of the sensor and the method of acquiring data are not limited thereto. According to various embodiments, at least one of the first sensor (102a) or the second sensor (104a) may be a component included in the electronic device (100).In this case, the data acquisition module (152) can acquire point cloud data through simulation using a sensor virtually implemented within the electronic device (100) instead of acquiring point cloud data from an external electronic device through a communication circuit (130).

[0054] If the first sensor (102a) (or actual sensor) is a sensor (e.g., a lidar sensor) that is sensitive to noise (e.g., rapid temperature change, vibration, etc.) and the second sensor (104a) (or virtual sensor) is a sensor that imitates the second sensor (104a) (e.g., a virtual lidar sensor), the data acquisition module (152) can acquire a plurality of actual point cloud data and a plurality of virtual point cloud data. Here, the plurality of actual point cloud data are acquired by sensing an actual space a specified number of times (e.g., 250 times) using the first sensor (102a), and the actual point cloud data acquired through one sensing can be configured as one frame. For example, the plurality of actual point cloud data can include a number of frames corresponding to the specified number of times (e.g., 250 frames). In addition, the plurality of virtual point cloud data are obtained by sensing the virtual space a specified number of times (e.g., 250 times) using the second sensor (104a), and the virtual point cloud data obtained by one sensing can also be configured as one frame. For example, the plurality of virtual point cloud data can include a number of frames corresponding to the specified number of times (e.g., 250 frames). In this way, by measuring the mimicry of the second sensor (104b) (or virtual sensor) with respect to the first sensor (102a) (or actual sensor) using the plurality of point cloud data (or the plurality of frames of point cloud data) obtained by sensing the same space multiple times, it is possible to support measuring the mimicry even for noise elements of the sensors (e.g., the first sensor (102a) and the second sensor (104a)).

[0055] The data preprocessing module (154) can divide the three-dimensional space represented by the point cloud data into a plurality of regions. According to one embodiment, the data preprocessing module (154) can divide the three-dimensional space represented by the point cloud data into a plurality of voxels. Here, the voxel can represent the minimum unit in the three-dimensional space and can have the shape of a cube that grids the three-dimensional space. According to one embodiment, the data preprocessing module (154) can divide the three-dimensional space represented by the plurality of actual point cloud data into a plurality of first voxels, and divide the three-dimensional space represented by the plurality of virtual point cloud data into a plurality of second voxels. Here, since the virtual point cloud data is acquired using the second sensor (104a) that simulates the first sensor (102a), it can have a three-dimensional space corresponding to the actual point cloud data, and accordingly, the plurality of second voxels can also correspond to the plurality of first voxels. In addition, the first voxel that divides the three-dimensional space represented by the actual point cloud data may be referred to as a voxel of the actual space or an actual voxel, etc., and the second voxel that divides the three-dimensional space represented by the virtual point cloud data may be referred to as a voxel of the virtual space or a virtual voxel, etc.

[0056] The data preprocessing module (154) may select data of a specified area from point cloud data. For example, the data preprocessing module (154) may select at least some of a plurality of voxels that divide the three-dimensional space represented by the point cloud data. According to one embodiment, the data preprocessing module (154) may select a plurality of first voxels to correspond to the location of at least one real object placed in real space. According to one embodiment, the data preprocessing module (154) may select a first voxel associated with the location of the real object placed in real space. Here, the first object may represent an object of interest. In addition, the data preprocessing module (154) may select a plurality of second voxels to correspond to the location of at least one virtual object placed in virtual space and corresponding to at least one real object. According to one embodiment, the data preprocessing module (154) may select a second voxel associated with a virtual object placed in virtual space and corresponding to the real object. Here, the selected voxels may be voxels included in a region of interest (ROI) in which the object of interest is located.

[0057] The data preprocessing module (154) can identify voxels in a three-dimensional space represented by point cloud data, and calculate a representative value associated with the identified voxels for the point cloud data to determine a set of representative values. Here, the set can represent a set of representative values ​​associated with the identified voxels (or representative values ​​that the identified voxels can have), and the representative value associated with the identified voxels can be obtained for each point cloud data, and can be sample information corresponding to the corresponding point cloud data of each of the plurality of voxels. According to one embodiment, the data preprocessing module (154) can identify a first voxel in a three-dimensional space represented by a plurality of actual point cloud data, and calculate a representative value associated with the first voxel for each of the plurality of actual point cloud data, thereby determining a first set of representative values. According to one embodiment, the data preprocessing module (154) can identify a second voxel in a three-dimensional space represented by a plurality of virtual point cloud data, and calculate a representative value associated with the second voxel for each of the plurality of virtual point cloud data, thereby determining a second set of representative values.

[0058] The data preprocessing module (154) may identify a point set associated with each of a plurality of voxels among the points included in the point cloud data, calculate an average coordinate of the identified point set, and determine the average coordinate of the calculated point set as a representative value associated with the voxel of the point cloud data. Here, the point set may represent a set of points included in the voxel. According to one embodiment, when the plurality of actual point cloud data includes first actual point cloud data and second actual point cloud data, the data preprocessing module (154) may identify a first point set associated with a first voxel among the points included in the first actual point cloud data, calculate an average coordinate of the identified first point set, and determine the average coordinate of the calculated first point set as a first representative value associated with the first voxel of the first actual point cloud data. Additionally, the data preprocessing module (154) may identify a second set of points associated with a first voxel among points included in the second actual point cloud data, calculate an average coordinate of the identified second set of points, and determine the average coordinate of the calculated second set of points as a second representative value associated with the first voxel of the second actual point cloud data. In this case, the first set of representative values ​​associated with the first voxel may include the first representative value associated with the first voxel of the first actual point cloud data and the second representative value associated with the first voxel of the second actual point cloud data.According to one embodiment, when the plurality of virtual point cloud data includes first virtual point cloud data and second virtual point cloud data, the data preprocessing module (154) may identify a third set of points associated with a second voxel among points included in the first virtual point cloud data, calculate an average coordinate of the identified third set of points, and determine the average coordinate of the calculated third set of points as a third representative value associated with the second voxel of the first virtual point cloud data. In addition, the data preprocessing module (154) may identify a fourth set of points associated with a second voxel among points included in the second virtual point cloud data, calculate an average coordinate of the identified fourth set of points, and determine the average coordinate of the calculated fourth set of points as a fourth representative value associated with the second voxel of the second virtual point cloud data. In this case, the second set of representative values ​​associated with the second voxel may include a third representative value associated with the second voxel of the first virtual point cloud data and a fourth representative value associated with the second voxel of the second virtual point cloud data.

[0059] The operation of determining the representative value associated with the above-described voxel (or sample information corresponding to the point cloud data of the voxel) can be performed as many times as the number of point cloud data (i.e., the number of frames). For example, when a plurality of point cloud data are composed of n frames (e.g., when the point cloud data are acquired through n sensing operations), the data preprocessing module (154) can determine (or set) a representative value associated with each of the plurality of voxels for each of the plurality of point cloud data. For example, when a plurality of actual point cloud data are composed of three frames (e.g., a first frame of actual point cloud data, a second frame of actual point cloud data, and a third frame of actual point cloud data), the data preprocessing module (154) can determine a first representative value associated with each of the plurality of first voxels for the first frame of the actual point cloud data, determine a second representative value associated with each of the plurality of first voxels for the second frame of the actual point cloud data, and determine a third representative value associated with each of the plurality of first voxels for the third frame of the actual point cloud data. Accordingly, each of the plurality of voxels may have a representative value less than or equal to the number of frames (e.g., a first representative value, a second representative value, and a third representative value). Here, the reason why each of the plurality of voxels has a representative value less than or equal to the number of frames is that, even when sensing is performed in the same environment, the positions of points included in the point cloud data may change or some points may not be acquired due to noise factors of the sensor. That is, if there is no point included in a voxel due to noise, the data preprocessing module (154) may not be able to determine (or set) the representative value associated with the voxel. In this case, the representative value associated with the corresponding voxel of the corresponding point cloud data may not be included in the set of representative values ​​associated with the corresponding voxel.

[0060] The data processing module (156) can calculate the similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data. According to one embodiment, the data processing module (156) can calculate the similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on a set of actual representative values ​​and a set of virtual representative values ​​associated with each of the plurality of voxels. According to one embodiment, the data processing module (156) can calculate the similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on a first set of representative values ​​associated with a first voxel (or an actual voxel) and a second set of representative values ​​associated with a second voxel (or a virtual voxel).

[0061] According to one embodiment, the data processing module (156) may calculate a similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on the number of representative values ​​of the first set associated with each of the plurality of first voxels (or actual voxels) and the number of representative values ​​of the second set associated with each of the plurality of second voxels (or virtual voxels). For example, the data processing module (156) may calculate a similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on the number of representative values ​​of the first set associated with the first voxel and the number of representative values ​​of the second set associated with the second voxel. Here, the number of representative values ​​of the set associated with each of the voxels may represent the number of set sample information.

[0062] According to one embodiment, the operation of calculating the similarity between point cloud data based on the number of representative value sets associated with each voxel can be performed by the following mathematical expression 1.

[0063]

[0064]

[0065] Here, N represents the number of voxels, NS(V) represents the number of representative value sets associated with voxel V, and V i real represents the i-th first voxel (or actual voxel), and V i virt can represent the ith second voxel (or virtual voxel). M1, calculated through mathematical expression 1, can have a value from 0 to 100, and the closer it is to 100, the greater the similarity between point cloud data can be determined. For example, M1 can represent the average value of the difference between the first number of representative values ​​of the first set associated with the first voxel (or actual voxel) and the second number of representative values ​​of the second set associated with the second voxel (or virtual voxel) and the ratio of the first number.

[0066] According to one embodiment, the data processing module (156) may calculate a first distance value for each of the plurality of first voxels associated with the first sensor (102a) (or the actual sensor) using a first set of representative values ​​associated with each of the plurality of first voxels. In addition, the data processing module (156) may calculate a second distance value for each of the plurality of second voxels associated with the second sensor (104a) (or the virtual sensor) using a second set of representative values ​​associated with each of the plurality of second voxels. Thereafter, the data processing module (156) may calculate a similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on the plurality of first distance values ​​and the plurality of second distance values. For example, the data processing module (156) may determine the first set of distance values ​​by calculating a distance between each of the first set of representative values ​​associated with the first voxels and the first sensor (102a). Similarly, the data processing module (156) can determine the second set of distance values ​​by calculating the distance between each of the second set of representative values ​​associated with the second voxel and the second sensor (104a). Thereafter, the data processing module (156) can calculate the similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on the first set of distance values ​​and the second set of distance values.

[0067] According to one embodiment, the operation of calculating the similarity between point cloud data based on the distance values ​​between the sensor and the voxel calculated (or determined) using a set of representative values ​​associated with each of the voxels can be performed by the following mathematical expression 2.

[0068]

[0069]

[0070] Here, N represents the number of voxels, AD(V) represents the distance value from the sensor calculated based on the set of representative values ​​associated with voxel V, and V i realrepresents the i-th first voxel, and V i virt can represent the i-th second voxel. In addition, when the representative value set associated with voxel V has (x, y, z) coordinate values, the distance value from the sensor calculated based on this is the square root of the sum of the squares of the x-coordinate value, the y-coordinate value, and the z-coordinate value, respectively (square_root(x 2 + y 2 + z 2 ) can be derived from. M2 derived through mathematical expression 2 can have a value from 0 to 100, and the closer it is to 100, the greater the similarity between point cloud data can be determined. For example, M2 can represent the average value of the difference between the first distance value from the actual sensor derived based on the first set of representative values ​​associated with the first voxel (or actual voxel) and the second distance value from the virtual sensor derived based on the second set of representative values ​​associated with the second voxel (or virtual voxel) and the ratio of the first distance value.

[0071] According to one embodiment, the data processing module (156) may generate a first probability variable for each of the plurality of first voxels by performing probability distribution fitting using a first set of representative values ​​associated with each of the plurality of first voxels, and obtain a first covariance matrix associated with each of the plurality of first voxels based on the generated first probability variable. In addition, the data processing module (156) may generate a second probability variable for each of the plurality of second voxels by performing probability distribution fitting using a second set of representative values ​​associated with each of the plurality of second voxels, and obtain a second covariance matrix associated with each of the plurality of second voxels based on the generated second probability variable. Thereafter, the data processing module (156) may calculate a similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on the first covariance matrix and the second covariance matrix. For example, the data processing module (156) may generate a first probability variable for a first voxel through probability distribution fitting using a first set of representative values ​​associated with a plurality of actual point cloud data, calculate a first covariance matrix associated with the first voxel based on the first probability variable, generate a second probability variable for a second voxel through probability distribution fitting using a second set of representative values ​​associated with a plurality of virtual point cloud data, calculate a second covariance matrix associated with the second voxel based on the second probability variable, and calculate a similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on the first covariance matrix and the second covariance matrix.

[0072] According to one embodiment, the operation of calculating the similarity between point cloud data based on the covariance matrix by the probability variable for each voxel can be performed by the following mathematical expression 3.

[0073]

[0074]

[0075] Here, N represents the number of voxels, Cov(V) represents the covariance matrix calculated based on the representative values ​​of the set associated with voxel V, and V i real represents the i-th first voxel, and V i virt can represent the i-th second voxel. Also, in mathematical expression 3, represents the Frobenius norm for the matrix H, which can be calculated from the square root of the sum of the squares of the absolute values ​​of all elements of the matrix H. According to one embodiment, the covariance matrix may include a 3X3 covariance matrix calculated based on representative values ​​expressed in three-dimensional coordinates (e.g., (x, y, z) coordinates). M3 calculated through mathematical expression 3 may have a value from 0 to infinity (∞), and the closer to 0, the greater the similarity between point cloud data may be determined. For example, M3 may represent the average value of the Frobenius norm for the difference between the first covariance matrix calculated based on the first set of representative values ​​associated with the first voxel (or actual voxel) and the second covariance matrix calculated based on the second set of representative values ​​associated with the second voxel (or virtual voxel).

[0076] According to one embodiment, the data processing module (156) may generate a first probability variable for each of the plurality of first voxels through probability distribution fitting using a first set of representative values ​​associated with each of the plurality of first voxels, and obtain a first mean value and a first covariance matrix associated with each of the plurality of first voxels based on the generated first probability variable. In addition, the data processing module (156) may generate a second probability variable for each of the plurality of second voxels through probability distribution fitting using a second set of representative values ​​associated with each of the plurality of second voxels, and obtain a second mean value and a second covariance matrix associated with each of the plurality of second voxels based on the generated second probability variable. Thereafter, the data processing module (156) may calculate a similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on the first mean value, the second mean value, the first covariance matrix, and the second covariance matrix. For example, the data processing module (156) may generate a first probability variable for a first voxel through probability distribution fitting using a first set of representative values ​​associated with a plurality of actual point cloud data, calculate a first mean value and a first covariance matrix associated with the first voxel based on the first probability variable, generate a second probability variable for a second voxel through probability distribution fitting using a second set of representative values ​​associated with a plurality of virtual point cloud data, calculate a second mean value and a second covariance matrix associated with the second voxel based on the second probability variable, and calculate a similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on the first mean value, the second mean value, the first covariance matrix, and the second covariance matrix.

[0077] According to one embodiment, the operation of calculating the similarity between point cloud data based on the mean value and covariance matrix by the probability variable for each voxel can be performed by the following mathematical expression 4.

[0078]

[0079]

[0080]

[0081]

[0082] Here, N represents the number of voxels, W2(X, Y) represents the 2-Wasserstein distance between the probability distribution of X and the probability distribution of Y, and V i real represents the i-th first voxel, and V i virt can represent the ith second voxel. In addition, mean(V) and Cov(V) represent the mean and covariance matrix respectively calculated based on the representative values ​​of the set associated with the voxel V, and trace(H) can represent the sum of the diagonal elements of the square matrix H. M4 calculated through mathematical expression 4 can have a value from 0 to infinity (∞), and the closer it is to 0, the greater the similarity between the point cloud data can be determined. For example, M4 can represent the average value of the values ​​calculated by fitting the representative values ​​of the first set associated with the first voxel (or actual voxel) and the representative values ​​of the second set associated with the second voxel (or virtual voxel) to a 3D Gaussian distribution, and then calculating the distance between the two 3D Gaussian distributions as the 2-Wasserstein distance.

[0083] According to one embodiment, the data processing module (156) may generate a first probability variable for each of the plurality of first voxels through probability distribution fitting using a first set of representative values ​​associated with each of the plurality of first voxels, and obtain a first covariance matrix associated with each of the plurality of first voxels based on the generated first probability variable. In addition, the data processing module (156) may generate a second probability variable for each of the plurality of second voxels through probability distribution fitting using a second set of representative values ​​associated with each of the plurality of second voxels, and obtain a second covariance matrix associated with each of the plurality of second voxels based on the generated second probability variable. Thereafter, based on the eigenvectors of the first covariance matrix and the eigenvectors of the second covariance matrix, a similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data may be calculated.

[0084] According to one embodiment, the operation of calculating the similarity between point cloud data based on the eigenvector of the covariance matrix by the probability variable for each voxel can be performed by the following mathematical expression 5.

[0085]

[0086]

[0087] Here, N represents the number of voxels, and w j real,i represents the j-th eigenvector derived from the first set of representative values ​​associated with the ith voxel of the actual point cloud data, and w j virt,ican represent the jth eigenvector calculated from the second set of representative values ​​associated with the ith voxel of the virtual point cloud data. M5 calculated through mathematical expression 5 can have a value from 0 to 100, and the closer it is to 100, the greater the similarity between the point cloud data can be determined. For example, M5 can represent the average value of the cosine similarity between the eigenvectors of the first covariance matrix calculated based on the first set of representative values ​​associated with the first voxel (or actual voxel) and the eigenvectors of the second covariance matrix calculated based on the second set of representative values ​​associated with the second voxel (or virtual voxel).

[0088] According to one embodiment, the data processing module (156) can identify a first voxel (or an actual voxel) in a three-dimensional space represented by a plurality of actual point cloud data, and calculate a mean value and a covariance matrix associated with the first voxel for each of the plurality of actual point cloud data to determine a first set of representative values. In addition, the data processing module (156) can identify a second voxel (or a virtual voxel) corresponding to the first voxel (or an actual voxel) in a three-dimensional space represented by the plurality of virtual point cloud data, and calculate a mean value and a covariance matrix associated with the second voxel for each of the plurality of virtual point cloud data to determine a second set of representative values. Thereafter, the data processing module (156) can calculate a similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on the first set of representative values ​​associated with the first voxel and the second set of representative values ​​associated with the second voxel.

[0089] The data analysis module (158) can determine the similarity between the first sensor (102a) (or the real sensor) and the second sensor (104a) (or the virtual sensor) based on a plurality of actual point cloud data and a plurality of virtual point cloud data. According to one embodiment, the data analysis module (158) can determine the similarity between the first sensor (102a) and the second sensor (104a) based on the similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data. For example, the data analysis module (158) can determine that the greater the similarity between the first sensor (102a) and the second sensor (104a), the greater the similarity between the first sensor (102a) and the second sensor (104a).

[0090] In the above description, as a method for calculating the similarity between point cloud data, a method based on the number of representative value sets associated with each voxel (e.g., using M1 calculated through Equation 1), a method based on distance values ​​between sensors and voxels calculated using representative values ​​of the sets associated with each voxel (e.g., using M2 calculated through Equation 2), a method based on a covariance matrix by a random variable for each voxel (e.g., using M3 calculated through Equation 3), a method based on the mean value and covariance matrix by a random variable for each voxel (e.g., using M4 calculated through Equation 4), and a method based on the eigenvector of the covariance matrix by a random variable for each voxel (e.g., using M5 calculated through Equation 5) have been described. However, the above-described methods may be used individually or two or more methods may be combined and used. For example, the data analysis module (158) can determine the degree of similarity between the first sensor (102a) and the second sensor (104a) based on at least one of the result values ​​M1 to M5 calculated using at least one of Equations 1 to 5. As another example, the data analysis module (158) can determine the degree of similarity between the first sensor (102a) and the second sensor (104a) based on a value obtained by applying a set weight to each of at least two of the result values ​​M1 to M5 calculated using at least two of Equations 1 to 5.

[0091] FIG. 2 is a schematic diagram illustrating a configuration in which an information processing system (230) is connected to a plurality of user terminals (210_1, 210_2, 210_3) so as to be able to communicate with each other, in relation to data processing according to one embodiment of the present disclosure. The information processing system (230) may include system(s) capable of providing data processing services (e.g., processing services of point cloud data). In one embodiment, the information processing system (230) may include one or more server devices and / or databases capable of storing, providing, and executing computer-executable programs (e.g., downloadable applications) and data related to the data processing services, or one or more distributed computing devices and / or distributed databases based on cloud computing services. For example, the information processing system (230) may include separate systems (e.g., servers) for data processing services.

[0092] Data processing services, etc. provided by the information processing system (230) can be provided to users through data processing applications, web browser applications, etc. installed on each of a plurality of user terminals (210_1, 210_2, 210_3).

[0093] A plurality of user terminals (210_1, 210_2, 210_3) can communicate with an information processing system (230) via a network (220). The network (220) can be configured to enable communication between the plurality of user terminals (210_1, 210_2, 210_3) and the information processing system (230). Depending on the installation environment, the network (220) can be configured as a wired network such as Ethernet, a wired home network (Power Line Communication), a telephone line communication device, and RS-serial communication, a wireless network such as a mobile communication network, WLAN (Wireless LAN), Wi-Fi, Bluetooth, and ZigBee, or a combination thereof. The communication method is not limited, and may include not only a communication method utilizing a communication network (e.g., a mobile communication network, wired Internet, wireless Internet, broadcasting network, satellite network, etc.) that the network (220) may include, but also short-range wireless communication between user terminals (210_1, 210_2, 210_3).

[0094] For example, multiple user terminals (210_1, 210_2, 210_3) can transmit data processing requests and commands related to user requests for data processing to an information processing system (230) via a network (220), and the information processing system (230) can receive them.

[0095] In FIG. 2, a mobile phone terminal (210_1), a tablet terminal (210_2), and a PC terminal (210_3) are illustrated as examples of user terminals, but are not limited thereto, and the user terminals (210_1, 210_2, 210_3) may be any computing device capable of wired and / or wireless communication and capable of installing and executing data processing applications, etc. For example, the user terminals may include smartphones, mobile phones, navigation devices, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, game consoles, wearable devices, IoT (Internet of Things) devices, VR (virtual reality) devices, AR (augmented reality) devices, etc. In addition, although FIG. 2 illustrates three user terminals (210_1, 210_2, 210_3) communicating with the information processing system (230) via the network (220), this is not limited thereto, and a different number of user terminals may be configured to communicate with the information processing system (230) via the network (220).

[0096] FIG. 3 is a block diagram illustrating the internal configuration of a user terminal (210) and an information processing system (230) according to one embodiment of the present disclosure. The user terminal (210) may refer to any computing device capable of executing a data processing application and capable of wired / wireless communication, and may include, for example, a mobile phone terminal (210_1), a tablet terminal (210_2), a PC terminal (210_3) of FIG. 2 . As illustrated, the user terminal (210) may include a memory (312), a processor (314), a communication module (316), and an input / output interface (318). Similarly, the information processing system (230) may include a memory (332), a processor (334), a communication module (336), and an input / output interface (338). As illustrated in FIG. 3, the user terminal (210) and the information processing system (230) may be configured to communicate information and / or data via a network (220) using respective communication modules (316, 336). In addition, the input / output device (320) may be configured to input information and / or data to the user terminal (210) or output information and / or data generated from the user terminal (210) via the input / output interface (318).

[0097] The memory (312, 332) may include any non-transitory computer-readable recording medium. According to one embodiment, the memory (312, 332) may include a permanent mass storage device such as a read-only memory (ROM), a disk drive, a solid state drive (SSD), a flash memory, etc. As another example, a permanent mass storage device such as a ROM, an SSD, a flash memory, a disk drive, etc. may be included in the user terminal (210) or the information processing system (230) as a separate permanent storage device distinct from the memory. In addition, the memory (312, 332) may store an operating system and at least one program code (e.g., code for an application associated with a data processing service, etc.).

[0098] These software components may be loaded from a computer-readable recording medium separate from the memory (312, 332). This separate computer-readable recording medium may include a recording medium directly connectable to the user terminal (210) and the information processing system (230), and may include, for example, a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. As another example, the software components may be loaded into the memory (312, 332) through a communication module (316, 336) other than a computer-readable recording medium. For example, at least one program may be loaded into the memory (312, 332) based on a computer program (e.g., an application associated with a data processing service, etc.) that is installed by files provided by developers or a file distribution system that distributes installation files of applications through a network (220).

[0099] The processor (314, 334) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (314, 334) by a memory (312, 332) or a communication module (316, 336). For example, the processor (314, 334) may be configured to execute instructions received according to program code stored in a storage device such as the memory (312, 332).

[0100] The communication module (316, 336) may provide a configuration or function for the user terminal (210) and the information processing system (230) to communicate with each other via the network (220), and may provide a configuration or function for the user terminal (210) and / or the information processing system (230) to communicate with another user terminal or another system (e.g., a separate cloud system, etc.). For example, a request or data (e.g., a data processing request or data, etc.) generated by the processor (314) of the user terminal (210) according to a program code stored in a recording device such as a memory (312) may be transmitted to the information processing system (230) via the network (220) under the control of the communication module (316). Conversely, a control signal or command provided under the control of the processor (334) of the information processing system (230) can be received by the user terminal (210) through the communication module (316) of the user terminal (210) via the communication module (336) and the network (220).

[0101] The input / output interface (318) may be a means for interfacing with an input / output device (320). As an example, the input device may include a device such as a camera, a keyboard, a microphone, a mouse, etc., including an audio sensor and / or an image sensor, and the output device may include a device such as a display, a speaker, a haptic feedback device, etc. As another example, the input / output interface (318) may be a means for interfacing with a device that has a configuration or function integrated into one for performing input and output, such as a touch screen. In FIG. 3, the input / output device (320) is illustrated as not being included in the user terminal (210), but is not limited thereto and may be configured as a single device with the user terminal (210). In addition, the input / output interface (338) of the information processing system (230) may be a means for interfacing with a device (not shown) for input or output that is connected to the information processing system (230) or that the information processing system (230) may include. In FIG. 3, the input / output interface (318, 338) is illustrated as an element configured separately from the processor (314, 334), but is not limited thereto, and the input / output interface (318, 338) may be configured to be included in the processor (314, 334).

[0102] The user terminal (210) and the information processing system (230) may include more components than those shown in FIG. 3. However, it is not necessary to explicitly illustrate most of the conventional technical components. In one embodiment, the user terminal (210) may be implemented to include at least some of the input / output devices (320) described above. In addition, the user terminal (210) may further include other components such as a transceiver, a Global Positioning System (GPS) module, a camera, various sensors, a database, etc. For example, if the user terminal (210) is a smartphone, it may include components that a smartphone generally includes, and for example, various components such as an acceleration sensor, a gyro sensor, a microphone module, a camera module, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration may be implemented to be further included in the user terminal (210).

[0103] According to one embodiment, the processor (314) of the user terminal (210) may be configured to operate a data processing application or a web browser application that provides a data processing service. At this time, program code associated with the application may be loaded into the memory (312) of the user terminal (210). While the application is operating, the processor (314) of the user terminal (210) may receive information and / or data provided from the input / output device (320) through the input / output interface (318) or may receive information and / or data from the information processing system (230) through the communication module (316), and may process the received information and / or data and store it in the memory (312). In addition, such information and / or data may be provided to the information processing system (230) through the communication module (316).

[0104] While the data processing application is running, the processor (314) may receive voice data, text, images, videos, etc. input or selected through input devices such as a camera, microphone, including a touch screen, keyboard, audio sensor, and / or image sensor connected to the input / output interface (318), and may store the received voice data, text, images, and / or videos in the memory (312) or provide them to the information processing system (230) through the communication module (316) and the network (220). In one embodiment, the processor (314) may receive user input input through the input device, and provide data / requests corresponding to the received user input to the information processing system (230) through the network (220) and the communication module (316).

[0105] The processor (314) of the user terminal (210) can output information and / or data by transmitting the information and / or data to an input / output device (320) through an input / output interface (318). For example, the processor (314) of the user terminal (210) can output the processed information and / or data through an output device (320), such as a display output capable device (e.g., a touch screen, a display, etc.) or a voice output capable device (e.g., a speaker).

[0106] The processor (334) of the information processing system (230) may be configured to manage, process, and / or store information and / or data received from multiple user terminals (210) and / or multiple external systems. Information and / or data processed by the processor (334) may be provided to the user terminal (210) via a communication module (336) and a network (220).

[0107] FIG. 4 is a diagram for explaining a virtual space (420) that simulates an actual space (410) of an outdoor environment according to one embodiment of the present disclosure. An electronic device (e.g., the electronic device (100) of FIG. 1) for processing point cloud data according to one embodiment of the present disclosure can determine the degree of mimicry between the actual sensor and the virtual sensor based on a plurality of actual point cloud data acquired by sensing the actual space (410) a specified number of times using an actual sensor (e.g., the first sensor (102a) of FIG. 1) and a plurality of virtual point cloud data acquired by sensing the virtual space (420) a specified number of times using a virtual sensor (e.g., the second sensor (104a) of FIG. 1). At this time, if the actual sensor is a sensor (e.g., a lidar sensor) that is sensitive to noise, such as rapid temperature changes or vibrations, the virtual sensor that simulates the actual sensor should also be designed by taking noise factors into consideration so that it can be determined that the degree of mimicry is high. Accordingly, in order to measure the degree of mimicry for the noise elements of such sensors, the electronic device according to one embodiment of the present disclosure can use multiple cloud data acquired by sensing the same space multiple times.

[0108] In FIG. 4, a real space (410) and a virtual space (420) for acquiring point cloud data will be described. The real space (410) may refer to a space where people can actually live and / or a space that actually exists in reality. However, for the convenience of experiments, the real space (410) may include a space actually created for autonomous driving experiments, etc. For example, the real space (410) may include a portion of an autonomous driving experiment city (e.g., K-City). The virtual space (420) may refer to a space that simulates the real space (410). However, the virtual space (420) may represent a space within a simulation environment created using a program (or software), rather than a space where people can actually live and / or a space that actually exists in reality. For example, the virtual space (420) may represent a space within a simulator that is a digital twin of the real space (410).

[0109] According to an embodiment of the present disclosure, an electronic device may efficiently use selected data rather than all data about a space to evaluate the degree of similarity between a real sensor and a virtual sensor. For example, a real object (e.g., a board (412) and / or a rubber cone (414)) may be placed in a real space (410), and a virtual object (e.g., a board (422) and / or a rubber cone (424)) corresponding to the real object (e.g., a digital twin of the real object) may be placed in a virtual space (420). In this case, the electronic device may select data in an area where the real object is placed from the real point cloud data, and may also select data in an area where the virtual object is placed from the virtual point cloud data, and process them. Accordingly, when processing point cloud data, the data processing speed may be improved and resource consumption may be reduced.

[0110] FIG. 5 is a diagram for explaining a virtual space (520) that simulates an actual space (510) of an indoor environment according to one embodiment of the present disclosure. An electronic device (e.g., the electronic device (100) of FIG. 1) that processes point cloud data according to one embodiment of the present disclosure can use point cloud data acquired in various environments to more accurately measure the degree of mimicry between an actual sensor (e.g., the first sensor (102a) of FIG. 1) and a virtual sensor (e.g., the second sensor (104a) of FIG. 1). For example, the electronic device can measure the degree of mimicry between an actual sensor and a virtual sensor by using not only point cloud data acquired in an actual space of an outdoor environment (e.g., the actual space (410) of FIG. 4) and a virtual space simulated therein (e.g., the virtual space (420) of FIG. 4) as shown in FIG. 4), but also point cloud data acquired in an actual space (510) of an indoor environment and a virtual space (520) simulated therein as shown in FIG. 5. In addition, in order to use only selected data in an indoor space, a real object (e.g., a board (512) and / or a rubber cone (514)) may be placed in the real space (510), and a virtual object (e.g., a board (522) and / or a rubber cone (524)) corresponding to the real object may be placed in the virtual space (520).

[0111] FIG. 6 is a diagram for explaining point cloud data (610, 620) acquired through a sensor according to one embodiment of the present disclosure. An electronic device (e.g., the electronic device (100) of FIG. 1) for processing point cloud data according to one embodiment of the present disclosure can acquire real point cloud data (610) acquired by sensing a real space (e.g., the real space (410) of an outdoor environment of FIG. 4 or the real space (510) of an indoor environment of FIG. 5) using a real sensor (e.g., the first sensor (102a) of FIG. 1). In addition, the electronic device can acquire virtual point cloud data (620) acquired by sensing a virtual space (e.g., the virtual space (420) of an outdoor environment of FIG. 4 or the virtual space (520) of an indoor environment of FIG. 5) using a virtual sensor (e.g., the second sensor (104a) of FIG. 1). Here, point cloud data (610, 620) is a set of points expressed as coordinate information in a three-dimensional space, and each point can represent location information within the space.

[0112] Referring to FIG. 6, the actual point cloud data (610) may include points (612, 614) corresponding to actual objects such as buildings or structures placed in an actual space. For example, the actual point cloud data (610) may include at least one first actual point (612) corresponding to a first actual object (e.g., a board (412) of FIG. 4 or a board (512) of FIG. 5) and at least one second actual point (614) corresponding to a second actual object (e.g., a rubber cone (414) of FIG. 4 or a rubber cone (514) of FIG. 5). In addition, the virtual point cloud data (620) may include points (622, 624) corresponding to a virtual object placed in a virtual space and digitally twinned with an actual object. For example, the virtual point cloud data (620) may include at least one first virtual point (622) corresponding to a first virtual object (e.g., a board (422) of FIG. 4 or a board (522) of FIG. 5) and at least one second virtual point (624) corresponding to a second virtual object (e.g., a rubber cone (424) of FIG. 4 or a rubber cone (524) of FIG. 5).

[0113] In the illustrated example, a space including at least one first real point (612) corresponding to a first real object may be classified as a first real voxel, and at least one second real point (614) corresponding to a second real object may be classified as a second real voxel. In addition, a space including at least one first virtual point (622) corresponding to a first virtual object may be classified as a first virtual voxel, and a space including at least one second virtual point (624) corresponding to a second virtual object may be classified as a second virtual voxel.

[0114] FIG. 7 is a diagram for explaining a method for selecting data of a specified area from point cloud data according to an embodiment of the present disclosure. An electronic device (e.g., the electronic device (100) of FIG. 1) for processing point cloud data according to an embodiment of the present disclosure may select data of a portion of an area from acquired (or received) point cloud data. According to an embodiment, the electronic device may select data (710) corresponding to an area (700) including a real object (702, 704) (e.g., a board (412) and a rubber cone (414) of FIG. 4 or a board (512) and a rubber cone (514) of FIG. 5) placed in a real space (e.g., a real space (410) of an outdoor environment of FIG. 4 or a real space (510) of an indoor environment of FIG. 5)) from real point cloud data (e.g., a real point cloud data (610) of FIG. 6). In this case, the selected real point cloud data (710) may include points (712, 714) corresponding to real objects (702, 704) (e.g., points (612, 614) of FIG. 6). In addition, the electronic device may select data (720) corresponding to an area including a virtual object (e.g., board (422) and rubber cone (424) of FIG. 4 or board (522) and rubber cone (524) of FIG. 5) corresponding to the real objects (702, 704) from the virtual point cloud data (e.g., virtual point cloud data (620) of FIG. 6). In this case as well, the selected virtual point cloud data (720) may include points (722, 724) corresponding to the virtual objects (e.g., points (622, 624) of FIG. 6).

[0115] As described above, electronic devices can utilize selected data instead of all spatial data when acquiring point cloud data. For example, the electronic device can extract and process only data within the region of interest where the object of interest is located. Accordingly, the electronic device can improve data processing speed and reduce resource consumption when processing point cloud data.

[0116] FIG. 8 is a diagram illustrating a method for voxelizing point cloud data (810, 820) according to an embodiment of the present disclosure. An electronic device (e.g., the electronic device (100) of FIG. 1) for processing point cloud data according to an embodiment of the present disclosure can process point cloud data by voxelizing it. For example, the electronic device can divide a three-dimensional space represented by the point cloud data (810, 820) into a plurality of voxels. Here, a voxel can represent a minimum unit in a three-dimensional space and can have a shape of a cube that grids the three-dimensional space. When such voxels are used, the electronic device can simplify and efficiently perform calculations and processing on the data.

[0117] According to one embodiment, the electronic device may divide a three-dimensional space represented by actual point cloud data (810) into a plurality of voxels, and divide a three-dimensional space represented by virtual point cloud data (820) into a plurality of voxels. Thereafter, the electronic device may identify a plurality of first voxels and a plurality of second voxels associated with a space in which an object of interest is placed among the plurality of voxels. Here, since the virtual point cloud data (820) is acquired using a virtual sensor (e.g., the second sensor (104a) of FIG. 1) that simulates an actual sensor (e.g., the first sensor (102a) of FIG. 1), it may have a three-dimensional space corresponding to the actual point cloud data (810), and accordingly, the plurality of second voxels may also correspond to the plurality of first voxels.

[0118] FIG. 9 is a diagram for explaining a method for calculating similarity between point cloud data based on a set of representative values ​​associated with voxels according to one embodiment of the present disclosure. An electronic device (e.g., the electronic device (100) of FIG. 1 ) for processing point cloud data according to one embodiment of the present disclosure can identify voxels (912, 922, 932) in a three-dimensional space represented by point cloud data (910, 920, 930), and calculate representative values ​​(916, 926, 936) associated with the voxels (912, 922, 932) identified for the point cloud data (910, 920, 930) to determine a set of representative values ​​(900). Here, the representative value set (900) may represent a set of representative values ​​(916, 926, 936) associated with the identified voxels (912, 922, 932), and the representative values ​​(916, 926, 936) associated with the identified voxels (912, 922, 932) may be obtained for each point cloud data (910, 920, 930).

[0119] According to one embodiment, the electronic device may identify a point set (914, 924, 934) associated with each of a plurality of voxels associated with an object of interest among points included in point cloud data (910, 920, 930), calculate an average coordinate of the identified point set (914, 924, 934), and determine the average coordinate of the calculated point set (914, 924, 934) as a representative value (916, 926, 936) associated with a voxel (912, 922, 932) of the point cloud data (910, 920, 930). Here, the point set (914, 924, 934) may represent a set of points included in the voxel (912, 922, 932).

[0120] The operation of determining the representative values ​​(916, 926, 936) associated with the above-described voxels (912, 922, 932) can be performed as many times as the number of point cloud data (910, 920, 930). For example, if a plurality of point cloud data is composed of three frames (e.g., a first frame (910) of point cloud data, a second frame (920) of point cloud data, and a third frame (930) of point cloud data), the electronic device may determine a first representative value (916) associated with each of the plurality of voxels (e.g., voxel (912)) for the first frame (910) of point cloud data, determine a second representative value (926) associated with each of the plurality of voxels (e.g., voxel (922)) for the second frame (920) of point cloud data, and determine a third representative value (936) associated with each of the plurality of voxels (e.g., voxel (932)) for the third frame (930) of point cloud data. Accordingly, each of the plurality of voxels may have a representative value less than or equal to the number of frames (e.g., the first representative value (916), the second representative value (926), and the third representative value (936)).

[0121] According to one embodiment, the electronic device may calculate a similarity between a plurality of real point cloud data and a plurality of virtual point cloud data, and determine a similarity between a real sensor (e.g., sensor (102a) of FIG. 1) and a virtual sensor (e.g., sensor (104a) of FIG. 1) based on the calculated similarity. According to one embodiment, the electronic device may calculate a similarity between the plurality of real point cloud data and the plurality of virtual point cloud data based on a first set of representative values ​​(e.g., set of representative values ​​(900)) associated with real voxels (e.g., voxels (912, 922, 932)) and a second set of representative values ​​associated with virtual voxels. Here, the second set of representative values ​​associated with the virtual voxels may be determined by a method identical to or similar to a method of determining the first set of representative values ​​associated with the real voxels.

[0122] According to one embodiment, the electronic device may calculate a similarity between a plurality of actual point cloud data and a plurality of virtual point cloud data based on a number of representative values ​​of a first set associated with an actual voxel and a number of representative values ​​of a second set associated with a virtual voxel. Here, the operation of calculating the similarity between the point cloud data based on the number of representative values ​​of a set associated with a voxel may be performed by the mathematical expression 1 described above.

[0123] According to one embodiment, the electronic device may calculate a first set of distance values ​​of a real voxel from a real sensor using a first set of representative values ​​associated with the real voxel, calculate a second set of distance values ​​of a virtual voxel from a virtual sensor using a second set of representative values ​​associated with the virtual voxel, and calculate a similarity between a plurality of real point cloud data and a plurality of virtual point cloud data based on the first set of distance values ​​and the second set of distance values. Here, the operation of calculating the similarity between the point cloud data based on the distance values ​​between the sensor and the voxel calculated using the set of representative values ​​associated with the voxel may be performed by the above-described mathematical expression 2.

[0124] According to one embodiment, the electronic device may generate a first probability variable for an actual voxel through probability distribution fitting using a first set of representative values ​​associated with the actual voxel, obtain a first covariance matrix associated with the actual voxel based on the generated first probability variable, generate a second probability variable for the virtual voxel through probability distribution fitting using a second set of representative values ​​associated with the virtual voxel, obtain a second covariance matrix associated with the virtual voxel based on the generated second probability variable, and calculate a similarity between a plurality of actual point cloud data and a plurality of virtual point cloud data based on the first covariance matrix and the second covariance matrix. Here, the operation of calculating the similarity between the point cloud data based on the covariance matrix by the probability variable for the voxel may be performed by the above-described mathematical expression 3.

[0125] According to one embodiment, the electronic device may generate a first probability variable for an actual voxel through probability distribution fitting using a first set of representative values ​​associated with the actual voxel, obtain a first mean value and a first covariance matrix associated with the actual voxel based on the generated first probability variable, generate a second probability variable for the virtual voxel through probability distribution fitting using a second set of representative values ​​associated with the virtual voxel, obtain a second mean value and a second covariance matrix associated with the virtual voxel based on the generated second probability variable, and calculate a similarity between a plurality of actual point cloud data and a plurality of virtual point cloud data based on the first mean value, the second mean value, the first covariance matrix, and the second covariance matrix. Here, the operation of calculating the similarity between the point cloud data based on the mean value and the covariance matrix by the probability variable for the voxel may be performed by the above-described mathematical expression 4.

[0126] According to one embodiment, the electronic device may generate a first probability variable for an actual voxel through probability distribution fitting using a first set of representative values ​​associated with the actual voxel, obtain a first covariance matrix associated with the actual voxel based on the generated first probability variable, generate a second probability variable for the virtual voxel through probability distribution fitting using a second set of representative values ​​associated with the virtual voxel, obtain a second covariance matrix associated with the virtual voxel based on the generated second probability variable, and calculate a similarity between a plurality of actual point cloud data and a plurality of virtual point cloud data based on an eigenvector of the first covariance matrix and an eigenvector of the second covariance matrix. Here, the operation of calculating the similarity between the point cloud data based on the eigenvector of the covariance matrix by the probability variable for the voxel may be performed by the above-described mathematical expression 5.

[0127] FIG. 10 is a diagram for explaining a method for processing point cloud data according to one embodiment of the present disclosure. Referring to FIG. 10, a processor (e.g., the processor (110) of FIG. 1) of an electronic device (e.g., the electronic device (100) of FIG. 1) for processing point cloud data may receive a plurality of actual point cloud data in step S1010. According to one embodiment, the processor may receive a plurality of actual point cloud data acquired by sensing an actual space using an actual sensor (e.g., the first sensor (102a) of FIG. 1) from an external electronic device (e.g., the first external electronic device (102) of FIG. 1) through a communication circuit (130) of FIG. 1). According to another embodiment, the processor may also receive a plurality of actual point cloud data stored in a memory (e.g., the memory (150) of FIG. 1).

[0128] In step S1020, the processor may receive a plurality of virtual point cloud data. According to one embodiment, the processor may receive a plurality of virtual point cloud data acquired by sensing a virtual space using a virtual sensor (e.g., the second sensor (104a) of FIG. 1) from an external electronic device (e.g., the second external electronic device (104) of FIG. 1) via a communication circuit. According to another embodiment, the processor may also receive a plurality of virtual point cloud data stored in the memory from a memory.

[0129] In step S1030, the processor may determine a similarity between the actual sensor and the virtual sensor based on a plurality of actual point cloud data and a plurality of virtual point cloud data. According to one embodiment, the processor may determine a similarity between the actual sensor and the virtual sensor based on the similarity of the plurality of actual point cloud data and the plurality of virtual point cloud data.

[0130] According to one embodiment, the processor may identify a first voxel in a three-dimensional space represented by a plurality of actual point cloud data, identify a second voxel corresponding to the first voxel in the three-dimensional space represented by the plurality of virtual point cloud data, calculate a representative value associated with the first voxel for each of the plurality of actual point cloud data to determine a first set of representative values, calculate a representative value associated with the second voxel for each of the plurality of virtual point cloud data to determine a second set of representative values, and calculate a similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on the first set of representative values ​​associated with the first voxel and the second set of representative values ​​associated with the second voxel.

[0131] According to one embodiment, the processor may identify a first voxel in a three-dimensional space represented by a plurality of actual point cloud data, identify a second voxel corresponding to the first voxel in the three-dimensional space represented by the plurality of virtual point cloud data, determine a first set of representative values ​​by calculating a mean value and a covariance matrix associated with the first voxel for each of the plurality of actual point cloud data, determine a second set of representative values ​​by calculating a mean value and a covariance matrix associated with the second voxel for each of the plurality of virtual point cloud data, and calculate a similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on the first set of representative values ​​associated with the first voxel and the second set of representative values ​​associated with the second voxel.

[0132] According to one embodiment, the processor can select a first voxel associated with a location of a real object placed in real space, and can select a second voxel associated with a virtual object placed in virtual space and corresponding to the real object.

[0133] According to one embodiment, when a plurality of real point cloud data includes first real point cloud data and second real point cloud data, the processor may identify a first set of points associated with a first voxel among points included in the first real point cloud data, identify a second set of points associated with the first voxel among points included in the second real point cloud data, calculate an average coordinate of the identified first set of points, calculate an average coordinate of the identified second set of points, determine the average coordinate of the calculated first set of points as a first representative value associated with the first voxel of the first real point cloud data, and determine the average coordinate of the calculated second set of points as a second representative value associated with the first voxel of the second real point cloud data.

[0134] According to one embodiment, the processor may calculate a similarity between a plurality of actual point cloud data and a plurality of virtual point cloud data based on a number of representative values ​​in a first set associated with a first voxel and a number of representative values ​​in a second set associated with a second voxel. Here, the operation of calculating the similarity between the point cloud data based on the number of representative values ​​in a set associated with a voxel may be performed by the mathematical expression 1 described above.

[0135] According to one embodiment, the processor may calculate distances between each of a first set of representative values ​​associated with a first voxel and an actual sensor to determine a first set of distance values, calculate distances between each of a second set of representative values ​​associated with a second voxel and a virtual sensor to determine a second set of distance values, and calculate a similarity between a plurality of actual point cloud data and a plurality of virtual point cloud data based on the first set of distance values ​​and the second set of distance values. Here, the operation of calculating the similarity between the point cloud data based on the distance values ​​between the sensors and the voxels calculated using the representative values ​​of the sets associated with the voxels may be performed by the above-described mathematical expression 2.

[0136] According to one embodiment, the processor may generate a first probability variable for a first voxel through probability distribution fitting using a first set of representative values ​​associated with a plurality of actual point cloud data, calculate a first covariance matrix associated with the first voxel based on the first probability variable, generate a second probability variable for a second voxel through probability distribution fitting using a second set of representative values ​​associated with a plurality of virtual point cloud data, calculate a second covariance matrix associated with the second voxel based on the second probability variable, and calculate a similarity between the plurality of actual point cloud data and the plurality of virtual point cloud data based on the first covariance matrix and the second covariance matrix. Here, the operation of calculating the similarity between the point cloud data based on the covariance matrix by the probability variable for the voxel may be performed by the above-described mathematical expression 3.

[0137] According to one embodiment, the processor may calculate a first mean associated with a first voxel based on a first probability variable, calculate a second mean associated with a second voxel based on a second probability variable, and calculate a similarity between a plurality of actual point cloud data and a plurality of virtual point cloud data based on the first mean, the second mean, the first covariance matrix, and the second covariance matrix. Here, the operation of calculating the similarity between the point cloud data based on the mean and the covariance matrix by the probability variable for the voxel may be performed by the above-described mathematical expression 4.

[0138] According to one embodiment, the processor may calculate a similarity between a plurality of actual point cloud data and a plurality of virtual point cloud data based on the eigenvectors of the first covariance matrix and the eigenvectors of the second covariance matrix. Here, the operation of calculating the similarity between the point cloud data based on the eigenvectors of the covariance matrix by the probability variable for the voxel may be performed by the above-described mathematical expression 5.

[0139] The above flowchart and description are merely examples, and some embodiments may implement the system differently. For example, in some embodiments, the order of each step may be changed, some steps may be repeated, some steps may be omitted, or some steps may be added.

[0140] The above-described method may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program instructions, including ROM, RAM, and flash memory. In addition, examples of other media may include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0141] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will appreciate that the various exemplary logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software will depend on the particular application and the design requirements imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0142] In a hardware implementation, the processing units used to perform the techniques may be implemented within one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, a computer, or a combination thereof.

[0143] Accordingly, the various exemplary logical blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0144] In a firmware and / or software implementation, the techniques may be implemented as instructions stored on a computer-readable medium, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, a compact disc (CD), a magnetic or marking data storage device, etc. The instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described herein.

[0145] When implemented in software, the techniques described above may be stored on or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media, including any medium that facilitates transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium.

[0146] For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of media. Disk and disc, as used herein, includes compact discs, laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, whereas discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0147] A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. Alternatively, the processor and the storage medium may reside as discrete components in the user terminal.

[0148] While the embodiments described above have been described as utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or distributed computing environment. Furthermore, aspects of the present disclosure may be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include personal computers, network servers, and portable devices.

[0149] While the present disclosure has been described in connection with certain embodiments herein, various modifications and variations may be made without departing from the scope of the present disclosure, which would be apparent to those skilled in the art. Furthermore, such modifications and variations are intended to fall within the scope of the claims appended to this specification.

Claims

1. In electronic devices, processor; and Memory operatively connected to said processor Including, The above memory, when executed, causes the processor to: Receive multiple real point cloud data acquired by sensing a real space a specified number of times using a real sensor, Receive multiple virtual point cloud data acquired by sensing a virtual space simulating the above real space a specified number of times using a virtual sensor simulating the above real sensor, An electronic device storing instructions for determining a degree of similarity between the real sensor and the virtual sensor based on the plurality of real point cloud data and the plurality of virtual point cloud data.

2. In claim 1, Determining the above-mentioned copy is: Identifying a first voxel in a three-dimensional space represented by the above multiple real point cloud data; Identifying a second voxel corresponding to the first voxel in a three-dimensional space represented by the plurality of virtual point cloud data; Determining the first set of representative values ​​by calculating the representative value associated with the first voxel for each of the plurality of actual point cloud data, Determining a second set of representative values ​​by calculating a representative value associated with the second voxel for each of the plurality of virtual point cloud data, and Calculating a similarity between the plurality of real point cloud data and the plurality of virtual point cloud data based on a first set of representative values ​​associated with the first voxel and a second set of representative values ​​associated with the second voxel. An electronic device comprising:

3. In claim 2, Identifying the first voxel above is: Selecting the first voxel associated with the location of the real object placed in the real space Including, Identifying the second voxel above is: Selecting the second voxel associated with a virtual object positioned in the virtual space and corresponding to the real object. An electronic device comprising:

4. In claim 2, The above multiple real point cloud data includes first real point cloud data and second real point cloud data, Determining the representative value of the first set above is: Identifying a first set of points associated with the first voxel among the points included in the first real point cloud data; Identifying a second set of points associated with the first voxel among the points included in the second real point cloud data; Calculating the average coordinates of the first set of points identified above; Calculating the average coordinates of the second set of points identified above; determining the average coordinate of the first point set produced above as the first representative value associated with the first voxel of the first actual point cloud data, and Determining the average coordinate of the second point set produced above as the second representative value associated with the first voxel of the second actual point cloud data An electronic device comprising:

5. In claim 2, To calculate the above similarity, Calculating a similarity between the plurality of real point cloud data and the plurality of virtual point cloud data based on the number of representative values ​​of the first set associated with the first voxel and the number of representative values ​​of the second set associated with the second voxel. An electronic device comprising:

6. In claim 2, To calculate the above similarity, Determining the distance values ​​of the first set by calculating the distance between each of the first set of representative values ​​associated with the first voxel and the actual sensor, Determining the distance values ​​of the second set by calculating the distance between each of the second set of representative values ​​associated with the second voxel and the virtual sensor, and Calculating the similarity between the plurality of real point cloud data and the plurality of virtual point cloud data based on the distance values ​​of the first set and the distance values ​​of the second set An electronic device comprising:

7. In claim 2, To calculate the above similarity, Generating a first probability variable for the first voxel by fitting a probability distribution using the representative values ​​of the first set associated with the plurality of actual point cloud data; Based on the first probability variable, calculating a first covariance matrix associated with the first voxel; Generating a second probability variable for the second voxel by fitting a probability distribution using the representative values ​​of the second set associated with the plurality of virtual point cloud data; Based on the second random variable, calculating a second covariance matrix associated with the second voxel, and Calculating the similarity between the plurality of real point cloud data and the plurality of virtual point cloud data based on the first covariance matrix and the second covariance matrix. An electronic device comprising:

8. In claim 7, To calculate the above similarity, Based on the first probability variable, calculating a first mean value associated with the first voxel; Calculating a second mean value associated with the second voxel based on the second probability variable Including more, An electronic device wherein the above similarity is calculated further based on the first average value and the second average value.

9. In claim 1, Determining the above-mentioned copy is: Identifying a first voxel in a three-dimensional space represented by the above plurality of real point cloud data; Identifying a second voxel corresponding to the first voxel in a three-dimensional space represented by the plurality of virtual point cloud data; For each of the plurality of real point cloud data, calculating the mean value and covariance matrix associated with the first voxel to determine the representative value of the first set; For each of the plurality of virtual point cloud data, calculating the mean value and covariance matrix associated with the second voxel to determine the representative value of the second set, and Calculating a similarity between the plurality of real point cloud data and the plurality of virtual point cloud data based on a first set of representative values ​​associated with the first voxel and a second set of representative values ​​associated with the second voxel. An electronic device comprising:

10. In the method of processing point cloud data, A step of receiving multiple real point cloud data acquired by sensing a real space a specified number of times using a real sensor; A step of receiving a plurality of virtual point cloud data acquired by sensing a virtual space simulating the actual space a specified number of times using a virtual sensor simulating the actual sensor; and A step of determining a degree of mimicry between the actual sensor and the virtual sensor based on the plurality of actual point cloud data and the plurality of virtual point cloud data. A method comprising:

Images