Method, device and computer program for compression of ndt map data

Abstract

This invention provides a method, apparatus, and computer program for compressing Normal Distribution Transform (NDT) map data for autonomous driving systems. The method for compressing NDT map data for autonomous driving systems according to various embodiments of this invention is a method for compressing Normal Distribution Transform (NDT) map data generated by modeling a 3D point cloud of a specified region as a normal distribution set, executed by a computing device. The method includes the following steps: processing the average vector data contained in the NDT map data; processing the covariance matrix data contained in the NDT map data; and generating compressed NDT map data using the processed average vector data and the processed covariance matrix data.

Description

Technical Field

[0001] Various embodiments of the present invention relate to methods, apparatus, and computer programs for compressing Normal Distribution Transform (NDT) map data for autonomous driving systems. Background Technology

[0002] To facilitate vehicle users, various sensors and electronic devices are being provided (e.g., Advanced Driver Assistance Systems (ADAS)). In particular, the development of autonomous driving systems, which identify the surrounding environment without driver intervention and automatically drive to a given destination based on the identified environment, is actively underway.

[0003] Among them, autonomous vehicles refer to vehicles equipped with the following functions: autonomous driving system functions that can identify the surrounding environment without driver intervention and automatically drive to a given destination based on the identified surrounding environment.

[0004] In order to control the driving actions of autonomous vehicles without driver intervention, maps that autonomous vehicles can understand are required. As mentioned above, "road environment information that can be understood by computers is pre-stored in the form of a database" and is called a digital map. In particular, digital maps for autonomous driving contain more detailed information than simple navigation maps, and are therefore called "precision maps", "high-definition (HD) maps" and "highly automated driving (HAD) maps".

[0005] Previously, such digital maps and precision maps utilized 3D point cloud maps generated from 3D point clouds (e.g., point clouds collected by LiDAR, Radar, or depth cameras). However, 3D point cloud maps have a large data volume, making it difficult to use them for a wide range of areas. Summary of the Invention

[0006] To address the existing problems, the present invention aims to provide a method, apparatus, and computer program for compressing normally distributed transform map data for autonomous driving systems. This method processes the representation of normally distributed transform map data generated by modeling a 3D point cloud of a specified region as a normally distributed set to generate compressed normally distributed transform map data. This significantly reduces the constraints on the scalability of the autonomous driving system's driving area and improves the efficiency of autonomous driving, which is crucial for stable real-time operation.

[0007] The problems to be solved by this invention are not limited to those mentioned above, and those skilled in the art can clearly understand other problems not mentioned through the following description.

[0008] An embodiment of the present invention for compressing normally distributed transform map data for an autonomous driving system is a method for compressing normally distributed transform (NDT) map data generated by modeling a three-dimensional point cloud of a specified area as a normally distributed set, executed by a computing device. The method may include the following steps: processing the average vector data contained in the normally distributed transform map data; processing the covariance matrix data contained in the normally distributed transform map data; and generating compressed normally distributed transform map data using the processed average vector and the processed covariance matrix.

[0009] In various embodiments, the present invention may further include the following steps: generating multiple three-dimensional grid spaces by meshing the three-dimensional point clouds of a specified region; modeling each three-dimensional point cloud contained in the generated multiple three-dimensional grid spaces as multiple normal distributions; calculating the average vector of each of the multiple normal distributions corresponding to the generated multiple three-dimensional grid spaces, and generating average vector data using the calculated average vectors; calculating the covariance matrix of each of the multiple normal distributions corresponding to the generated multiple three-dimensional grid spaces, and generating covariance matrix data using the calculated covariance matrix; and generating normal distribution transformation map data for the specified region using the generated average vector data and the generated covariance matrix data.

[0010] In various embodiments, the generated average vector data includes a center vector and an offset vector with respect to the center positions of the generated plurality of three-dimensional mesh spaces. The steps of processing the average vector data may include the following steps: transforming the expression of the center vector; transforming the expression of the offset vector; and generating compressed average vector data using the coordinate values ​​of the transformed center vector and the transformed offset vector.

[0011] In various embodiments, the step of transforming the representation of the center vector may include the following steps: transforming the center vector into the form of a tree data structure, wherein the tree data structure contains one or more nodes, and the one or more nodes have a non-linear hierarchical structure.

[0012] In various embodiments, the step of transforming the expression of the offset vector may include the following steps: setting a first reference range based on the length of one edge of each of the generated plurality of three-dimensional mesh spaces; and transforming the components of the offset vectors of each of the generated plurality of three-dimensional mesh spaces into values ​​within the set first reference range.

[0013] In various embodiments, the generated covariance matrix data includes the standard deviation and rotation transformation angle of each of the multiple points contained in the generated multiple three-dimensional mesh spaces. The steps of processing the covariance matrix data may include the following steps: changing the expression of the standard deviation; changing the expression of the rotation transformation angle; and using the changed standard deviation and the changed rotation transformation angle to generate compressed covariance matrix data.

[0014] In various embodiments, the step of transforming the expression of the above standard deviation may include the following steps: setting a second reference range based on the length of the longest diagonal in the generated plurality of three-dimensional mesh spaces; and transforming the standard deviation of each of the generated plurality of three-dimensional mesh spaces into a value within the set second reference range.

[0015] In various embodiments, the step of changing the expression of the above-mentioned rotation transformation angle may include the following steps: setting the reference rotation ranges of the X-axis, Y-axis and Z-axis respectively; and transforming the above-mentioned rotation transformation angle into a value within the set reference rotation range.

[0016] In various embodiments, the present invention may further include the following steps: based on a preset matching performance, setting the compression degree of the average vector data and the covariance matrix data respectively; processing the average vector data and the covariance matrix data according to the set compression degree; and setting the compression degree of the covariance matrix data to be greater than or equal to the compression degree of the average vector data when the same matching performance is set.

[0017] In various embodiments, the present invention may further include the following steps: calculating the frequency of each object contained in the above-mentioned normal distribution transformed map data, and based on the calculated frequency, setting the compression degree of the average vector data and covariance matrix data of the regions corresponding to each of the above-mentioned objects on the above-mentioned normal distribution transformed map data.

[0018] In various embodiments, the following steps may also be included: when a first object and a second object located at adjacent positions among the multiple objects included in the above-mentioned normal distribution transformed map data have the same attributes, the first object and the second object are grouped into a group; the step of processing the average vector data may include the following steps: generating an average vector data by combining multiple average vector data of multiple objects included in the above-mentioned group, and processing the generated average vector data; the step of processing the covariance matrix data may include the following steps: generating a covariance matrix data by combining multiple covariance matrix data of multiple objects included in the above-mentioned group, and processing the generated covariance matrix data.

[0019] In various embodiments, the present invention may further include the following steps: grouping two or more objects with a pre-set positional relationship among the multiple objects contained in the above-mentioned normal distribution transformed map data; the step of processing the above-mentioned average vector data may include the following steps: generating an average vector data by combining multiple average vector data of multiple objects contained in the above-mentioned group; and processing the generated average vector data; the step of processing the above-mentioned covariance matrix data may include the following steps: generating a covariance matrix data by combining multiple covariance matrix data of multiple objects contained in the above-mentioned group; and processing the generated covariance matrix data.

[0020] In various embodiments, the present invention may include the following steps: extracting compressed normal distribution transformation map data by using the above-mentioned normal distribution transformation map data as input to a pre-learned artificial intelligence model, wherein the pre-learned artificial intelligence model includes an encoder and a decoder, and is a model that learns by using multiple compressed normal distribution transformation map data as learning data.

[0021] A computing device for implementing a method for compressing normally distributed transformed map data for an autonomous driving system, according to another embodiment of the present invention for solving the above-mentioned problems, includes: a processor; a network interface; a memory; and a computer program loaded into the memory and executed by the processor. The computer program may include instructions for: processing average vector data contained in normally distributed transformed map data generated by modeling a 3D point cloud of a specified region as a normally distributed set; processing covariance matrix data contained in the normally distributed transformed map data; and generating compressed normally distributed transformed map data using the processed average vector and the processed covariance matrix.

[0022] A computer program of another embodiment of the present invention for solving the above-mentioned problems can be stored in a computing device readable recording medium to be combined with a computing device and to perform a method for compressing normally distributed transformed map data including the following steps: processing average vector data contained in normally distributed transformed map data generated by modeling a three-dimensional point cloud of a specified region as a normally distributed set; processing the average vector data contained in the normally distributed transformed map data; processing the covariance matrix data contained in the normally distributed transformed map data; and generating compressed normally distributed transformed map data using the processed average vector and the processed covariance matrix.

[0023] Other specific aspects of the invention are included in the detailed description and accompanying drawings.

[0024] According to various embodiments of the present invention, the following advantages are achieved: by processing the representation of the normally distributed transformed map data generated by modeling the three-dimensional point cloud of a specified area as a normally distributed set to generate compressed normally distributed transformed map data, the constraints on the scalability of the driving area of ​​the autonomous driving system are greatly reduced, and the efficiency of autonomous driving, which is crucial for stable real-time operation, can be improved.

[0025] The effects of this invention are not limited to those mentioned above, and those skilled in the art can clearly understand other effects not mentioned from the following description. Attached Figure Description

[0026] Figure 1 A diagram illustrating an embodiment of the autonomous driving system of the present invention.

[0027] Figure 2 This is a hardware structure diagram of a computing device for implementing a method for compressing normally distributed transformed map data for an autonomous driving system, according to another embodiment of the present invention.

[0028] Figure 3 This is a flowchart of a method for compressing normally distributed transformed map data for an autonomous driving system, according to another embodiment of the present invention.

[0029] Figure 4 This is a flowchart illustrating a method for processing average vector data in various embodiments.

[0030] Figure 5 This is a flowchart illustrating a method for processing covariance matrix data in various embodiments.

[0031] Figure 6 The diagrams illustrating the covariance matrix data expressed by ellipsoids in various embodiments are shown as examples.

[0032] Explanation of reference numerals in the attached figures

[0033] 100: Computing devices

[0034] 200: User Terminal

[0035] 300: External Server

[0036] 400: Network Detailed Implementation

[0037] The advantages, features, and methods for achieving these advantages and features of the present invention will become clear from the detailed embodiments and accompanying drawings described below. However, the present invention is not limited to the embodiments disclosed below, but can be embodied in many different forms. These embodiments are provided only to make the disclosure of the present invention more complete and to fully inform those skilled in the art of the scope of the present invention. The present invention is defined only by the scope of the claims.

[0038] The terminology used in this specification is for illustrative purposes and does not limit the invention. In this specification, the singular includes the plural unless specifically stated in the context. The terms "comprises" and / or "comprising" as used in the specification do not exclude the presence or addition of more than one other component besides those mentioned. Throughout the specification, the same reference numerals refer to the same components, and "and / or" includes each mentioned component and all combinations thereof. "First," "second," etc., are used to describe various components, but these components are not limited to these terms. These terms are used only to distinguish one component from another. Therefore, the first component mentioned below may also refer to the second component within the technical concept of the invention.

[0039] Unless otherwise defined, all terms used in this specification (including technical and scientific terms) have the meanings commonly understood by one of ordinary skill in the art to which this invention pertains. Furthermore, unless explicitly defined otherwise, terms as defined by commonly used dictionaries should not be idealized or over-interpreted.

[0040] The terms "part" or "module" as used in this specification refer to software, such as a field-programmable gate array (FPGA) or application-specific integrated circuit (ASIC), and the "part" or "module" performs any function. However, a "part" or "module" is not limited to software or hardware. A "part" or "module" may be configured to reside on an addressable storage medium, or may be configured to reproduce one or more processors. Thus, as an example, a "part" or "module" includes components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided in components and "parts" or "modules" may be combined from fewer components and "parts" or "modules," or may be separated into additional components and "parts" or "modules."

[0041] As shown in the figure, spatial relative terms such as "below," "below," "lower," "above," and "upper" are used to simply describe the relationship between one component and another. Spatial relative terms should be understood to include not only the directions shown in the figure but also the different orientations of the components during use or operation. For example, when the components shown in the figure are flipped, a component described as "below" or "below" to another component may be located "above" to that component. Therefore, the illustrative term "below" can include both the down and up directions. Components can also be oriented in other directions, thus the spatial relative terms can be interpreted according to orientation.

[0042] In this specification, "computer" refers to all kinds of hardware devices including at least one processor, and according to embodiments, it can be understood to include the software architecture running on the corresponding hardware device. For example, "computer" can be understood to include smartphones, tablets, desktop computers, laptops, and user clients and applications running on each device, but is not limited thereto.

[0043] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0044] The steps described in this specification are illustrated in the case of execution by a computer, but the main body of each step is not limited thereto. According to the embodiments, at least a portion of each step may also be executed in different devices.

[0045] To overcome the problems inherent in traditional digital maps, normally transformed map data can be generated by modeling 3D point clouds as normally distributed sets based on normal distribution transformation. However, in situations where the driving area of ​​an autonomous driving system is very wide, the data size of the 3D point cloud representing it is also very large. Therefore, the size of the normally transformed map data generated using this data may also be significantly limited.

[0046] This affects the scalability of the autonomous driving system's driving area. As the driving area of ​​the autonomous driving system expands, the size of the normally distributed transform map data also increases, and the excessively large size of the normally distributed transform map data also limits the expansion of the autonomous driving system's driving area.

[0047] To address this problem, a method, apparatus, and computer program for compressing normally distributed transformed map data for autonomous driving systems are proposed: ensuring the scalability of the driving area and effectively compressing the normally distributed transformed map data for efficient use in real-time autonomous driving systems. The following refers to... Figures 1 to 6 Various embodiments of the present invention will be described, including methods, apparatus, and computer programs for compressing normally distributed transformed map data for autonomous driving systems.

[0048] Figure 1 A diagram illustrating an embodiment of the autonomous driving system of the present invention.

[0049] Reference Figure 1 An embodiment of the autonomous driving system of the present invention may include a computing device 100, a user terminal 200, an external server 300, and a network 400.

[0050] in, Figure 1 The autonomous driving system shown is according to one embodiment, and its components are not limited to... Figure 1 The embodiments shown can be added, modified, or deleted as needed.

[0051] In one embodiment, the computing device 100 can perform various operations for controlling the autonomous driving of the autonomous vehicle 10.

[0052] To this end, firstly, the computing device 100 can perform operations to detect the position and orientation of the autonomous vehicle 10 or to identify the surrounding environment of the autonomous vehicle 10. For example, the computing device 100 can collect sensor data from sensors (e.g., lidar sensors, laser sensors, camera sensors, etc.) configured inside the autonomous vehicle 10, and can use the collected sensor data to detect the position and orientation of the autonomous vehicle 10 or to identify the surrounding environment of the autonomous vehicle 10.

[0053] Furthermore, in order to control the autonomous driving of the autonomous vehicle 10, the computing device 100 generates and stores normal distribution transformation map data for a specified area, processes the normal distribution set contained in the normal distribution transformation map data to generate and store compressed normal distribution transformation map data, that is, generates normal distribution transformation map data of varying sizes. This results in normal distribution transformation map data that ensures the scalability of the driving area and can be effectively used in a real-time autonomous driving system. This will be explained in detail later.

[0054] In various embodiments, computing device 100 can be connected to user terminal 200 via network 400, and can provide user terminal 200 with various information related to autonomous driving, such as the detected position and posture of autonomous vehicle 10, the identified surrounding environment of autonomous vehicle 10, and the generated normal distribution transformation map data, by analyzing sensor data.

[0055] The user terminal 200 may be an infotainment system configured inside the autonomous vehicle 10, but is not limited to this. As a wireless communication device ensuring portability and mobility, it may be a portable terminal that can be carried by passengers inside the autonomous vehicle 10. For example, the user terminal 200 may include all types of handheld wireless communication devices such as navigators, Personal Communication Systems (PCS), Global System for Mobile Communications (GSM), Personal Digital Cellular (PDC), Personal Handyphone Systems (PHS), Personal Digital Assistants (PDAs), International Mobile Telecommunications (IMT)-2000, Code Division Multiple Access (CDMA)-2000, W-Code Division Multiple Access (W-CDMA), Wireless Broadband Internet (Wibro) terminals, smartphones, smartpads, and tablet PCs, but is not limited to these.

[0056] Furthermore, network 400 can refer to a connection structure capable of exchanging information between various nodes such as multiple terminals and servers. For example, network 400 may include a local area network (LAN), a wide area network (WAN), the Internet (WWW), wired / wireless data communication networks, telephone networks, wired / wireless television communication networks, etc.

[0057] Furthermore, the wireless data communication network may include, but is not limited to, third-generation mobile communication technology (3G), fourth-generation mobile communication technology (4G), fifth-generation mobile communication technology (5G), the 3rd Generation Partnership Project (3GPP), the 5th Generation Partnership Project (5GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, the Internet, Local Area Network (LAN), Wireless Local Area Network (WirelessLAN), Wide Area Network (WAN), Personal Area Network (PAN), Radio Frequency (RF), Bluetooth networks, Near-Field Communication (NFC) networks, satellite broadcasting networks, analog broadcasting networks, and Digital Multimedia Broadcasting (DMB) networks.

[0058] In one embodiment, an external server 300 can be connected to the computing device 100 via a network 400. It can store and manage various information and data (e.g., multiple point clouds) required by the computing device 100 to perform a compression method for normally distributed transform map data used in an autonomous driving system, or it can receive, store, and manage various information and data (e.g., compressed normally distributed transform map data) generated during the execution of the compression method for normally distributed transform map data used in an autonomous driving system. For example, the external server 300 can be a storage server separately configured outside the computing device 100, but is not limited thereto. Referring hereafter... Figure 2The hardware structure of the computing device 100 that performs a compression method for normal distribution transformation map data for an autonomous driving system will be described.

[0059] Figure 2 This is a hardware structure diagram of a computing device for implementing a method for compressing normally distributed transformed map data for an autonomous driving system, according to another embodiment of the present invention.

[0060] Reference Figure 2 In another embodiment of the present invention, a computing device 100 for executing a method for compressing normally distributed transformed map data for an autonomous driving system may include: one or more processors 110; a memory 120 for loading a computer program 151 running through the processor 110; a bus 130; a communication interface 140; and a storage device 150 for storing the computer program 151. Figure 2 Only components relevant to embodiments of the present invention are shown. Therefore, it will be apparent to those skilled in the art that, in addition to [other components], [other components] are not necessarily [specifically, specific components]. Figure 2 In addition to the components shown, other general-purpose components may also be included.

[0061] The processor 110 is used to control the overall operation of the various structures of the computing device 100. The processor 110 may be configured as a central processing unit (CPU), a microprocessor unit (MPU), a micro controller unit (MCU), a graphics processing unit (GPU), or any type of processor known in the art of this invention.

[0062] Furthermore, the processor 110 can run operations of at least one application or program for executing the methods of the embodiments of the present invention, and the computing device 100 can be configured with more than one processor.

[0063] In various embodiments, the processor 110 may further include random access memory (RAM; not shown) and read-only memory (ROM; not shown) for temporary and / or permanent storage of signals (or data) processed internally by the processor 110. Furthermore, the processor 110 can be implemented as a system-on-chip (SoC) comprising at least one of a graphics processing unit, random access memory, and read-only memory.

[0064] Memory 120 is used to store various data, commands, and / or information. To perform the methods / operations of various embodiments of the present invention, memory 120 may load a computer program 151 from storage device 150. If computer program 151 is loaded into memory 120, processor 110 executes one or more instructions constituting computer program 151, thereby performing the aforementioned methods / operations. Memory 120 may be implemented using volatile memory such as random access memory, but the scope of this disclosure is not limited thereto.

[0065] Bus 130 provides communication functionality between components of computing device 100. Bus 130 can be implemented as various types of buses, such as address bus, data bus, and control bus.

[0066] The communication interface 140 supports wired / wireless internet communication of the computing device 100. Furthermore, the communication interface 140 can also support various communication methods other than internet communication. Therefore, the communication interface 140 may include communication modules known in the art. In some embodiments, the communication interface 140 may be omitted.

[0067] Storage device 150 can non-temporarily store computer program 151. When the computing device 100 runs a compression process for normally distributed transformed map data for an autonomous driving system, storage device 150 can store various information required to provide the compression process for normally distributed transformed map data for an autonomous driving system.

[0068] The storage device 150 may be configured as a non-volatile memory such as a read-only memory, an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), flash memory, a hard disk, a removable hard disk, or any form of computer-readable recording medium known in the art to which this invention pertains.

[0069] Computer program 151 may include one or more instructions to cause processor 110 to perform the methods / operations of various embodiments of the present invention when loaded into memory 120. That is, processor 110 can perform the methods / operations of various embodiments of the present invention by executing one or more of the above instructions.

[0070] In one embodiment, computer program 151 may include one or more instructions for performing a method for compressing normally distributed transformed map data, comprising the steps of: processing average vector data contained in normally distributed transformed map data generated by modeling a 3D point cloud of a specified region as a normally distributed set; processing the average vector data contained in the normally distributed transformed map data; processing the covariance matrix data contained in the normally distributed transformed map data; and generating compressed normally distributed transformed map data using the processed average vectors and the processed covariance matrix.

[0071] The steps of the methods or algorithms described in conjunction with the embodiments of the present invention can be implemented directly in hardware, or by software modules running through hardware, or by a combination thereof. The software modules can reside in random access memory, read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, flash memory, hard disk, removable disk, CD-ROM, or any form of computer-readable recording medium known in the art to which this invention pertains.

[0072] To operate in conjunction with a computer as hardware, the components of this invention can be implemented by a program (or application program) and stored in a medium. The components of this invention can be run by software programming or software components; similarly, embodiments can be implemented by programming or scripting languages ​​such as C, C++, Java, assembler, etc., using various algorithms including combinations of data structures, procedures, routines, or other programming structures. Functionality can be implemented by algorithms running on more than one processor. Hereinafter, reference will be made to... Figures 3 to 6 The method for compressing normally distributed transformed map data for autonomous driving systems, performed by computing device 100, is explained.

[0073] Figure 3 This is a flowchart of a method for compressing normally distributed transformed map data for an autonomous driving system, according to another embodiment of the present invention.

[0074] Reference Figure 3 In step S110, the computing device 100 can generate normal distribution transformed map data for the specified area.

[0075] In various embodiments, computing device 100 can generate normally distributed transformed map data by modeling a three-dimensional point cloud of a specified area as a normally distributed set based on normal distribution transformation technology.

[0076] More specifically, firstly, the computing device 100 can collect a three-dimensional point cloud measured by sensors (e.g., lidar, laser, and cameras) of an autonomous vehicle 10 located within a specified area or configured to drive within the specified area. As the collected three-dimensional point cloud is meshed, multiple three-dimensional mesh spaces can be generated, that is, the three-dimensional point cloud can be divided into mesh space units of a specified size.

[0077] In this embodiment, multiple three-dimensional mesh spaces may have the same size cube shape (e.g., 10cm×10cm×10cm or 2cm×2cm×2cm, etc.), but are not limited to this. In various embodiments, multiple three-dimensional mesh spaces may have a cuboid shape, etc.

[0078] Then, the computing device 100 can model multiple three-dimensional grid spaces as a normally distributed set.

[0079] For example, firstly, the computing device 100 models each three-dimensional mesh space as a normal distribution, thereby generating multiple normal distributions corresponding to each three-dimensional mesh space.

[0080] Subsequently, the computing device 100 can calculate the average vector of multiple points contained in each normal distribution as the positional information of each normal distribution, and can generate average vector data using the calculated average vector. The average vector may include a center vector about the center position of each three-dimensional grid space and an offset vector indicating the degree of separation from the center position, but is not limited to this.

[0081] Subsequently, the computing device 100 calculates the covariance matrix of multiple points contained in each normal distribution as information about the shape and direction of each normal distribution. Covariance matrix data can be generated using the calculated covariance matrix, but is not limited to this.

[0082] Next, the computing device 100 can generate normal distribution transformed map data containing the generated average vector data and covariance matrix data according to the above process.

[0083] It is explained that the computing device 100 directly generates normal distribution transformation map data for a specified area by directly collecting sensor data about the specified area, but it is not limited to this. It can also be implemented by receiving the generated normal distribution transformation map data through other external systems and utilizing its shape.

[0084] In step S120, the computing device 100 can process the average vector data contained in the generated normal distribution transformed map data in step S110 to generate compressed average vector data.

[0085] In the case of the average vector, there are 3 values ​​(μ) x μy μ z To represent a three-dimensional vector, if the coordinates of the center position of each three-dimensional grid space are set to c... x c y and c z Then the average vector can be expressed by the following mathematical formula 1.

[0086] Mathematical formula 1: μ i =c i +d i

[0087] Where i refers to x, y, or z, and μ i This refers to the i-axis component value of the average vector, c i This refers to the i-axis component value of the center vector, d i It refers to the i-axis component value of the offset vector (a vector representing the degree to which it deviates from the center position of the three-dimensional mesh space).

[0088] Therefore, the computing device 100 can process the center vector and the offset vector separately to generate compressed average vector data.

[0089] In step S130, the computing device 100 can process the covariance matrix data contained in the normal distribution transformed map data generated in step S110 to generate compressed covariance matrix data.

[0090] Each covariance matrix data can be represented by a corresponding three-dimensional elliptic. This three-dimensional elliptic can be defined by its length along three axes and three rotational transformation angles, and can contain three length information and three rotational transformation angle information corresponding to the covariance matrix. The length information along the elliptic's axes can refer to the standard deviation, which will be described later. Therefore, the computing device 100 can process the standard deviation and rotational transformation angles of the three-dimensional elliptic corresponding to the covariance matrix data to generate the final compressed covariance matrix data.

[0091] Furthermore, it is shown that... Figure 3 The method for compressing normally distributed map data for autonomous driving systems, as shown, processes the covariance matrix data after processing the average vector data. However, this is only to distinguish between the processing of the average vector data and the processing of the covariance matrix data, and is not a limitation. The processing of the covariance matrix data can be performed first, or the processing of both the average vector data and the covariance matrix data can be performed simultaneously. See below for reference. Figures 4 to 6 This will explain in more detail the process of generating compressed average vector data and the process of generating compressed covariance matrix data.

[0092] Figure 4This is a flowchart illustrating a method for processing average vector data in various embodiments.

[0093] Reference Figure 4 In step S210, the computing device 100 can change the way the center vector contained in the average vector data is expressed.

[0094] In various embodiments, computing device 100 can transform the center vector into a tree-like data structure. The tree-like data structure contains one or more nodes, which may refer to one or more nodes having a non-linear hierarchical structure, but is not limited to this.

[0095] For example, computing device 100 can transform the center vector into an octree form. For instance, computing device 100 can use an octree with a depth of L to represent the center vector (x, y, z coordinate values).

[0096] Typically, in the environment of an autonomous driving system's driving area, objects are not uniformly distributed from underground to above ground. Instead, most of the underground or above-ground space is empty, while the above-ground portion has a dense distribution of terrain features such as buildings, roads, and trees. Therefore, when expressing the center vector representing the center position of each 3D mesh space, each component of the center vector is not expressed separately by data of a defined size (bytes) (e.g., real numbers). x c y and c z Instead of using tree-based data structures (e.g., octrees), it uses tree-based data structures (e.g., octrees), which has the advantage of being able to express data more efficiently.

[0097] There are various known techniques for transforming vectors into tree data structures, and one of these known techniques can be selectively applied. This specification does not describe or limit the specific method of transforming into a tree data structure.

[0098] In step S220, the computing device 100 can change the way the offset vector is expressed.

[0099] In various embodiments, the computing device 100 can set a first reference range based on the length of an edge of each three-dimensional mesh space, and can transform the offset vector of each three-dimensional mesh space into a value within the first reference range.

[0100] The average vector is calculated using multiple points contained in each three-dimensional mesh space, and the average vector cannot exceed the three-dimensional mesh space. Considering this, the computing device 100 can set a first reference range based on the length of the three-dimensional mesh space, and can transform the offset vector into a value within the first reference range set according to the length of the three-dimensional mesh space. For example, if each three-dimensional mesh space is a cube with a side length of l, then the following mathematical formula 2 holds.

[0101] Mathematical formula 2:

[0102] Where, d i It can refer to the i-axis component value of the offset vector (the degree to which it deviates from the center).

[0103] Therefore, the computing device 100 utilizes d for length l i Within a finite range, the components of the offset vectors in each three-dimensional mesh space are mapped to the range of mathematical formula 2. Therefore, it can be transformed into... to Values ​​within the range.

[0104] Furthermore, if each three-dimensional grid space is a cuboid, then the following mathematical formula 3 holds true.

[0105] Mathematical formula 3:

[0106] Where, m i It can refer to the length of the cuboid along the i-axis, d i This can refer to the i-axis component value of the offset vector. Therefore, the computing device 100 in the three-dimensional mesh space is of length m. i In the case of a cuboid, the same principle applies as with a cube, using d i With m i Within a defined range, the components of the offset vectors in each three-dimensional mesh space correspond to the range of mathematical formula 3. Therefore, it can be transformed into... to Values ​​within the range.

[0107] In the process of dividing the three-dimensional mesh space, the length l (or m) of one side of the mesh space has been obtained. i Therefore, only the proportion of the offset vector of length l (or mi) is specified, thereby specifying the offset vector, which is a value within the range of mathematical formula 2 or mathematical formula 3. Therefore, by utilizing the above proportion, which has a finite range of values ​​from -1 / 2 to 1 / 2, reliable compression can be performed using small amounts of data. For example, in the above method, from 1 byte (2 8 When expressing the proportion of the offset vector, d can be expressed within 1 / 256% of the error. i .

[0108] In step S230, the computing device 100 can generate compressed average vector data using the center position coordinates of the average vector data processed in step S210 and the offset vector processed in step S220.

[0109] Typically, a three-dimensional average vector is represented by three real numbers. If each real number is represented by a single-precision floating-point number, then each real number is represented by 4 bytes. In order to represent a three-dimensional average vector, a total of 12 bytes are required.

[0110] Conversely, when the center vector is represented by an octree of depth L as described above, the size of the data required to represent the center vector is only about 1 / 7 byte per 3D grid space in the best case, and only about L-ceil(log8n)+(1+1 / 7) bytes per 3D grid space in the worst case. Thus, it has the advantage of effectively compressing the center vector.

[0111] Furthermore, when expressing offset vectors in multiple three-dimensional mesh spaces, in order to express the correct real value for the offset vector, it is not necessary to use several bytes, but only 1 byte to express the value within the first reference range. Thus, it has the advantage of effectively compressing the offset vector to an error of less than 0.5% (1 / 256%).

[0112] Figure 5 This is a flowchart illustrating a method for processing covariance matrix data in various embodiments.

[0113] Reference Figure 5 In step S310, the computing device 100 can change the way the standard deviation is expressed.

[0114] In various embodiments, computing device 100 can set a second reference range based on the length of the longest diagonal in multiple three-dimensional mesh spaces, and can transform the standard deviation of each three-dimensional mesh space into a value within the second reference range.

[0115] The standard deviation is calculated using multiple points contained in each three-dimensional grid space. The standard deviation cannot exceed the three-dimensional grid space and must be less than or equal to half the length of the longest diagonal of the three-dimensional grid space. Considering this, the computing device 100 can set a second reference range using half the length of the longest diagonal of the three-dimensional grid space, and can transform the standard deviation into a value within this second reference range set according to half the length of the longest diagonal of the three-dimensional grid space.

[0116] In one embodiment, if the covariance matrix is ​​expressed by an elliptic, then as follows: Figure 6As shown, the lengths of each axis are expressed by three standard deviations. By rotating this ellipse, all ellipses in any direction can be represented. At this point, the direction can be arbitrarily set by rotating the ellipse, and even with conditions such as those in Equation 4 below, representing all ellipses is not a problem.

[0117] Mathematical expression 4: σ1≥σ2≥σ3

[0118] Where σ1 can refer to the standard deviation of the longest axis when the covariance matrix is ​​expressed by an elliptic, σ2 can refer to the standard deviation of the second longest axis when the covariance matrix is ​​expressed by an elliptic, and σ3 can refer to the standard deviation of the shortest axis when the covariance matrix is ​​expressed by an elliptic.

[0119] The covariance matrix is ​​calculated using multiple points contained in each three-dimensional mesh space. The standard deviation σ1 of the longest axis must be equal to or less than half the length of the longest diagonal in the three-dimensional mesh space. Therefore, as in one embodiment, if the multiple three-dimensional mesh spaces are cubes, each with one side of length l, then the following mathematical formula 5 holds.

[0120] Mathematical formula 5:

[0121] If we use it, σ1, σ2 and σ3 are expressed as in the following mathematical formula 6, which satisfies the following mathematical formula 7.

[0122] Mathematical formula 6:

[0123] Mathematical expression 7: 0 ≤ α, β, γ ≤ 1

[0124] Therefore, since l is a known value, the computing device 100 can use specific α, β, γ to specify the standard deviation (σ1, σ2, σ3), utilize the values ​​of α, β, γ within a finite range from 0 to 1, and perform reliable compression using small data.

[0125] In various embodiments, the computing device 100 may set the compression degree of the average vector data and the covariance matrix data respectively based on the preset matching performance, and process the average vector data and the covariance matrix data according to the set compression degree. Under the same matching performance, the compression degree of the covariance matrix data may be set to be greater than or equal to the compression degree of the average vector data.

[0126] Matching techniques based on normal distribution transformation are among the most well-known techniques in the SLAM field. They refer to a technique that finds the transformation with the highest similarity between a pre-made normal distribution transformation map and real-time sensor data (e.g., LiDAR point cloud data).

[0127] Registration refers to the process of improving similarity by matching different 3D point clouds (e.g., a normal distribution transformation map made using pre-acquired point cloud data and real-time acquired point cloud data). Registration performance can mean the similarity and / or matching speed between two different data (e.g., different 3D point clouds).

[0128] In matching techniques based on normal distribution transformations, the standard deviation of morphological values ​​has less impact on matching performance compared to rotation transformation angles or average vectors, and can actively approximate (compress) compared to average vectors or rotation transformation angles. Therefore, even if the computing device 100 requires the same matching performance, it can perform a more proactive approximation of the standard deviation compared to rotation transformation angles or average vectors. For example, β and γ can each be expressed using 4 bits in 1 / 16 units, and α can be expressed using 1 byte in 1 / 256 units (based on the commonly used value of l).

[0129] In step S320, the computing device 100 can change the way the rotation angle is expressed.

[0130] In various embodiments, the computing device 100 may set reference rotation ranges for each of the X-axis, Y-axis, and Z-axis, and the rotation transformation angle may be converted to a value within the reference rotation range. For example, the X-axis, Y-axis, and Z-axis may be any axes that are mutually orthogonal.

[0131] The rotation transformation angle of each normal distribution is determined by three values ​​(θ). x θ y θ z This expresses the fact that when the rotation transformation of each of the x, y, and z axes is set to R... x R y R z When the rotation transformation is normal, it can be expressed as mathematical formula 8 below, and each rotation range can satisfy mathematical formula 9 below.

[0132] Mathematical expression 8: R(θ) x θ y θ z ) = R z (θ z )R y (θ y )R x (θ x )

[0133] Mathematical expression 9: 0°≤θ x ≤180°, 0°≤θ y ≤90°, 0°≤θ z ≤360°

[0134] Therefore, the computing device 100 can be expressed from a finite range of values ​​as described in the mathematical formula 9 above, according to the required accuracy, instead of expressing arbitrary real values ​​with respect to direction.

[0135] In step S330, the computing device 100 can generate compressed covariance matrix data using the standard deviation of the covariance matrix data processed in step S310 and the rotation transformation angle of the covariance matrix data processed in step S320.

[0136] That is, the covariance matrix data is expressed by the product of a matrix (D) containing standard deviation information and a matrix (Q) containing rotation transformation angle information, as shown in the following mathematical formula 10. The covariance matrix data is finally compressed by compressing the standard deviation contained in matrix (D) and the rotation transformation angle contained in matrix (Q).

[0137] Mathematical expression 10: (C) = (Q)(D)(Q T )

[0138] Where C can refer to the covariance matrix data, Q can refer to the matrix containing rotation transformation angle information, and D can refer to the matrix containing standard deviation information.

[0139] Typically, each three-dimensional covariance matrix is ​​a 3×3 symmetric matrix, and therefore is represented by 6 real numbers. If each real number is represented by a single-precision floating-point number, then each real number is represented by 4 bytes. In order to represent the three-dimensional covariance matrix data, a total of 24 bytes are required.

[0140] In contrast, instead of using several bytes to express the standard deviation of the covariance matrix data with accurate real values ​​as described above, the values ​​within a second reference range are expressed with fewer bytes, such as a total of 2 bytes (e.g., α is 1 byte, β and γ are 4 bits each), depending on the required accuracy within a limited range. This has the advantage of effectively compressing the standard deviation.

[0141] Furthermore, instead of expressing arbitrary real values ​​for direction, values ​​within a finite range can be used to express the desired accuracy. For example, 8 bits can express 256 values, thus adequately representing 180 degrees. This is further supported by using 3 values ​​(θ). x θ y θ z The values ​​are represented by 8, 7, and 9 bits respectively, totaling only 3 bytes, thus effectively compressing the rotation angle. Specific numerical values ​​are given as examples; therefore, this invention does not need to limit their application.

[0142] Refer again Figure 3In step S140, the computing device 100 can generate a compressed normal distribution transformation map using the compressed average vector data generated by processing the average vector data in step S120 and the compressed covariance matrix data generated by processing the covariance matrix data in step S130.

[0143] That is, the normal distribution transformed map data is a set of normal distributions, each of which is defined by a three-dimensional mean vector and a 3×3 covariance matrix. Since the three-dimensional mean vector is expressed by 3 real numbers and the covariance matrix is ​​expressed by 6 real numbers (if expressed by single-precision floating-point numbers), it has a total size of 36 bytes. In contrast, the compressed normal distribution transformed map data generated by the compression method of the normal distribution transformed map data according to various embodiments of the present invention has a size of about 10 bytes, thereby greatly reducing the size of the normal distribution transformed map data.

[0144] In various embodiments, the computing device 100 calculates the frequency of each object contained in the normal distribution transformed map data, and can set the compression degree of the average vector data and covariance matrix data of the region corresponding to each object on the normal distribution transformed map data based on the calculated frequency, and can process the average vector data and covariance matrix data respectively according to the set compression degree.

[0145] Typically, the normal distributions contained in the normally distributed transformed map data used in autonomous driving systems are composed of normal distributions with specific tendencies, rather than arbitrary normal distributions. For example, in the normal distributions contained in the normally distributed transformed map data, those representing road surfaces and those representing the terrain around the road, such as buildings and trees, make up the majority.

[0146] If we analyze the data based on the shape of the covariance matrix, typically, in the case of road surfaces or building surfaces, it mainly has a flat, disc-shaped elliptical shape, while trees or utility poles have a long, rugby ball-shaped elliptical shape, and other parts have various other elliptical shapes.

[0147] Furthermore, since the normal distribution is a structure obtained by utilizing multiple points contained in each three-dimensional grid space, it can be known that there is a dependency between the average vector data and the covariance matrix data, as well as a dependency between the direction and shape of the covariance matrix data.

[0148] That is, there exist the following impossible combinations: when the covariance matrix data has a specific value, the mean vector data cannot have a specific value, or when the covariance matrix data has a specific value, the mean vector data has a specific value.

[0149] Considering this, the computing device 100 statistically calculates the frequency of each object by discretizing the normal distribution corresponding to each three-dimensional grid space. Based on the calculated frequency, the compression degree of the normal distribution corresponding to objects with a frequency above the reference value is set high, the frequently occurring normal distribution is more aggressively compressed to be expressed by a smaller data size, and the compression degree of the normal distribution corresponding to objects with a frequency below the reference value is set low, thereby allowing the less frequently occurring normal distribution to be expressed by a larger data size.

[0150] In various embodiments, when a first object and a second object located at adjacent positions in a normal distribution transformed map data have the same attributes, the computing device 100 can group the first object and the second object into a group, generate an average vector data and a covariance matrix data by integrating the normal distribution sets of the first object and the second object grouped into a group, and process the generated average vector data and covariance matrix data.

[0151] Typically, in normally distributed map data, there are often dependencies between adjacent normal distributions. For example, adjacent normal distributions on a road surface are mostly flat, disc-shaped and uniformly distributed. In the case of trees or utility poles, the rugby ball-shaped normal distributions representing the pillars and the flat, disc-shaped normal distributions representing the ground are arranged in a manner with specific positional relationships. Therefore, the computing device 100 considers this problem and integrates normal distributions with the same attributes (e.g., species) or specific positional relationships into a single object for processing, thereby enabling more powerful compression.

[0152] In various embodiments, computing device 100 may use normally distributed transformed map data as input to a pre-learned artificial intelligence model to extract compressed normally distributed transformed map data.

[0153] The pre-learning model includes an encoder and a decoder. It can be a model that learns by using multiple compressed normally distributed transformed map data as learning data, such as an autoencoder, but is not limited to this.

[0154] An autoencoder consists of an encoder that transforms given data into code, which is data in small dimensions, and a decoder that uses the code to recover the given data. By learning from a large amount of training data, it learns to find the optimal encoder-decoder pair model. That is, by learning from a large amount of training data, it automatically uses statistical properties, namely, the tendency and / or dependence between normal sets as mentioned above, to find the optimal encoding technique model.

[0155] Therefore, the computing device 100 processes the normal distribution transformation map data by using multiple compressed normal distribution transformation map data, that is, by using the code for designing the size of the normal distribution or the normal distribution set as learning data to learn the normal distribution transformation map data, thereby generating compressed normal distribution transformation map data according to the optimal encoding technique.

[0156] The aforementioned method for compressing normally distributed transformed map data for autonomous driving systems is explained with reference to the flowcharts shown in the accompanying drawings. For simplicity, a series of block diagrams are used to illustrate the method for compressing normally distributed transformed map data for autonomous driving systems. However, the invention is not limited to the order of the blocks; some blocks may be executed in a different order or simultaneously than those shown and described in this specification. Furthermore, new blocks not shown in this specification and the accompanying drawings may be added, or the method may be executed with some blocks deleted or modified.

[0157] The embodiments of the present invention have been described above with reference to the accompanying drawings. It will be understood by those skilled in the art that the present invention can be implemented in other specific forms without altering the technical concept or essential features of the invention. Therefore, the embodiments described above are merely illustrative in all respects and should not be construed as limiting.

Claims

1. A compression method of normal distribution transformed map data for an automated driving system, which is a compression method of normal distribution transformed map data generated as a three-dimensional point cloud of a prescribed area is modeled as a set of normal distributions by a computing device, characterized by comprising steps of: processing mean vector data included in the normal distribution transformed map data; processing covariance matrix data included in the normal distribution transformed map data; and generating compressed normal distribution transformed map data using the processed mean vector data and the processed covariance matrix data, further comprising steps of: generating a plurality of three-dimensional grid spaces by gridding the three-dimensional point cloud of the prescribed area; modeling each of the three-dimensional point clouds included in the generated plurality of three-dimensional grid spaces as a plurality of normal distributions; calculating a mean vector of each of the plurality of normal distributions corresponding to each of the plurality of three-dimensional grid spaces generated, and generating mean vector data using the calculated mean vectors; calculating a covariance matrix of each of the plurality of normal distributions corresponding to each of the plurality of three-dimensional grid spaces generated, and generating covariance matrix data using the calculated covariance matrices; and generating normal distribution transformed map data on the prescribed area using the generated mean vector data and the generated covariance matrix data, the generated mean vector data including a center vector on a center position of each of the plurality of three-dimensional grid spaces generated and an offset vector from the center position, the step of processing the mean vector data including steps of: transforming an expression of the center vector to transform the center vector into a form of a tree data structure; transforming an expression of the offset vector; and generating compressed mean vector data using the transformed center vector and the transformed offset vector, the step of transforming the expression of the offset vector including steps of: setting a first reference range based on a length of one side of each of the plurality of three-dimensional grid spaces generated; and transforming components of the offset vector on each of the plurality of three-dimensional grid spaces generated into values within the set first reference range. The tree data structure includes one or more nodes, and the one or more nodes have a non-linear hierarchical structure.

3. The compression method of normal distribution transformed map data for an automated driving system according to claim 1, characterized in that the generated covariance matrix data includes a standard deviation of a plurality of points included in each of the plurality of three-dimensional grid spaces generated and a rotation transformation angle, the step of processing the covariance matrix data includes steps of: transforming an expression of the standard deviation; transforming an expression of the rotation transformation angle; and generating compressed covariance matrix data using the transformed standard deviation and the transformed rotation transformation angle, the step of transforming the expression of the standard deviation includes steps of: ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 2. The compression method of normal distribution transformed map data for an autonomous driving system according to claim 1, wherein, ​ ​ ​ ​ ​ ​ ​ ​ setting a second reference range based on the length of the diagonal line in the plurality of three-dimensional grid spaces; transforming the standard deviation of each of the plurality of three-dimensional grid spaces into a value within the second reference range, the step of transforming the expression of the rotation transformation angle includes the steps of: setting a reference rotation range for each of the X-axis, Y-axis, and Z-axis; and transforming the rotation transformation angle into a value within the reference rotation range.

4. The compression method of normal distribution transformed map data for an autonomous driving system according to claim 1, wherein, further comprising the steps of: setting the compression degree of the mean vector data and the covariance matrix data based on the predetermined matching performance, and processing the mean vector data and the covariance matrix data according to the set compression degree, wherein the compression degree of the covariance matrix data is set to be greater than or equal to the compression degree of the mean vector data in the case of setting the same matching performance.

5. The compression method of normal distribution transformed map data for an autonomous driving system according to claim 1, wherein, further comprising the steps of: calculating the frequency of each object included in the normal distribution transformation map data, and setting the compression degree of the mean vector data and the covariance matrix data of the region corresponding to each object on the normal distribution transformation map data based on the calculated frequency.

6. The compression method of normal distribution transformed map data for an autonomous driving system according to claim 1, wherein, further comprising the steps of: grouping a first object and a second object disposed at adjacent positions among the plurality of objects included in the normal distribution transformation map data into one group in the case where the first object and the second object have the same attribute, the step of processing the mean vector data includes the steps of generating one mean vector data by combining the plurality of mean vector data of the plurality of objects included in the one group, and processing the generated one mean vector data, the step of processing the covariance matrix data includes the steps of generating one covariance matrix data by combining the plurality of covariance matrix data of the plurality of objects included in the one group, and processing the generated one covariance matrix data.

7. The compression method of normal distribution transformed map data for an autonomous driving system according to claim 1, wherein, further comprising the steps of: grouping two or more objects having a pre-set positional relationship among the plurality of objects included in the normal distribution transformation map data into one group, the step of processing the mean vector data includes the steps of generating one mean vector data by combining the plurality of mean vector data of the plurality of objects included in the one group, and processing the generated one mean vector data, the step of processing the covariance matrix data includes the steps of generating one covariance matrix data by combining the plurality of covariance matrix data of the plurality of objects included in the one group, and processing the generated one covariance matrix data.

8. The compression method of normal distribution transformed map data for an autonomous driving system according to claim 1, wherein, including the steps of: extracting the compressed normal distribution transformation map data as an input of a pre-learned artificial intelligence model including an encoder and a decoder, which is a model learned using a plurality of compressed normal distribution transformation map data as learning data.

9. A computing device that performs the compression method of the normal distribution transformed map data for an autonomous driving system of claim 1, characterized in that, including: a processor; a network interface; a memory; and a memory. A computer program, loaded on the above-mentioned memory, executed by the above-mentioned processor, The above-mentioned computer program contains instructions as follows: processing mean vector data contained in normal distribution transformation map data generated as a normal distribution set modeling a three-dimensional point cloud of a prescribed area; processing covariance matrix data contained in the above-mentioned normal distribution transformation map data; and generating compressed normal distribution transformation map data using the processed above-mentioned mean vector data and the processed above-mentioned covariance matrix data.

10. A computer program product, characterised in that, A computer program, loaded on the above-mentioned memory, executed by the above-mentioned processor, The above-mentioned computer program contains instructions as follows: processing mean vector data contained in normal distribution transformation map data generated as a normal distribution set modeling a three-dimensional point cloud of a prescribed area; processing mean vector data contained in the above-mentioned normal distribution transformation map data; processing covariance matrix data contained in the above-mentioned normal distribution transformation map data; and generating compressed normal distribution transformation map data using the processed above-mentioned mean vector data and the processed above-mentioned covariance matrix data. A computer program, loaded on the above-mentioned memory, executed by the above-mentioned processor, The above-mentioned computer program contains instructions as follows: processing mean vector data contained in normal distribution transformation map data generated as a normal distribution set modeling a three-dimensional point cloud of a prescribed area; processing mean vector data contained in the above-mentioned normal distribution transformation map data; processing covariance matrix data contained in the above-mentioned normal distribution transformation map data; and generating compressed normal distribution transformation map data using the processed above-mentioned mean vector data and the processed above-mentioned covariance matrix data.

Images