Apparatus and method for encoding 3D maps

Abstract

An apparatus and method for encoding a 3D map is provided. The apparatus for encoding a 3D map includes a compression module (51) and a transmission module (52). The compression module (51) is configured to compress data of a 3D map to obtain a bitstream of the 3D map, where the 3D map includes a plurality of 3D map points, and the data of the 3D map includes data of the plurality of 3D map points. The transmission module (52) is configured to transmit the bitstream of the 3D map. The use of the apparatus can reduce the amount of data of the 3D map, thereby reducing the transmission bandwidth and improving the transmission efficiency.

Description

Technical Field

[0001] This application relates to 3D map technology, and particularly to an apparatus and method for encoding a 3D map.

Background Art

[0002] Virtual reality (VR), augmented reality (AR), and mixed reality (MR) are multimedia virtual scene technologies that have emerged in recent years. Such technologies can be used to create virtual realities, overlay them with the real world, and generate new visual environments and interactive experiences. In such applications, an electronic device needs to determine the pose information of the electronic device in the current environment and accurately implement the fusion between virtual objects and the actual scene.

[0003] In addition, in applications such as autonomous driving, autonomous navigation, automatic inspection of unmanned aerial vehicles, and industrial robots, a transport device such as a vehicle, unmanned aerial vehicle, or robot determines the pose of the transport device in the current environment by determining the pose of the electronic device carried by the transport device, and thus needs to execute accurate route planning, navigation, detection, and control.

[0004] In the case of the problem that the pose of an electronic device in the current environment needs to be determined in the above-mentioned applications, a typical solution is as follows: The electronic device receives a 3D map of the environment in which the electronic device is located from a server or another device, collects visual information in the environment by using a local sensor, and determines the current pose of the electronic device based on the collected visual information and the downloaded 3D map.

[0005] However, original 3D maps typically contain a large amount of data, and transmitting these maps requires significant bandwidth and time, which severely limits application performance and impacts the user experience. [Overview of the project]

[0006] This application provides an apparatus and method for encoding a 3D map, reducing the amount of data in the 3D map, thereby reducing the transmission bandwidth and improving transmission efficiency.

[0007] According to a first aspect, the present application provides a codec system for a 3D map, comprising an encoding device and a decoding device. The encoding device is communically connected to the decoding device. The encoding device is configured to compress data of a 3D map to obtain a bitstream of the 3D map, and to transmit the bitstream of the 3D map to the decoding device, wherein the 3D map comprises a plurality of 3D map points, and the data of the 3D map comprises data of the plurality of 3D map points. The decoding device is configured to receive the bitstream of the 3D map, and to decompress the bitstream of the 3D map to obtain data of the 3D map.

[0008] A 3D map may contain multiple 3D map points, and accordingly, the data of the 3D map may contain data for multiple 3D map points. A 3D map point is a point of interest or a point with significant features within the environment.

[0009] In this embodiment of the present application, the compression module can compress the data of a 3D map to reduce the data size of the 3D map, for example, from the terabyte (TB) level to the gigabyte (GB) level. Therefore, in scenarios where a 3D map needs to be transmitted, transmitting the compressed data of the 3D map instead of the original data of the 3D map can reduce the amount of data to be transmitted, further reduce the bandwidth occupied by the transmission, and thereby improve the transmission efficiency of the 3D map.

[0010] In possible implementations, the encoding device is a cloud server and the decoding device is an electronic device; or the encoding device is a first electronic device and the decoding device is a second electronic device. The decoding device is further configured to send a 3D map download request to the decoding device, wherein the 3D map download request includes location information. The encoding device is further configured to receive the 3D map download request and to send to the decoding device a bitstream of the 3D map corresponding to the location information in accordance with the 3D map download request.

[0011] The aforementioned electronic device may be a user terminal device or electronic device being transported by a transport device.

[0012] In possible implementations, the encoding device is an electronic device, and the decoding device is a cloud server. The encoding device is specifically configured to transmit the bitstream of the 3D map to the decoding device after the 3D map has been created.

[0013] Optionally, in this embodiment of the present application, the electronic device may collect visual information by using sensors and determine the current pose of the electronic device by referring to the visual information and a 3D map from a server.

[0014] The 3D map is provided by the server. Specifically, the server creates the 3D map, then compresses it, and transmits the compressed data to the electronic device. After receiving the compressed data, the electronic device decompresses it to obtain the reconstructed 3D map data, and determines the current pose of the electronic device by referring to the collected visual information and the 3D map. The pose is information about the position and orientation of the electronic device, and can be an absolute pose in the world coordinate system or a pose relative to a point in the environment.

[0015] In this embodiment of the present invention, the server may pre-create a 3D map, compress the 3D map, and then store the compressed 3D map data locally. In this way, storage space can be saved. In addition, the server may transmit the compressed 3D map data to another device, such as cloud storage.

[0016] 1. The server creates a 3D map, compresses the 3D map to obtain compressed data of the 3D map, and stores the compressed data locally.

[0017] The server compresses the 3D map to save local storage space.

[0018] 2. The electronic device sends a map download request to the server. Map download requests are triggered in two ways.

[0019] (1) The user starts a map application installed on an electronic device, and the application uploads location information obtained based on GPS positioning or Wi-Fi® positioning to the server corresponding to the application. The upload operation may trigger a map download request. Since the uploaded content includes location information, the server may perform a preliminary estimation based on the location information and transmit compressed data of a 3D map of the area to which the positioning point indicated by the location information belongs to the electronic device. The extent of the area to which the positioning point indicated by the location information belongs can be predetermined. For example, the area to which the positioning point belongs may be an administrative area (including a county, city, country, or administrative region) at any level in which the positioning point is located, or it may be a circular area centered on the positioning point and using a specified distance as its radius.

[0020] (2) The user starts a map application installed on the electronic device and actively inputs or selects an area on the application. For example, the user actively inputs "xx Business Center" or selects "Street A" from the list "Street A, Street B, and Street C". The user's actions described above may trigger a map download request. Regardless of whether the user inputs or selects a geographic location, the server accordingly transmits compressed data of a 3D map of the geographic location to the electronic device.

[0021] In this embodiment of the present application, it should be understood that, in addition to the two methods described above, another method may be used to trigger a map download request. For example, the electronic device may automatically detect whether the conditions for downloading or initiating a 3D map download have been met, or the electronic device may initiate a 3D map download and request a download of a 3D map of a region from the server when it detects a change in ambient light or environmental changes. The size of the region is not specifically limited.

[0022] 3. The server sends the compressed 3D map data to the electronic device.

[0023] 4. The electronic device collects visual information.

[0024] Note that step 3 and step 4 are independent of each other and the sequence is not limited.

[0025] 5. The electronic device decompresses the compressed data of the 3D map to obtain the reconstructed data of the 3D map.

[0026] 6. The electronic device performs positioning in the 3D map based on the visual information and obtains the pose corresponding to the visual information.

[0027] After receiving the compressed data of the 3D map, the electronic device does not necessarily decompress the compressed data immediately. By decompressing the compressed data, it is only necessary to obtain the reconstructed data of the 3D map before performing positioning based on the visual information. For example, the user can download the compressed data of the 3D map of the area range in advance by downloading "offline mapping" and decompress the compressed data of the 3D map only when positioning is required.

[0028] Optionally, in this embodiment of the present application, the electronic device may collect visual information by using a sensor, and the server determines the current pose of the electronic device by referring to the visual information and the 3D map from the electronic device.

[0029] The 3D map is provided by the server. Specifically, the server creates a 3D map, then compresses the 3D map, and stores the compressed data of the 3D map locally. When receiving the visual information from the electronic device, the server obtains the reconstructed data of the 3D map by performing decompression and determines the current pose of the electronic device by referring to the visual information and the 3D map.

[0030] 1. The server creates a 3D map, obtains compressed data of the 3D map by compressing the 3D map, and stores the compressed data locally.

[0031] 2. The electronic device collects visual information.

[0032] 3. The electronic device transmits the visual information to the server.

[0033] 4. The server decompresses the compressed data of the 3D map and obtains reconstructed data of the 3D map.

[0034] It should be understood that the server saves storage space by compressing the 3D map.

[0035] 5. The server performs positioning in the 3D map based on the visual information and obtains the pose corresponding to the visual information.

[0036] 6. The server transmits the pose to the electronic device.

[0037] Optionally, in this embodiment of the present application, the electronic device may collect visual information by using a sensor and determine the current pose of the electronic device by referring to the visual information and the 3D map.

[0038] The 3D map is provided by the electronic device. Specifically, the electronic device creates a 3D map, then compresses the 3D map, and stores the compressed data of the 3D map locally. When the visual information is collected, the electronic device obtains the reconstructed data of the 3D map by performing decompression and determines the current pose of the electronic device by referring to the collected visual information and the 3D map.

[0039] 1. The electronic device creates a 3D map, obtains compressed data of the 3D map by compressing the 3D map, and stores the compressed data locally.

[0040] It's important to understand that electronic devices save storage space by compressing 3D maps.

[0041] 2. Electronic devices collect visual information by using sensors.

[0042] 3. The electronic device decompresses the compressed data of the 3D map and obtains the reconstructed data of the 3D map.

[0043] 4. The electronic device performs positioning on a 3D map based on visual information and acquires a pose corresponding to the visual information.

[0044] Optionally, in this embodiment of the present application, the second electronic device may collect visual information by using a sensor and determine the current pose of the second electronic device by referring to the visual information and a 3D map from a server.

[0045] The 3D map is created by the first electronic device. Specifically, the first electronic device creates the 3D map, compresses it, and then sends the compressed 3D map data to a server. The server then sends the compressed 3D map data to the second electronic device. The second electronic device obtains the reconstructed 3D map data by performing decompression, and determines the current pose of the second electronic device by referring to the collected visual information and the 3D map.

[0046] In this embodiment of the present invention, the first electronic device can pre-create a 3D map, compress the 3D map, and then transmit the compressed data of the 3D map to a server. In this way, the transmission bandwidth can be reduced.

[0047] 1. The first electronic device creates a 3D map and obtains compressed data of the 3D map by compressing the 3D map.

[0048] 2. The first electronic device sends the compressed 3D map data to the server.

[0049] The first electronic device compresses the 3D map, then transmits the compressed 3D map data, reducing the transmission bandwidth and improving transmission efficiency.

[0050] 3. The second electronic device sends a map download request to the server.

[0051] The second electronic device may send a map download request based on the trigger scheme shown in Figure 4a.

[0052] 4. The server transmits the compressed 3D map data to the second electronic device.

[0053] 5. The second electronic device decompresses the compressed data of the 3D map and obtains the reconstructed data of the 3D map.

[0054] 6. The second electronic device collects visual information by using sensors.

[0055] 7. The second electronic device performs positioning on a 3D map based on visual information and acquires a pose corresponding to the visual information.

[0056] Optionally, in this embodiment of the present application, the second electronic device may collect visual information by using a sensor, and the server determines the current pose of the second electronic device by referring to the visual information from the second electronic device and a 3D map from the first electronic device.

[0057] The 3D map is created by the first electronic device. Specifically, the first electronic device creates the 3D map, compresses it, and then sends the compressed 3D map data to the server. The server obtains the reconstructed 3D map data by decompressing it, and determines the current pose of the second electronic device by referring to the visual information from the second electronic device and the 3D map.

[0058] 1. The first electronic device creates a 3D map and obtains compressed data of the 3D map by compressing the 3D map.

[0059] 2. The first electronic device sends the compressed 3D map data to the server.

[0060] 3. The second electronic device collects visual information by using sensors.

[0061] 4. The second electronic device sends a positioning request to the server, where the positioning request contains visual information.

[0062] 5. The server decompresses the compressed 3D map data and obtains the reconstructed 3D map data.

[0063] 6. The server performs positioning on the 3D map based on visual information and obtains a pose corresponding to the visual information.

[0064] 7. The server transmits the pose obtained through positioning to the second electronic device.

[0065] Optionally, in this embodiment of the present application, the second electronic device may collect visual information by using a sensor and determine the current pose of the second electronic device by referring to the visual information and a 3D map from the first electronic device.

[0066] The 3D map is created by the first electronic device. Specifically, the first electronic device creates the 3D map, compresses the 3D map, and then sends the compressed 3D map data to the second electronic device. The second electronic device obtains the reconstructed 3D map data by performing decompression, and determines the current pose of the second electronic device by referring to the collected visual information and the 3D map from the first electronic device.

[0067] 1. The first electronic device creates a 3D map, compresses the 3D map to obtain compressed data of the 3D map, and stores the compressed data locally.

[0068] 2. The second electronic device sends a map download request to the first electronic device.

[0069] 3. The first electronic device transmits the compressed 3D map data to the second electronic device.

[0070] 4. The second electronic device decompresses the compressed data of the 3D map and obtains the reconstructed data of the 3D map.

[0071] 5. The second electronic device collects visual information by using sensors.

[0072] 6. The second electronic device performs positioning on a 3D map based on visual information and acquires a pose corresponding to the visual information.

[0073] In a possible implementation, the data of the 3D map further includes a plurality of region descriptors, any one of which describes the characteristics of some or all of the plurality of 3D map points.

[0074] In the case of any one of multiple domain descriptors, the domain descriptor may describe the features of some or all of the 3D map points. In this case, there is a one-to-many relationship between the domain descriptor and the 3D map points. Each feature of the multiple 3D map points may be described by some or all of the domain descriptors of the multiple domain descriptors. In this case, there is a one-to-many relationship between the 3D map points and the domain descriptors. It can be seen that there is a many-to-many relationship between multiple domain descriptors and multiple 3D map points. Methods for generating domain descriptors are not limited to but include conventional methods such as the bag of words (BOW) and the vector of locally aggregated descriptors (VLAD), as well as newer methods based on NetVLAD or artificial intelligence (AI). Similarly, multiple domain descriptors may be identified by a number to distinguish between them. Similarly, however, the number is not intended to limit the sequence of multiple domain descriptors.

[0075] In a possible implementation, the data for any one of the plurality of 3D map points includes a 3D map point descriptor and a 3D map point spatial location.

[0076] A 3D map point descriptor is a vector used to represent the local features of the corresponding 3D map point.

[0077] The spatial location of a 3D map point may be represented using X, Y, and Z on the three-dimensional spatial axes, or using longitude, latitude, and altitude, or using polar coordinates, and so on. The method of representing the spatial location of a 3D map point is not specifically limited to the embodiments of this application. The spatial location of a 3D map point may be the absolute position of the 3D map point or the relative position of the 3D map point.

[0078] In possible implementations, the encoding device is further configured to create the 3D map.

[0079] In possible implementations, the decoding device is further configured to perform positioning based on the 3D map.

[0080] According to a second aspect, the present invention provides an apparatus for encoding a 3D map, comprising a compression module and a transmission module. The compression module is configured to compress data of a 3D map to obtain a bitstream of the 3D map, wherein the 3D map comprises a plurality of 3D map points, and the data of the 3D map comprises data of the plurality of 3D map points. The transmission module is configured to transmit the bitstream of the 3D map.

[0081] In this embodiment of the present application, the compression module can perform compression on the data of a 3D map to reduce the amount of data in the 3D map. In scenarios where the 3D map needs to be transmitted, transmitting the compressed data of the 3D map instead of the original data of the 3D map can reduce the amount of data for transmission, further reduce the bandwidth occupied by the transmission, and thereby improve the transmission efficiency of the 3D map.

[0082] In this embodiment of the present application, compression may include at least one of compactification and prediction, and compactification may include at least one of quantization and binarization.

[0083] Quantization means mapping the data to be processed to one or more quantization exponents, where each quantization exponent corresponds to one quantization center. The number of bits in a quantization exponent is usually significantly smaller than the number of bits in the original data, saving storage or transmission bandwidth. Quantization methods are not limited to but include scalar quantization, vector quantization, and direct product quantization.

[0084] Binarization means processing data into a binary sequence represented by binary symbols. Binarization can be, for example, hashing. The principle of hashing is to map the data to be processed into Hamming space (binary space) and generate a binary hash code. The number of bits in the hash code is usually significantly smaller than the number of bits in the original data, saving storage and transmission bandwidth. In addition, the computational complexity of the Hamming distance between hash codes is usually smaller than that of the Euclidean distance of the original data, thereby reducing computational complexity. Hashing methods are not limited to but include iterative quantization (ITQ) hashing methods and locality-sensitive hashing (LSH) methods.

[0085] Prediction means performing a prediction on the data to be processed by using the processed data, and obtaining the residual data of the data to be processed. Clearly, the amount of data in the residual data is smaller than the amount of data in the original data, thereby implementing data compression. The selection of reference data can be agreed upon in advance. For example, previously processed data is fixedly used as reference data, in which case the reference data does not need to be identified in the bitstream. In another example, any processed data is used as reference data, in which case the identifying information of the reference data needs to be written to the bitstream, including the number of reference data or other information that can be used to infer the reference data.

[0086] In the above description of 3D map data, it can be seen that sequences of multiple 3D map points included in a 3D map are meaningless. Therefore, when compression or encoding of 3D map points is involved, the sequences of multiple 3D map points are not limited; that is, multiple 3D map points can be compressed or encoded separately in any sequence.

[0087] Based on the prediction principle, if the similarity between the predicted data and the data to be processed is relatively high, the probability that the acquired residual data for the data to be processed is 0 is relatively high, and as a result, compression performance may be improved, thereby reducing the amount of data required for encoding. In this embodiment of the present application, multiple data to be processed may be reordered before prediction to improve the correlation between adjacent data to be processed, thereby further reducing the amount of residual data. Optionally, one data to be processed may be predicted based on one or more processed data, and residual data for the data to be processed may be acquired; or multiple data to be processed may be predicted based on one or more processed data, and residual data for multiple data to be processed may be acquired.

[0088] In addition, compression may further involve encapsulation to encapsulate the data to be encoded in a bitstream. Encapsulation can use any encoding algorithm, such as entropy coding. Entropy coding is a lossless data compression method. Entropy coding algorithms include, but are not limited to, Huffman coding, arithmetic coding, improved compression / decompression algorithms based on the LZ77 compression algorithm (lempel-ziv-markov chain-algorithm, LZMA), and function library algorithms for data compression (zlib).

[0089] In possible implementations, the 3D map data further includes multiple region descriptors, any one of which describes the characteristics of some or all of the 3D map points.

[0090] In possible implementations, the data for any one of multiple 3D map points includes a 3D map point descriptor and a 3D map point spatial location.

[0091] In possible implementations, the device for encoding the 3D map is a cloud server or an electronic device. The transmission module is further configured to: receive a 3D map download request, wherein the 3D map download request includes location information; and transmit, in accordance with the 3D map download request, a bitstream of the 3D map corresponding to the location information.

[0092] In possible implementations, the apparatus for encoding the 3D map is an electronic device. The transmission module is specifically configured to transmit the bitstream of the 3D map after the 3D map has been created.

[0093] The following are several implementations of the compression module:

[0094] In a possible implementation, the compression module includes a compactification module and / or a prediction module, and an encapsulation module. The compactification module is configured to perform compactification on input first data and output the compactified data of the first data. The prediction module is configured to perform prediction on input second data and output residual data of the second data. The encapsulation module is configured to process input third data and output the bitstream of the 3D map. The first data is the data of the 3D map, the second data is the data of the 3D map or the compacted data of the first data, and the third data is the compacted data of the first data or the residual data of the second data.

[0095] Optionally, the compression module may include only the compactification module or the prediction module, and the encapsulation module.

[0096] Optionally, the compression module includes a compactification module, a prediction module, and an encapsulation module.

[0097] In a possible implementation, the compactification module includes a quantization module and / or a binarization module. The quantization module is configured to perform quantization on an input fourth data and output the quantized data of the fourth data. The binarization module is configured to perform binarization on an input fifth data and output the binary data of the fifth data. The fourth data is the data of the 3D map, and the fifth data is the data of the 3D map or the quantized data of the fourth data.

[0098] Optionally, the compactification module may include only the quantization module or the binarization module.

[0099] Optionally, the compactification module includes a quantization module and a binarization module.

[0100] In a possible implementation, the quantization module includes a first quantization module and a second quantization module, and / or the binarization module includes a first binarization module and a second binarization module, and the prediction module includes a first prediction module and a second prediction module. The first quantization module is configured to perform quantization on an input sixth data and output the quantized data of the sixth data. The first binarization module is configured to perform binarization on an input seventh data and output the binary data of the seventh data. The first prediction module is configured to perform prediction on an input eighth data and output the residual data of the eighth data. The sixth data is one of the plurality of region descriptors, the seventh data is one of the plurality of region descriptors or the quantized data of the sixth data, and the eighth data is one of the plurality of region descriptors, the quantized data of the sixth data, or the binary data of the seventh data. The second quantization module is configured to perform quantization on an input ninth data and output the quantized data of the ninth data. The second binarization module is configured to perform binarization on the input 10th data and output the binary data of the 10th data. The first prediction module is configured to perform prediction on the input 11th data and output the residual data of the 11th data. The 9th data is one of the plurality of 3D map points, the 10th data is one of the plurality of 3D map points or the quantized data of the 9th data, and the 11th data is one of the plurality of 3D map points, the quantized data of the 9th data, or the binary data of the 10th data.

[0101] In a possible implementation, the quantization module includes a first quantization module, a second quantization module, and a third quantization module, and / or the binarization module includes a first binarization module, a second binarization module, and a third binarization module, and the prediction module includes a first prediction module, a second prediction module, and a third prediction module. The first quantization module is configured to perform quantization on an input 12th data and output the quantized data of the 12th data. The first binarization module is configured to perform binarization on an input 13th data and output the binary data of the 13th data. The first prediction module is configured to perform prediction on an input 14th data and output the residual data of the 14th data. The 12th data is one of the plurality of region descriptors, the 13th data is one of the plurality of region descriptors or the quantized data of the 12th data, and the 14th data is one of the plurality of region descriptors, the quantized data of the 12th data, or the binary data of the 13th data. The second quantization module is configured to perform quantization on the input 15th data and output the quantized data of the 15th data. The second binarization module is configured to perform binarization on the input 16th data and output the binary data of the 16th data. The second prediction module is configured to perform prediction on the input 17th data and output the residual data of the 17th data. The 15th data is a 3D map point descriptor of one of the plurality of 3D map points, the 16th data is a 3D map point descriptor of one of the plurality of 3D map points or the quantized data of the 15th data, and the 17th data is a 3D map point descriptor of one of the plurality of 3D map points, the quantized data of the 15th data, or the binary data of the 16th data. The third quantization module is configured to perform quantization on the input 18th data and output the quantized data of the 18th data.The third binarization module is configured to perform binarization on the input 19th data and output the binary data of the 19th data. The third prediction module is configured to perform prediction on the input 20th data and output the residual data of the 20th data. The 18th data is the spatial location of one of the plurality of 3D map points, the 19th data is the spatial location of one of the plurality of 3D map points or the quantized data of the 18th data, and the 20th data is the spatial location of one of the plurality of 3D map points, the quantized data of the 18th data, or the binary data of the 19th data.

[0102] In a possible implementation, the compression module includes a first compression submodule and a second compression submodule. The first compression submodule is configured to compress input 21st data and output a bitstream of the 21st data. The second compression submodule is configured to compress input 22nd data and output a bitstream of the 22nd data. The 21st data is one of the plurality of region descriptors, and the 22nd data is data of one of the plurality of 3D map points.

[0103] In a possible implementation, the first compression submodule includes a first compactification module and / or a first prediction module, and a first encapsulation module; the second compression submodule includes a second compactification module and / or a second prediction module, and a second encapsulation module. The first compactification module is configured to perform compactification on an input 23rd data and output the compactified data of the 23rd data. The first prediction module is configured to perform prediction on an input 24th data and output the residual data of the 24th data. The first encapsulation module is configured to process an input 25th data and output a bitstream of the 25th data. The 23rd data is one of the plurality of region descriptors, the 24th data is one of the plurality of region descriptors or the compacted data of the 23rd data, and the 25th data is the compacted data of the 23rd data or the residual data of the 24th data. The second compactification module is configured to perform compactification on an input 26th data and output the compacted data of the 26th data. The second prediction module is configured to perform a prediction on the input 27th data and output residual data for the 27th data. The second encapsulation module is configured to process the input 28th data and output a bitstream of the 28th data. The 26th data is data for one of the plurality of 3D map points, the 27th data is data for one of the plurality of 3D map points or the compacted data for the 26th data, and the 28th data is the compacted data for the 26th data or the residual data for the 27th data.

[0104] In a possible implementation, the first compactification module includes a first quantization module and / or a first binarization module; the second compactification module includes a second quantization module and / or a second binarization module. The first quantization module is configured to perform quantization on an input 29th data and output the quantized data of the 29th data. The first binarization module is configured to perform binarization on an input 30th data and output the binary data of the 30th data. The 29th data is one of the plurality of region descriptors, and the 30th data is one of the plurality of region descriptors or the quantized data of the 29th data. The second quantization module is configured to perform quantization on an input 31st data and output the quantized data of the 31st data. The second binarization module is configured to perform binarization on an input 32nd data and output the binary data of the 32nd data. The 31st data is one of the plurality of 3D map points, and the 30th data is either one of the plurality of 3D map points or the quantized data of the 31st data.

[0105] In a possible implementation, the compression module includes a first compression submodule, a second compression submodule, and a third compression submodule. The first compression submodule is configured to compress input 33 data and output a bitstream of the 33 data. The second compression submodule is configured to compress input 34 data and output a bitstream of the 34 data. The third compression submodule is configured to compress input 35 data and output a bitstream of the 35 data. The 33 data is one of the plurality of region descriptors, the 34 data is a 3D map point descriptor of one of the plurality of 3D map points, and the 35 data is the spatial location of one of the plurality of 3D map points.

[0106] In a possible implementation, the first compression submodule includes a first compactification module and / or a first prediction module, and a first encapsulation module; the second compression submodule includes a second compactification module and / or a second prediction module, and a second encapsulation module; and the third compression submodule includes a third compactification module and / or a third prediction module, and a third encapsulation module. The first compactification module is configured to perform compactification on an input 36th data and output the compactified data of the 36th data. The first prediction module is configured to perform prediction on an input 37th data and obtain residual data of the 37th data. The first encapsulation module is configured to process an input 38th data and obtain a bitstream of the 38th data. The 36th data is one of the plurality of region descriptors, the 37th data is one of the plurality of region descriptors or the compacted data of the 36th data, and the 38th data is the compacted data of the 36th data or the residual data of the 37th data. The second compactification module is configured to perform compactification on the input 39th data and output the compactified data of the 39th data. The second prediction module is configured to perform prediction on the input 40th data and obtain residual data of the 40th data. The second encapsulation module is configured to process the input 41st data and obtain a bitstream of the 41st data. The 39th data is a 3D map point descriptor for one of the plurality of 3D map points, the 40th data is either the 3D map point descriptor for one of the plurality of 3D map points or the compacted data of the 39th data, and the 41st data is either the compacted data of the 39th data or the residual data of the 40th data.The third compactification module is configured to perform compactification on the input 42nd data and output the compactified data of the 42nd data. The third prediction module is configured to perform prediction on the input 43rd data and obtain the residual data of the 43rd data. The third encapsulation module is configured to process the input 44th data and obtain the bitstream of the 44th data. The 42nd data is the spatial location of one of the plurality of 3D map points, the 43rd data is the spatial location of one of the plurality of 3D map points or the compacted data of the 42nd data, and the 44th data is the compacted data of the 42nd data or the residual data of the 43rd data.

[0107] In a possible implementation, the first compactification module includes a first quantization module and / or a first binarization module; the second compactification module includes a second quantization module and / or a second binarization module; and the third compactification module includes a third quantization module and / or a third binarization module. The first quantization module is configured to perform quantization on an input 45th data and output the quantized data of the 45th data. The first binarization module is configured to perform binarization on an input 46th data and output the binary data of the 46th data. The 45th data is one of the plurality of region descriptors, and the 46th data is one of the plurality of region descriptors or the quantized data of the 45th data. The second quantization module is configured to perform quantization on an input 47th data and output the quantized data of the 47th data. The second binarization module is configured to perform binarization on an input 48th data and output the binary data of the 48th data. The 47th data is a 3D map point descriptor for one of the plurality of 3D map points, and the 48th data is the quantized data of either the 3D map point descriptor for one of the plurality of 3D map points or the 47th data. The third quantization module is configured to perform quantization on the input 49th data and output the quantized data of the 49th data. The third binarization module is configured to perform binarization on the input 50th data and output the binary data of the 50th data. The 49th data is a spatial position for one of the plurality of 3D map points, and the 50th data is the spatial position for one of the plurality of 3D map points or the quantized data of the 49th data.

[0108] According to a third aspect, the present invention relates to a method for encoding a 3D map, comprising the steps of processing data of a 3D map and obtaining a bitstream of the 3D map, wherein the 3D map comprises a plurality of 3D map points, and the data of the 3D map comprises data of the plurality of 3D map points; and The step of transmitting the bitstream of the 3D map. This provides a method that includes [something].

[0109] In this embodiment of the present application, the compression module can perform compression on the data of a 3D map to reduce the amount of data in the 3D map. In scenarios where the 3D map needs to be transmitted, transmitting the compressed data of the 3D map instead of the original data of the 3D map can reduce the amount of data for transmission, further reduce the bandwidth occupied by the transmission, and thereby improve the transmission efficiency of the 3D map.

[0110] In possible implementations, the 3D map data further includes multiple region descriptors, any one of which describes the characteristics of some or all of the 3D map points.

[0111] In possible implementations, the data for any one of multiple 3D map points includes a 3D map point descriptor and a 3D map point spatial location.

[0112] In possible implementations, the method further: The step of receiving a 3D map download request, wherein the 3D map download request includes location information; the step of transmitting a bitstream of the 3D map includes: transmitting a bitstream of the 3D map corresponding to the location information in accordance with the 3D map download request.

[0113] In a possible implementation, the step of transmitting the bitstream of the 3D map includes the step of transmitting the bitstream of the 3D map after the 3D map has been created.

[0114] In a possible implementation, the step of processing the data of the 3D map and obtaining a bitstream of the 3D map includes: performing compaction on first data and obtaining compacted data of the first data, and / or performing prediction on second data and obtaining residual data of the second data; and processing third data and obtaining the bitstream of the 3D map. The first data is the data of the 3D map, the second data is the data of the 3D map or the compacted data of the first data, and the third data is the compacted data of the first data or the residual data of the second data.

[0115] In a possible implementation, prior to the step of performing a prediction on the second data and obtaining residual data for the second data, the method further includes: a step of reordering a plurality of the second data, and the step of performing a prediction on the second data and obtaining residual data for the second data includes: a step of performing a prediction on at least a portion of the second data and obtaining residual data for at least a portion of the second data based on the results of the reordering.

[0116] In a possible implementation, the step of performing compactification on a first data and obtaining the compactified data of the first data includes: performing quantization on a fourth data and obtaining the quantized data of the fourth data, and / or performing binarization on a fifth data and obtaining the binary data of the fifth data. The fourth data is the first data, and the fifth data is the quantized data of the first data or the fourth data, and accordingly, the compactified data of the first data includes the quantized data of the fourth data and / or the binary data of the fifth data.

[0117] According to a fourth aspect, the present invention provides an apparatus for encoding a 3D map, comprising an encoder configured to compress data of a 3D map to obtain a 3D map obtained by compression and in bitstream form, wherein the 3D map comprises a plurality of 3D map points, the data of the 3D map comprises the data of the plurality of 3D map points; and a memory configured to store the 3D map obtained by compression and in bitstream form.

[0118] In this embodiment of the present application, the compression module can perform compression on the 3D map data to reduce the amount of data in the 3D map. In scenarios where the 3D map needs to be stored, storing the compressed 3D map in bitstream format instead of storing the original 3D map data can reduce the amount of data for transmission and further save storage space.

[0119] In possible implementations, the 3D map data further includes multiple region descriptors, any one of which describes the characteristics of some or all of the 3D map points.

[0120] In possible implementations, the data for any one of multiple 3D map points includes a 3D map point descriptor and a 3D map point spatial location.

[0121] In possible implementations, the device for encoding the 3D map is either a cloud server or an electronic device.

[0122] According to the fifth aspect, the present invention provides a computer-readable storage medium containing a computer program. When the computer program is executed on a computer, the computer is able to perform the method described in any one of the implementations of the third aspect.

[0123] According to the sixth aspect, the present application provides a computer program. When the computer program is executed on a computer, the computer is able to perform the method described in any one of the implementations of the third aspect.

[0124] According to the seventh aspect, the present application provides a non-temporary storage medium comprising a bitstream encoded according to the method described in the third aspect or in any one of the implementations of the third aspect. [Brief explanation of the drawing]

[0125] [Figure 1] This is a schematic diagram of the application architecture according to the present invention.

[0126] [Figure 2] This is a schematic diagram of the structure of the electronic device 20 according to the present invention.

[0127] [Figure 3] This is a schematic diagram of the structure of the server 30 according to the present invention.

[0128] [Figure 4a] This is a schematic diagram of an application scenario according to the present invention.

[0129] [Figure 4b] This is a schematic diagram of an application scenario according to the present invention.

[0130] [Figure 4c] This is a schematic diagram of an application scenario according to the present invention.

[0131] [Figure 4d] This is a schematic diagram of an application scenario according to the present invention.

[0132] [Figure 4e] This is a schematic diagram of an application scenario according to the present invention.

[0133] [Figure 4f] This is a schematic diagram of an application scenario according to the present invention.

[0134] [Figure 4g] This is a schematic diagram of a user interface displayed by an electronic device according to an embodiment of the present application.

[0135] [Figure 5] This is a diagram showing the structure of a device 50 for encoding a 3D map according to an embodiment of the present application.

[0136] [Figure 6a] This is a diagram showing the structure of a device 60-1 for encoding a 3D map according to an embodiment of the present application.

[0137] [Figure 6b] This is a diagram showing the structure of a device 60-2 for encoding a 3D map according to an embodiment of the present application.

[0138] [Figure 6c] This is a diagram showing the structure of a device 60-3 for encoding a 3D map according to an embodiment of the present application.

[0139] [Figure 6d] This is a diagram showing the structure of the compact module 611-1 / 611-2 according to an embodiment of the present application.

[0140] [Figure 6e] This is a diagram showing the structure of a device 60-4 for encoding a 3D map according to an embodiment of the present application.

[0141] [Figure 6f] This is a diagram showing the structure of a device 60-5 for encoding a 3D map according to an embodiment of the present application.

[0142] [Figure 7a] This is a diagram showing the structure of a device 70-1 for encoding a 3D map according to an embodiment of the present application.

[0143] [Figure 7b] This is a diagram showing the structure of a device 70-2 for encoding a 3D map according to an embodiment of the present application.

[0144] [Figure 8a] This is a diagram showing the structure of an apparatus 80-1 for encoding a 3D map according to an embodiment of the present application.

[0145] [Figure 8b] This is a diagram showing the structure of an apparatus 80-2 for encoding a 3D map according to an embodiment of the present application.

[0146] [Figure 9] This is a flowchart of process 900 of a method for encoding a 3D map according to an embodiment of the present application.

[0147] [Figure 10] This is a diagram showing the structure of the apparatus 100 for encoding a 3D map according to an embodiment of the present application. [Modes for carrying out the invention]

[0148] To further clarify the purpose, technical solution, and advantages of this application, the technical solution described herein will be explained below with reference to the accompanying drawings. Certainly Clearly, the embodiments described herein are not all but a selection of the embodiments of this application. All other embodiments obtained by those skilled in the art without creative effort based on the embodiments of this application are included within the scope of protection of this application.

[0149] In the embodiments, claims, and accompanying drawings of this specification, terms such as “first” and “second” are intended merely for distinction and descriptive purposes and are not to be understood as indicating or suggesting relative importance or order. In addition, the terms “include” and “have” and any variations thereof are intended to cover non-exclusive inclusion, e.g., inclusion of a set of steps or units. A method, system, product, or device is not necessarily limited to the steps or units explicitly listed, and may include other steps or units specific to the process, method, product, or device that are not explicitly listed.

[0150] In this application, it should be understood that "at least one (item)" means one or more, and "a plurality of" means two or more. The terms "and / or" describe the correspondence of related objects and indicate that there can be three relationships. For example, "A and / or B" can indicate the following three cases: only A exists, only B exists, and both A and B exist. A and B can be singular or plural. The letter " / " usually indicates an "or" relationship between related objects. "At least one of the following items" or a similar expression indicates any combination of items that includes any combination of one or more of the items. For example, at least one of a, b or c may indicate a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be singular or plural.

[0151] Figure 1 is a schematic diagram of an application architecture according to an embodiment of the present application. As shown in Figure 1, the application architecture includes a plurality of electronic devices and a server. The plurality of electronic devices may include a first electronic device and one or more second electronic devices (two second electronic devices are used as an example in Figure 1). One or more second electronic devices are electronic devices other than the first electronic device. Communication may be performed between the plurality of electronic devices and the server, and between the plurality of electronic devices. For example, any device in the application architecture may communicate with another device using methods such as wireless fidelity (Wi-Fi), Bluetooth®, or cellular 2nd / 3rd / 4th / 5th generation (2G / 3G / 4G / 5G) communication. It should be understood that other communication methods, including future communication methods, may be used further between the server and the electronic devices. This specification does not particularly limit this. In this embodiment of the present application, "one or more second electronic devices" is used simply to represent electronic devices other than the first electronic device, but it should be noted that this does not limit whether the types of the multiple electronic devices are the same.

[0152] The electronic device can be any of the various types of devices provided with camera and display components. For example, the electronic device may be a terminal device such as a mobile phone, tablet computer, notebook computer, or video recorder (a mobile phone is used as an example of an electronic device in Figure 1). Alternatively, the electronic device may be a device used for interaction in virtual scenarios, including VR glasses, AR devices, and MR interaction devices. Alternatively, the electronic device may be a wearable electronic device such as a smartwatch or smart band. Alternatively, the electronic device may be a device carried in a transport device such as a vehicle, unmanned vehicle, unmanned aerial vehicle, or industrial robot. The specific form of the electronic device is not particularly limited to the embodiments of this application.

[0153] In addition, electronic devices may also be referred to as user equipment (UE), subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, terminal device, access terminal, mobile terminal, wireless terminal, smart terminal, remote terminal, handheld terminal, user agent, mobile client, client, or other appropriate terminology.

[0154] A server may be one or more physical servers (one physical server is used as an example in Figure 1), or a computer cluster, or a virtual machine or cloud server in a cloud computing scenario, and so on.

[0155] In this embodiment of the present application, a virtual scenario application (application, APP), such as a VR application, AR application, or MR application, may be installed on an electronic device, and the VR application, AR application, or MR application may be executed based on user interaction (e.g., tap, touch, slide, shake, or voice control). The electronic device may collect visual information of any object in the environment by using sensors, and then display virtual objects on a display component based on the collected visual information. The virtual objects may be virtual objects in a VR scenario, AR scenario, or MR scenario (i.e., objects in a virtual environment).

[0156] In this embodiment of the present application, navigation, detection, or control applications may be installed on an electronic device, and the corresponding applications may be executed based on user operation and control or on a pre-configured program. The electronic device may execute applications such as route planning, object detection, and operation and control of transport devices based on the electronic device's pause and other status information in the current environment.

[0157] The visual information in the embodiments of this application includes, but is not limited to, images and videos collected by a camera (without depth information), images and videos collected by a depth sensor (with depth information), data collected by a LiDAR, and data collected by a millimeter-wave radar (RaDAR).

[0158] In this embodiment of the present application, it should be noted that the virtual scenario application for an electronic device may be an application built into the electronic device, or an application provided by a third-party service provider and installed by the user. This specification is not limited to this.

[0159] In this embodiment of the present application, a simultaneous localization and mapping (SLAM) system may be further configured for an electronic device. The SLAM system may create a map of a completely unknown environment and use the map to perform positioning, pose (location and pose) determination, navigation, etc. In this embodiment of the present application, the map created by the SLAM system is referred to as a SLAM map. A SLAM map may be understood as a map drawn by the SLAM system based on environmental information collected by an acquisition device. The acquisition device may include a visual information acquisition device and an inertial measurement unit (IMU) in an electronic device. The visual information acquisition device may include, for example, a camera, a depth camera, a lidar, and a millimeter-wave radar. The IMU may include, for example, sensors such as a gyroscope and an accelerometer.

[0160] In embodiments of this application, the SLAM map is also referred to as a 3D map. It should be noted that the 3D map may include, but is not limited to, a SLAM map, and may further include a three-dimensional map created by using another technique. This is not specifically limited to embodiments of this application.

[0161] In possible implementations, a 3D map may contain multiple 3D map points, and accordingly, the data of the 3D map may contain data for multiple 3D map points. A 3D map point is a point of interest or a point with significant features within the environment.

[0162] Possible methods for acquiring 3D map points include using multiple devices such as LiDAR, aerial photography from the field of view of an unmanned aerial vehicle (tilt-shift photography), high-resolution panoramic cameras, and high-resolution industrial cameras to perform the photography. From the data acquired by the aforementioned devices, directional features from accelerated segment test (FAST) and rotational binary robust independent elementary features (BRIEF) are extracted. ), directional FAST and rotational BRIEF ( 3D map points are extracted using methods such as oriented FAST and rotated BRIEF (ORB), scale invariant feature transform (SIFT), speeded-up robust feature (SURF), BRIEF, binary robust invariant scalable keypoints (BRISK), fast retina keypoint (FREAK), or repeatable and reliable detector and descriptor (R2D2).

[0163] 3D map point data may include the following:

[0164] (1) 3D map point descriptor

[0165] A 3D map point descriptor is a vector used to represent the local features of a 3D map point. In visual positioning algorithms, 3D map point descriptors are used for matching between 3D map points. Possible methods include: calculating the distance between two 3D map point descriptors (which could be the Euclidean distance, dot product distance, or Hamming distance, etc.); and considering the two 3D map points to be a match if the above distance is less than a threshold.

[0166] (2) 3D map point spatial position

[0167] The spatial location of a 3D map point may be represented using X, Y, and Z on the three-dimensional spatial axes, or using longitude, latitude, and altitude, or using polar coordinates, and so on. The method of representing the spatial location of a 3D map point is not specifically limited to the embodiments of this application. The spatial location of a 3D map point may be the absolute position of the 3D map point or the relative position of the 3D map point. For example, the center of the entire region may be used as the origin, and all 3D map point spatial locations may be offset positions relative to the spatial location of the origin.

[0168] In embodiments of the present invention, each 3D map point may be assigned a number which can be written to the data of the 3D map, or a storage sequence of multiple 3D map points in memory may be used to implicitly indicate the number of the 3D map point. Note that a sequence of multiple 3D map points included in a 3D map is meaningless. Therefore, the aforementioned numbers can be considered identifiers used to identify 3D map points in order to distinguish them. However, the numbers are not intended to limit the sequence of multiple 3D map points. For example, a 3D map may include three 3D map points numbered 1, 2, and 3, respectively, and these three 3D map points may be processed in the order 1, 2, and 3, or 3, 2, and 1, or 2, 1, and 3, and so on.

[0169] In possible implementations, the 3D map data further includes multiple region descriptors, one of which describes the features of some or all of the 3D map points. Specifically, if there is one of the multiple region descriptors, the region descriptor may describe the features of some or all of the 3D map points. In this case, there is a one-to-many relationship between the region descriptor and the 3D map points. Each feature of the 3D map points may be described by some or all of the multiple region descriptors. In this case, there is a one-to-many relationship between the 3D map points and the region descriptors. It can be seen that there is a many-to-many relationship between the multiple region descriptors and the multiple 3D map points. The methods for generating region descriptors are not limited to but include conventional methods such as the bag of words (BOW) and vectors of locally aggregated descriptors (VLAD), as well as newer methods based on NetVLAD or artificial intelligence (AI). Similarly, multiple region descriptors may be identified by a number to distinguish them from each other. However, the number is not intended to limit the sequence of multiple region descriptors.

[0170] In possible implementations, 3D map data further includes correspondences between 3D map points and region descriptors. These correspondences clearly describe which 3D map point corresponds to any given region descriptor, and which region descriptor corresponds to any given 3D map point.

[0171] Optionally, the aforementioned correspondence can be explicitly described by using a correspondence table between region descriptor numbers and 3D map point numbers. For example, a 3D map contains three region descriptors whose numbers are T1 to T3, and six 3D map points whose numbers are P1 to P6. The correspondence table is: 1 It is shown. [Table 1] [Table 1]

[0172] Please note that Table 1 is an example of a correspondence table between region descriptor numbers and 3D map point numbers. The correspondence table may be presented in a different format or manner, and is not specifically limited in this application.

[0173] Optionally, the aforementioned correspondence can be implicitly explained by using the storage locations of region descriptors and 3D map points. For example, T1 is stored in memory first, then the data for P1, P2, and P3; then T2 is stored, then the data for P2 and P3; and finally T3 is stored, then the data for P3, P4, P5, and P6.

[0174] Figure 2 is a schematic diagram of the structure of an electronic device 20 according to an embodiment of the present application. As shown in Figure 2, the electronic device 20 may be at least one of the first electronic device and one or more second electronic devices in the embodiment shown in Figure 1. It should be understood that the structure shown in Figure 2 does not constitute a specific limitation on the electronic device 20. In some other embodiments of the present application, the electronic device 20 may include more or fewer components than those shown in Figure 2, or it may be a combination of some components, or a split of some components, or it may have a different component arrangement. The components shown in Figure 2 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0175] The electronic device 20 may include a chip 21, memory 22 (one or more computer-readable storage media), a user interface 23, a display component 24, a camera 25, a sensor 26, a positioning module 27 configured to perform device positioning, and a transceiver 28 configured to perform communication. These components may communicate with each other by using one or more buses 29.

[0176] One or more processors 211, a clock module 212, and a power management module 213 may be integrated into the chip 21. The clock module 212 integrated into the chip 21 is mainly configured to provide the processor 211 with timers necessary for data transmission and timing control. The timers may implement clock functions for data transmission and timing control. The processor 211 can perform operations based on instruction operation codes and timing signals, generate operation control signals, and complete instruction fetch and instruction execution control. The power management module 213 integrated into the chip 21 is mainly configured to provide stable and high-precision voltages to the chip 21 and other components of the electronic device 20.

[0177] The processor 211 may also be referred to as a central processing unit (CPU). The processor 211 may specifically include one or more processing units. For example, the processor 211 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU), and / or similar. Multiple different processing units may be independent components or may be integrated into one or more processors.

[0178] In possible implementations, the processor 211 may include one or more interfaces. These interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, a universal serial bus (USB) port, and / or similar.

[0179] Memory 22 may be connected to the processor 211 by bus 29, or the processor To Sa211 It can be combined and configured to store various software programs and / or groups of instructions. Memory 22 may include high-speed random-access memory (e.g., a cache) or may include non-volatile memory such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 22 may store an operating system such as an embedded operating system such as Android®, Apple® Mobile Platform (iOS®), Microsoft® Window Operating System (Windows®), or UNIX®-based operating system (Linux®). Memory 22 may further store data such as image data, point cloud data, 3D map data, pose data, coordinate system transformation information, and map update information. Memory 22 may further store computer executable program code. Computer executable program code includes instructions such as communication program instructions and related program instructions for a SLAM system. Memory 22 may further store one or more applications, such as virtual scenario applications like AR / VR / MR, map applications, image management applications, and navigation and control applications. Memory 22 may further store a user interface program. The user interface program may, for example, clearly display the contents of an application, such as virtual objects in a virtual scenario like AR / VR / MR, by using a graphical operation interface, present the contents by using a display component 24, and receive control operations performed by the user on the application by using input controls such as menus, dialog boxes, or buttons.

[0180] The user interface 23 may be, for example, a touch panel. The touch panel may detect commands for operations performed by the user on the touch panel. The user interface 23 may be, for example, a keypad, physical buttons, or a mouse.

[0181] The electronic device 20 may include one or more display components 24. The electronic device 20 may also implement display functions by using the display components 24, a graphics processing unit (GPU) and an application processor (AP) in the chip 21, etc. The GPU is a microprocessor for implementing image processing and is connected to the display components 24 and the application processor. The GPU performs mathematical and geometric calculations for graphics rendering. The display component 24 may display interface content output by the electronic device 20, for example, images and videos in virtual scenarios such as AR / VR / MR. The interface content may include the interface of the running application and system-level menus, and specifically may include the following interface elements: input interface elements such as buttons, text input boxes, scrollbars and menus; and output interface elements such as windows, labels, images, videos and animations.

[0182] The display component 24 may be a display panel, a lens (e.g., VR glasses), or a projection screen. The display panel may also be referred to as a display screen, and may be, for example, a touchscreen, a flexible screen, or a curved screen, or may be another optical component. It should be understood that the display screen of the electronic device in the embodiments of this application may be a touchscreen, a flexible screen, a curved screen, or another form of screen. In other words, the display screen of the electronic device has the function of displaying an image, and the specific material and shape of the display screen are not specifically limited.

[0183] For example, when the display component 24 includes a display panel, the display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), MiniLed, MicroLed, Micro-oLed, or a quantum dot light-emitting diode (QLED). In addition, in possible implementations, a touch panel in the user interface 23 may be coupled to the display panel in the display component 24. For example, the touch panel may be located below the display panel and is configured to detect touch pressure acting on the display panel when a user inputs a touch operation (e.g., tap, slide, or touch) by using the display panel, and the display panel is configured to display content.

[0184] Camera 25 may be a monocular, binocular, or depth camera and is configured to acquire images / videos by capturing / recording the environment. The images / videos collected by camera 25 may be used, for example, as input data for a SLAM system, or the images / videos may be displayed using a display component 24.

[0185] In possible implementations, camera 25 may also be considered a sensor. Images collected by camera 25 may be in IMG format or other format types. This is not limited to the embodiments of the present application.

[0186] Sensor 26 may be configured to collect data on changes in the status of the electronic device 20 (e.g., rotation, oscillation, motion, or jitter). The data collected by sensor 26 may be used as input data for the SLAM system. Sensor 26 may include one or more sensors, such as an inertial measurement unit (IMU) and a time of flight (TOF) sensor. The IMU may include sensors such as a gyroscope and an accelerometer. The gyroscope is configured to measure the angular velocity of the electronic device when it is moving, and the accelerometer is configured to measure the acceleration of the electronic device when it is moving. The TOF sensor may include an optical transmitter and a photodetector. The optical transmitter is configured to emit light outward, such as laser light, infrared light, or radar waves. The photodetector is configured to detect reflected light, such as reflected laser light, infrared light, or radar waves.

[0187] It should be noted that sensor 26 may further include many other sensors, such as inertial sensors, barometers, magnetometers, and wheel speedometers. This is not limited to the embodiments of the present application.

[0188] The positioning module 27 is configured to implement the physical positioning of the electronic device 20, for example, to acquire the initial position of the electronic device 20. The positioning module 27 may include one or more of the following: a Wi-Fi positioning module, a Bluetooth positioning module, a base station positioning module, and a satellite positioning module. A global navigation satellite system (GNSS) may be placed within the satellite positioning module to assist in positioning. The GNSS is not limited to the BeiDou system, the global positioning system (GPS) system, the global navigation satellite system (GLONASS) system, and the Galileo navigation satellite system (Galileo) system.

[0189] The transceiver 28 is configured to implement communication between the electronic device 20 and another device (e.g., a server or another electronic device). The transceiver 28 integrates a transmitter and a receiver, which are configured to transmit and receive radio frequency signals, respectively. In a specific implementation, the transceiver 28 may include, but is not limited to, an antenna system, a radio frequency (RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec chip, a subscriber identity module (SIM) card, and a storage medium. In possible implementations, the transceiver 28 may alternatively be implemented on separate chips. The transceiver 28 supports at least one data network communication in at least one data network such as 2G / 3G / 4G / 5G, and / or supports at least one of the following short-range wireless communication methods, namely: Bluetooth (BT) communication, wireless fidelity (Wi-Fi) communication, near-field communication (NFC), infrared (IR) wireless communication, ultra-wideband (UWB) communication, and ZigBee® protocol communication.

[0190] In this embodiment of the present application, the processor 211 executes program code stored in the memory 22 to perform various functional applications and data processing of the electronic device 20.

[0191] Figure 3 is a schematic diagram of the structure of a server 30 according to an embodiment of the present application. As shown in Figure 3, the server 30 may be the server in the embodiment shown in Figure 1. The server 30 includes a processor 301, memory 302 (one or more computer-readable storage media), and transceiver 303. These components can communicate with each other by using one or more buses 304.

[0192] The processor 301 may be one or more CPUs. If the processor 301 is a single CPU, this CPU may be a single-core CPU or a multi-core CPU.

[0193] Memory 302 may be connected to or coupled to processor 301 by bus 304, and is configured to store various program codes and / or multiple groups of instructions and data (e.g., map data and pause data). In specific implementations, memory 302 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or portable read-only memory (Compact Disc Read-Only Memory, CD-ROM).

[0194] The transceiver 303 primarily integrates a receiver and a transmitter. The receiver is configured to receive data (e.g., requests or images) transmitted by an electronic device, and the transmitter is configured to transmit data (e.g., map data or pause data) to an electronic device.

[0195] The server 30 shown in Figure 3 is merely an example provided in this embodiment of the present application, and it should be understood that the server 30 may have more components than those shown in the drawings. This is not limited to the embodiments of the present application.

[0196] In this embodiment of the present invention, the processor 301 executes program code stored in the memory 302 and performs various functional applications and data processing of the server 30.

[0197] As used in the embodiments of this application, the term "coupling" means direct connection or connection by one or more intermediate components or circuits.

[0198] Figure 4a is a schematic diagram of an application scenario according to an embodiment of the present application. As shown in Figure 4a, in the application scenario, the electronic device collects visual information by using sensors and determines the current pose of the electronic device by referring to the visual information and a 3D map from a server.

[0199] The 3D map is provided by the server. Specifically, the server creates the 3D map, then compresses it, and transmits the compressed data to the electronic device. After receiving the compressed data, the electronic device decompresses it to obtain the reconstructed 3D map data, and determines the current pose of the electronic device by referring to the collected visual information and the 3D map. The pose is the positional information of the electronic device and can be an absolute pose in the world coordinate system or a pose relative to a point in the environment.

[0200] In this embodiment of the present invention, the server may pre-create a 3D map, compress the 3D map, and then store the compressed 3D map data locally. In this way, storage space can be saved. In addition, the server may transmit the compressed 3D map data to another device, such as cloud storage.

[0201] 1. The server creates a 3D map, compresses the 3D map to obtain compressed data of the 3D map, and stores the compressed data locally.

[0202] The server compresses the 3D map to save local storage space.

[0203] 2. The electronic device sends a map download request to the server. Map download requests are triggered in two ways.

[0204] (1) The user starts a map application installed on the electronic device, and the application uploads location information obtained based on GPS positioning or Wi-Fi positioning to the server corresponding to the application. The upload operation may trigger a map download request. Since the uploaded content includes location information, the server may perform a preliminary estimation based on the location information and transmit compressed data of a 3D map of the area to which the positioning point indicated by the location information belongs to the electronic device. The extent of the area to which the positioning point indicated by the location information belongs can be predetermined. For example, the area to which the positioning point belongs may be an administrative area (including a county, city, country, or administrative region) at any level in which the positioning point is located, or it may be a circular area centered on the positioning point and using a specified distance as its radius.

[0205] (2) The user starts a map application installed on the electronic device and actively inputs or selects an area on the application. For example, the user actively inputs "xx Business Center" or selects "Street A" from the list "Street A, Street B, and Street C". The user's actions described above may trigger a map download request. Regardless of whether the user inputs or selects a geographic location, the server accordingly transmits compressed data of a 3D map of the geographic location to the electronic device.

[0206] In this embodiment of the present application, it should be understood that, in addition to the two methods described above, another method may be used to trigger a map download request. For example, the electronic device may automatically detect whether the conditions for downloading or initiating a 3D map download have been met, or the electronic device may initiate a 3D map download and request a download of a 3D map of a region from the server when it detects a change in ambient light or environmental changes. The size of the region is not specifically limited.

[0207] 3. The server sends the compressed 3D map data to the electronic device.

[0208] 4. Electronic devices collect visual information.

[0209] Please note that stages 3 and 4 are independent of each other, and the sequence is not limited.

[0210] 5. The electronic device decompresses the compressed data of the 3D map and obtains the reconstructed data of the 3D map.

[0211] 6. The electronic device performs positioning on a 3D map based on visual information and acquires a pose corresponding to the visual information.

[0212] After receiving compressed 3D map data, electronic devices do not need to immediately decompress the data. They only need to decompress the data to obtain reconstructed 3D map data before performing positioning based on visual information. For example, a user can pre-download compressed 3D map data for a given area by downloading "offline mapping," and then decompress the 3D map data only when positioning is required.

[0213] Figure 4b is a schematic diagram of an application scenario according to an embodiment of the present application. As shown in Figure 4b, in the application scenario, the electronic device collects visual information by using sensors, and the server determines the current pose of the electronic device by referring to the visual information from the electronic device and a 3D map.

[0214] The 3D map is provided by the server. Specifically, the server creates the 3D map, then compresses it, and stores the compressed data locally. When visual information is received from the electronic device, the server retrieves the reconstructed 3D map data by decompressing it, and uses the visual information and the 3D map to determine the current pose of the electronic device.

[0215] 1. The server creates a 3D map, compresses the 3D map to obtain compressed data of the 3D map, and stores the compressed data locally.

[0216] 2. Electronic devices collect visual information.

[0217] 3. Electronic devices transmit visual information to the server.

[0218] 4. The server decompresses the compressed 3D map data and obtains the reconstructed 3D map data.

[0219] Please understand that the server saves storage space by compressing the 3D map.

[0220] 5. The server performs positioning on the 3D map based on the visual information and obtains the pose corresponding to the visual information.

[0221] 6. The server sends a pause message to the electronic device.

[0222] Figure 4c is a schematic diagram of an application scenario according to an embodiment of the present application. As shown in Figure 4c, in the application scenario, the electronic device collects visual information by using sensors and determines the current pose of the electronic device by referring to the visual information and a 3D map.

[0223] The 3D map is provided by an electronic device. Specifically, the electronic device creates a 3D map, then compresses the 3D map, and stores the compressed data of the 3D map locally. When visual information is collected, the electronic device obtains the reconstructed data of the 3D map by performing decompression, and determines the current pose of the electronic device by referring to the collected visual information and the 3D map.

[0224] 1. The electronic device creates a 3D map, compresses the 3D map to obtain compressed data of the 3D map, and stores the compressed data locally.

[0225] It's important to understand that electronic devices save storage space by compressing 3D maps.

[0226] 2. Electronic devices collect visual information by using sensors.

[0227] 3. The electronic device decompresses the compressed data of the 3D map and obtains the reconstructed data of the 3D map.

[0228] 4. The electronic device performs positioning on a 3D map based on visual information and acquires a pose corresponding to the visual information.

[0229] Figure 4d is a schematic diagram of an application scenario according to the embodiment of the present application. 4D As shown, in the application scenario, the second electronic device collects visual information by using sensors and determines the current pose of the second electronic device by referring to the visual information and a 3D map from the server.

[0230] The 3D map is created by the first electronic device. Specifically, the first electronic device creates the 3D map, compresses it, and then sends the compressed 3D map data to a server. The server then sends the compressed 3D map data to the second electronic device. The second electronic device obtains the reconstructed 3D map data by performing decompression, and determines the current pose of the second electronic device by referring to the collected visual information and the 3D map.

[0231] In this embodiment of the present invention, the first electronic device can pre-create a 3D map, compress the 3D map, and then transmit the compressed data of the 3D map to a server. In this way, the transmission bandwidth can be reduced.

[0232] 1. The first electronic device creates a 3D map and obtains compressed data of the 3D map by compressing the 3D map.

[0233] 2. The first electronic device sends the compressed 3D map data to the server.

[0234] The first electronic device compresses the 3D map, then transmits the compressed 3D map data, reducing the transmission bandwidth and improving transmission efficiency.

[0235] 3. The second electronic device sends a map download request to the server.

[0236] The second electronic device may send a map download request based on the trigger scheme shown in Figure 4a.

[0237] 4. The server transmits the compressed 3D map data to the second electronic device.

[0238] 5. The second electronic device decompresses the compressed data of the 3D map and obtains the reconstructed data of the 3D map.

[0239] 6. The second electronic device collects visual information by using sensors.

[0240] 7. The second electronic device performs positioning on a 3D map based on visual information and acquires a pose corresponding to the visual information.

[0241] Figure 4e is a schematic diagram of an application scenario according to an embodiment of the present application. As shown in Figure 4c, in the application scenario, the second electronic device collects visual information by using a sensor, and the server determines the current pose of the second electronic device by referring to the visual information from the second electronic device and a 3D map from the first electronic device.

[0242] The 3D map is created by the first electronic device. Specifically, the first electronic device creates the 3D map, compresses it, and then sends the compressed 3D map data to the server. The server obtains the reconstructed 3D map data by decompressing it, and determines the current pose of the second electronic device by referring to the visual information from the second electronic device and the 3D map.

[0243] 1. The first electronic device creates a 3D map and obtains compressed data of the 3D map by compressing the 3D map.

[0244] 2. The first electronic device sends the compressed 3D map data to the server.

[0245] 3. The second electronic device collects visual information by using sensors.

[0246] 4. The second electronic device sends a positioning request to the server, where the positioning request contains visual information.

[0247] 5. The server decompresses the compressed 3D map data and obtains the reconstructed 3D map data.

[0248] 6. The server performs positioning on the 3D map based on visual information and obtains a pose corresponding to the visual information.

[0249] 7. The server transmits the pose obtained through positioning to the second electronic device.

[0250] Figure 4f is a schematic diagram of an application scenario according to an embodiment of the present application. As shown in Figure 4d, in the application scenario, the second electronic device collects visual information by using a sensor and determines the current pose of the second electronic device by referring to the visual information and a 3D map from the first electronic device.

[0251] The 3D map is created by the first electronic device. Specifically, the first electronic device creates the 3D map, compresses the 3D map, and then sends the compressed 3D map data to the second electronic device. The second electronic device obtains the reconstructed 3D map data by performing decompression, and determines the current pose of the second electronic device by referring to the collected visual information and the 3D map from the first electronic device.

[0252] 1. The first electronic device creates a 3D map, compresses the 3D map to obtain compressed data of the 3D map, and stores the compressed data locally.

[0253] 2. The second electronic device sends a map download request to the first electronic device.

[0254] 3. The first electronic device transmits the compressed 3D map data to the second electronic device.

[0255] 4. The second electronic device decompresses the compressed data of the 3D map and obtains the reconstructed data of the 3D map.

[0256] 5. The second electronic device collects visual information by using sensors.

[0257] 6. The second electronic device performs positioning on a 3D map based on visual information and acquires a pose corresponding to the visual information.

[0258] The positioning algorithm used in the embodiments shown in Figures 4a to 4f may include the following:

[0259] (1) The region descriptors to be acquired are extracted from the visual information, and the algorithm used to extract the region descriptors to be acquired is the same as the algorithm used to extract region descriptors from a 3D map.

[0260] (2) The target 3D map points are extracted from the visual information, and the spatial location of the target 3D map points and the target 3D map point descriptors are obtained. Here, the algorithm for extracting the target 3D map point descriptors is the same as the algorithm for extracting 3D map point descriptors from a 3D map.

[0261] (3) Acquisition is performed on multiple region descriptors included in the 3D map data based on the region descriptor to be acquired, and multiple candidate region descriptors are acquired.

[0262] In embodiments of the present application, the distance between each region descriptor in the region descriptor to be acquired and the plurality of region descriptors can be calculated. The distance may include the Hamming distance, Manhattan distance, or Euclidean distance, etc. Next, at least one region descriptor that satisfies a condition (e.g., the distance is less than a threshold) is selected as a candidate region descriptor.

[0263] (4) Matching is performed separately between the 3D map point descriptor to be acquired and the 3D map point descriptors corresponding to the multiple candidate region descriptors. The matching calculates the similarity between the 3D map point descriptor to be acquired and the 3D map point descriptors corresponding to the multiple candidate region descriptors separately and finds the most similar 3D map point.

[0264] (5) The pose of the electronic device is calculated based on the discovered 3D map points by using pose-solving algorithms such as perspective-n-point (PnP) camera pose estimation and efficient perspective-n-point camera pose estimation (EPnP).

[0265] The embodiments shown in Figures 4a to 4f all relate to the compression of 3D maps. Embodiments of the present invention provide multiple device frameworks for performing the aforementioned compression. These multiple device frameworks are described below.

[0266] In any one of the application scenarios in Figures 4a to 4f, positioning is performed based on a 3D map in an embodiment of the present application, and the current pose of the electronic device is obtained. The pose can be applied to fields such as AR navigation, AR human-computer interaction, assisted driving, and autonomous driving. For example, pose-based AR navigation is used as an example. Figure 4g is a schematic diagram of a user interface displayed by an electronic device according to an embodiment of the present application. Based on the pose, the electronic device may display the user interface shown in Figure 4g. The user interface may include navigation arrow instructions for navigation to conference room 2, and the navigation arrow instructions toward conference room 2 may be virtual objects obtained from a server based on the pose or obtained locally based on the pose. The user interface may further include visual information collected by sensors, for example, the building shown in Figure 4g. The user goes to conference room 2 by referring to the user interface of the electronic device shown in Figure 4g.

[0267] It should be noted that the 3D map reconstruction data obtained by decompression in the embodiment of this application may also be referred to as 3D map reconstruction data.

[0268] Figure 5 is a diagram showing the structure of a device 50 for encoding a 3D map according to an embodiment of the present application. As shown in Figure 5, the encoding device 50 may be used in a server or electronic device in the embodiments described above, in particular a device that needs to compress and transmit a 3D map, for example, a server in the embodiment shown in Figure 4a, or a first electronic device in the embodiments shown in Figures 4d to 4f.

[0269] The apparatus 50 for encoding a 3D map in this embodiment of the present application includes a compression module 51 and a transmission module 52. The compression module 51 is configured to compress the data to be encoded of a 3D map to obtain a bitstream of the 3D map, where the 3D map includes a plurality of 3D map points, and the data of the 3D map includes the data of the plurality of 3D map points. The transmission module 52 is configured to transmit the bitstream of the 3D map. It can be seen that the input data of the compression module 51 is the data of the 3D map, the output data is the bitstream of the 3D map, and the data transmitted by the transmission module 52 is the bitstream of the 3D map output by the compression module 51.

[0270] The compression module 51 can perform compression on the 3D map data, reducing the data size of the 3D map. In scenarios where the 3D map needs to be transmitted, transmitting the compressed data of the 3D map instead of the original data of the 3D map can reduce the amount of data to be transmitted, for example, by compressing the data size from TB level to GB level, further reducing the bandwidth occupied by the transmission, thereby improving the transmission efficiency of the 3D map.

[0271] It should be noted that in this embodiment of the present application, compression may include at least one of compactification and prediction, and compactification may include at least one of quantization and binarization.

[0272] Quantization means mapping the data to be processed to one or more quantization exponents, where each quantization exponent corresponds to one quantization center. The number of bits in a quantization exponent is usually significantly smaller than the number of bits in the original data, saving storage or transmission bandwidth. Quantization methods are not limited to but include scalar quantization, vector quantization, and direct product quantization.

[0273] Binarization means processing the data to be processed into a binary sequence represented by binary symbols. Binarization can be, for example, hashing. The principle of hashing is to map the data to be processed into a Hamming space (binary space) and generate a binary hash code. The number of bits in the hash code is usually clearly smaller than the number of bits in the original data, saving storage and transmission bandwidth. In addition, the computational complexity of calculating the Hamming distance between hash codes is usually smaller than that of the Euclidean distance of the original data, thereby reducing the computational complexity. Hashing methods include, but are not limited to, iterative quantization (ITQ) hashing methods, locality sensitive hashing (LSH) methods, etc.

[0274] Prediction means performing a prediction on the data to be processed by using the processed data and obtaining the residual data of the data to be processed. Obviously, the amount of residual data is smaller than the amount of the original data, thereby implementing data compression. The selection of reference data can be agreed in advance. For example, the previously processed data is fixedly used as reference data. In this case, the reference data does not need to be identified in the bit stream. In another example, any processed data is used as reference data. In this case, the identification information of the reference data needs to be written into the bit stream and includes the number of reference data or other information that can be used to infer the reference data.

[0275] In the above description of the 3D map data, it can be found that the sequence of a plurality of 3D map points included in the 3D map is meaningless. Therefore, when compression or encoding of 3D map points is included, the sequence of a plurality of 3D map points is not limited, that is, a plurality of 3D map points can be separately compressed or encoded in any sequence.

[0276] Based on the prediction principle, if the similarity between the predicted data of the data to be processed and the data to be processed is relatively high, the probability that the obtained residual data of the data to be processed is 0 is relatively high. As a result, the compression performance can be improved, and thereby, the data amount for encoding is considered to be reduced. In this embodiment of the present application, a plurality of data to be processed are reordered before prediction to improve the correlation between adjacent data to be processed, and thereby, the data amount of the residual data can be further reduced. Optionally, one data to be processed can be predicted based on one or more processed data to obtain the residual data of the data to be processed; or a plurality of data to be processed can be predicted based on one or more processed data to obtain the residual data of the plurality of data to be processed.

[0277] In addition, compression may further include encapsulation to encapsulate the data to be encoded into a bit stream. Encapsulation may use any encoding algorithm, such as entropy encoding. Entropy encoding is a lossless data compression method. Entropy encoding algorithms include, but are not limited to, Huffman encoding, arithmetic encoding, a compression / decompression algorithm improved based on the LZ77 compression algorithm (Lempel-Ziv-Markov chain-algorithm, LZMA), and a function library algorithm for data compression (zlib).

[0278] Based on the embodiment shown in FIG. 5, the embodiment of the present application provides examples of a plurality of implementable structures of a compression module, and the following embodiments are used for illustration.

[0279] In a possible implementation, FIG. 6a is a structural diagram of an apparatus 60-1 for encoding a 3D map according to an embodiment of the present application. As shown in FIG. 6a, the encoding apparatus 60-1 can use the server or the electronic device in the foregoing embodiment, particularly, a device that needs to compress and transmit a 3D map, such as the server in the embodiment shown in FIG. 4a, or the first electronic device in the embodiments shown in FIGS. 4d to 4f.

[0280] Referring to the embodiment shown in Figure 5, in this embodiment of the present application, the encoding device 60-1 includes a compression module 61-1 and a transmission module 62-1, and the compression module 61-1 includes a compaction module 611-1 and an encapsulation module 613-1. Details are as follows.

[0281] The compactification module 611-1 is configured to perform compactification on the input first data and output the compactified data of the first data, while the encapsulation module 613-1 is configured to process the input third data and output a bitstream of the 3D map.

[0282] The first data is the 3D map data, and the third data is the compactified data of the first data. The input data of the compactification module 611-1 is the 3D map data, and the output data is the compactified data obtained by compactification; and the input data of the encapsulation module 613-1 is the compactified data output by the compactification module 611-1, and the output of the encapsulation module 613-1 is a bitstream of the 3D map obtained based on the compactified data.

[0283] In a possible implementation, Figure 6b shows the structure of a device 60-2 for encoding a 3D map according to an embodiment of the present application. As shown in Figure 6b, the encoding device 60-2 may use a server or electronic device in the embodiments described above, in particular a device that needs to compress and transmit the 3D map, for example, the server in the embodiment shown in Figure 4a, or the first electronic device in the embodiments shown in Figures 4d to 4f.

[0284] Referring to the embodiment shown in Figure 5, in this embodiment of the present application, the encoding device 60-2 includes a compression module 61-2 and a transmission module 62-2, and the compression module 61-2 includes a prediction module 612-1 and an encapsulation module 613-2. Details are as follows.

[0285] The prediction module 612-1 is configured to perform a prediction on the input second data and output residual data for the second data, while the encapsulation module 613-2 is configured to process the input third data and output a bitstream of the 3D map. Optionally, the prediction module 612-1 is further configured to reorder multiple sets of second data to be processed. Note that reordering is an optional function of the prediction module 612-1; that is, the prediction module 612-1 can directly perform a prediction on each of the randomly sorted second data sets, or it can first reorder the randomly sorted second data sets and then perform a prediction on each of the second data sets based on the reordering result.

[0286] The second data set is 3D map data, and the third data set is residual data of the second data set. The input data for prediction module 612-1 is 3D map data, and the output data is residual data obtained by prediction; and the input data for encapsulation module 613-2 is residual data output by prediction module 612-1, and the output of encapsulation module 613-2 is a bitstream of the 3D map obtained based on the residual data.

[0287] In a possible implementation, Figure 6c shows the structure of a device 60-3 for encoding a 3D map according to an embodiment of the present application. As shown in Figure 6c, the encoding device 60-3 may use a server or electronic device in the embodiments described above, in particular a device that needs to compress and transmit the 3D map, for example, the server in the embodiment shown in Figure 4a, or the first electronic device in the embodiments shown in Figures 4d to 4f.

[0288] Referring to the embodiment shown in Figure 5, in this embodiment of the present application, the encoding device 60-3 includes a compression module 61-3 and a transmission module 62-3, and the compression module 61-3 includes a compaction module 611-2, a prediction module 612-2 and an encapsulation module 613-3. Details are as follows.

[0289] The compactification module 611-2 is configured to perform compactification on the input first data and output the compactified data of the first data; the prediction module 612-2 is configured to perform prediction on the input second data and output the residual data of the second data; and the encapsulation module 613-3 is configured to process the input third data and output a bitstream of the 3D map.

[0290] The first data set is 3D map data, the second data set is the compactified data of the first data set, and the third data set is the residual data of the second data set. The input data for the compactification module 611-2 is 3D map data, and the output data is the compactified data obtained by compactification; the input data for the prediction module 612-2 is the compactified data output by the compactification module 611-2, and the output data is the residual data obtained by prediction; and the input data for the encapsulation module 613-3 is the residual data output by the prediction module 612-2, and the output of the encapsulation module 613-3 is a bitstream of the 3D map obtained based on the residual data.

[0291] In this embodiment of the present application, based on the embodiment shown in Figure 6a or Figure 6c, the compactification module 611-1 / 611-2 may include a quantization module and / or a binarization module.

[0292] In a possible implementation, Figure 6d is a structural diagram of a compactification module 611-1 / 611-2 according to an embodiment of the present application. As shown in Figure 6d, the compactification module 611-1 / 611-2 includes a quantization module 6111 and / or a binarization module 6112. The quantization module 6111 is configured to perform quantization on input data and obtain quantized data of the input data, and the binarization module 6112 is configured to perform binarization on input data and obtain binary data of the input data.

[0293] Optionally, the compactification module 611-1 / 611-2 may include only the quantization module 6111. In this case, the input data for the quantization module 6111 may be the input data for the compactification module 611-1 / 611-2, for example, the data of a 3D map. Accordingly, the output data for the quantization module 6111 (i.e., the output data for the compactification module 611-1 / 611-2) is the quantized data of the 3D map.

[0294] Optionally, the compactification module 611-1 / 611-2 may include only the binarization module 6112. In this case, the input data for the binarization module 6112 may be the input data for the compactification module 611-1 / 611-2, for example, the data of a 3D map. Accordingly, the output data for the binarization module 6112 (i.e., the output data for the compactification module 611-1 / 611-2) is the binary data of the 3D map.

[0295] Optionally, the compactification module 611-1 / 611-2 includes the quantization module 6111 and the binarization module 6112. In this case, the input data to the quantization module 6111 may be the input data to the compactification module 611-1 / 611-2, for example, the data of a 3D map. The input data to the binarization module 6112 may also be the input data to the compactification module 611-1 / 611-2, in which case the output data to the compactification module 611-1 / 611-2 includes the quantized data and binary data of the 3D map. Alternatively, the input data to the quantization module 6111 may be the input data to the compactification module 611-1 / 611-2, for example, the data of a 3D map. The input data to the binarization module 6112 may be the output data to the quantization module 6111, i.e., the quantized data of the 3D map. In this case, the output data to the compactification module 611-1 / 611-2 is the binary data of the 3D map.

[0296] As described above regarding 3D map data, 3D map data may include data for multiple region descriptors and multiple 3D map points. Accordingly, in the embodiments shown in Figures 6a to 6d, a separate module in the compression module, distinct from the encapsulation module, may be divided into a first module configured to process region descriptors and a second module configured to process data for 3D map points. For example, the compactification module may include a first compactification module and a second compactification module; the prediction module may include a first prediction module and a second prediction module; the quantization module may include a first quantization module and a second quantization module; and the binarization module may include a first binarization module and a second binarization module. The difference between the first and second modules obtained after the division from the above embodiments is that the input data for the first module and the input data for the second module correspond to data for region descriptors and 3D map points, respectively.

[0297] Figure 6e is a diagram showing the structure of a device 60-4 for encoding a 3D map according to an embodiment of the present application. As shown in Figure 6e, the encoding device 60-4 may use a server or electronic device in the embodiments described above, in particular a device that needs to compress and transmit the 3D map, for example, the server in the embodiment shown in Figure 4a, or the first electronic device in the embodiments shown in Figures 4d to 4f.

[0298] Referring to the embodiment shown in FIG. 6c, in this embodiment of the present application, the encoding device 60-4 includes a compression module 61-4 and a transmission module 62-4. The compression module 61-4 includes a compacting module 611-3, a prediction module 612-3, and a encapsulation module 613-4. The compacting module 611-3 includes a first quantization module 6111a, a first binarization module 6112a, a second quantization module 6111b, and a second binarization module 6112b. The prediction module 612-3 includes a first prediction module 612a and a second prediction module 612b, and the first prediction module 612a and the second prediction module 612b can be further configured to implement an order reset function.

[0299] The input data of the first quantization module 6111a is a region descriptor, and the output data is the quantized data of the region descriptor. The input data of the first binarization module 6112a is a region descriptor, and the output data is the binary data of the region descriptor. The input data of the first prediction module 612a includes the quantized data of the region descriptor and the binary data of the region descriptor, and the output data is the residual data of the region descriptor. The input data of the second quantization module 6111b is the data of the 3D map point, and the output data is the quantized data of the 3D map point. The input data of the second binarization module 6112b is the data of the 3D map point, and the output data is the binary data of the 3D map point. The input data of the second prediction module 612b includes the quantized data of the 3D map point and the binary data of the 3D map point, and the output data is the residual data of the 3D map point.

[0300] Based on this, the input data of the encapsulation module 613-4 includes the residual data of the region descriptor and the residual data of the 3D map point. The encapsulation module 613-4 performs encapsulation separately on the residual data of the region descriptor and the residual data of the 3D map point to obtain the bitstream of the 3D map.

[0301] The transmission module 62-4 is configured to transmit a bitstream of the 3D map.

[0302] Figure 6e shows an exemplary structure of an apparatus for encoding a 3D map, and it should be noted that this structure is derived based on the content contained in the 3D map data. However, the structure does not constitute a limitation on the encoding apparatus. The encoding apparatus may include a wider variety of modules than those in the embodiment shown in Figure 6e. For example, referring to the embodiment shown in Figure 6a or Figure 6b, another structure may be derived based on the content contained in the 3D map data. In the embodiment shown in Figure 6d, the quantization module and the binarization module are in an "and / or" relationship, i.e., the compression module may include either the quantization module or the binarization module, or it may include both the quantization module and the binarization module. When a distinction is made between the first and second modules, the processing methods may be set independently for the module for processing region descriptors and the module for processing 3D map point data, and do not need to be exactly the same. For example, region descriptors may be processed using the first quantization module, the first binarization module and the first prediction module, and 3D map point data may be processed using the second quantization module and the second prediction module. In the embodiments shown in Figures 6a to 6c, the compactification module and the prediction module are in an "and / or" relationship, that is, the compression module may include either the compactification module or the prediction module, or may include both the compactification module and the prediction module. When a distinction is made between the first module and the second module, the processing methods may be set independently for the module for processing region descriptors and the module for processing 3D map point data, and do not need to be exactly the same. For example, region descriptors may be processed using the first compactification module, and 3D map point data may be processed using the second prediction module. Specific implementations of the compression module are not specifically limited herein.

[0303] As described above regarding 3D map data, 3D map data may include multiple region descriptors, multiple 3D map point descriptors, and multiple 3D map point spatial locations. Therefore, in the embodiments shown in Figures 6a to 6d, a module in the compression module that is different from the encapsulation module may be divided into a first module configured to process region descriptors, a second module configured to process 3D map point descriptors, and a third module configured to process 3D map point spatial locations. For example, the compactification module may include a first compactification module, a second compactification module, and a third compactification module. The prediction module may include a first prediction module, a second prediction module, and a third prediction module. The quantization module may include a first quantization module, a second quantization module, and a third quantization module. The binarization module may include a first binarization module, a second binarization module, and a third binarization module. The difference between the first, second, and third modules obtained after the classification from the aforementioned embodiment is that the input data of the first module, the input data of the second module, and the input data of the third module correspond to a region descriptor, a 3D map point descriptor, and a 3D map point spatial position, respectively.

[0304] Figure 6f is a diagram showing the structure of a device 60-5 for encoding a 3D map according to an embodiment of the present application. As shown in Figure 6f, the encoding device 60-5 may use a server or electronic device in the embodiments described above, in particular a device that needs to compress and transmit the 3D map, for example, the server in the embodiment shown in Figure 4a, or the first electronic device in the embodiments shown in Figures 4d to 4f.

[0305] Referring to the embodiment shown in Figure 6c, in this embodiment of the present application, the encoding device 60-5 includes a compression module 61-5 and a transmission module 62-5, the compression module 61-5 includes a compactification module 611-4, a prediction module 612-4 and an encapsulation module 613-5. The compactification module 611-4 includes a first quantization module 6111a, a first binarization module 6112a, a second quantization module 6111b, a second binarization module 6112b, a third quantization module 6111c and a third binarization module 6112c. The prediction module 612-4 includes a first prediction module 612a, a second prediction module 612b and a third prediction module 612c. The first prediction module 612a, the second prediction module 612b and the third prediction module 612c may be further configured to implement a reordering function.

[0306] The input data for the first quantization module 6111a is a region descriptor, and the output data is the quantized data of the region descriptor. The input data for the first binarization module 6112a is a region descriptor, and the output data is the binary data of the region descriptor. The input data for the first prediction module 612a includes the quantized data of the region descriptor and the binary data of the region descriptor, and the output data is the residual data of the region descriptor. The input data for the second quantization module 6111b is a 3D map point descriptor, and the output data is the quantized data of the 3D map point descriptor. The input data for the second binarization module 6112b is a 3D map point descriptor, and the output data is the binary data of the 3D map point descriptor. The input data for the second prediction module 612b includes the quantized data of the 3D map point descriptor and the binary data of the 3D map point descriptor, and the output data is the residual data of the 3D map point descriptor. The input data for the third quantization module 6111c is the 3D map point spatial position, and the output data is the quantized data of the 3D map point spatial position. Third binarization module Ru6112c The input data is the spatial position of 3D map points, and the output data is binary data of the spatial position of 3D map points. Third prediction module Ru612cThe input data includes quantized data and binary data of the spatial positions of 3D map points, and the output data is residual data of the spatial positions of 3D map points.

[0307] Based on this, the input data for the encapsulation module 613-5 includes residual data for region descriptors, residual data for 3D map point descriptors, and residual data for the spatial locations of 3D map points. The encapsulation module 613-5 performs encapsulation separately on the residual data for region descriptors, residual data for 3D map point descriptors, and residual data for the spatial locations of 3D map points to obtain a bitstream of the 3D map.

[0308] The transmission module 62-5 is configured to transmit a bitstream of the 3D map.

[0309] Figure 6f shows an exemplary structure of a device 60 for encoding a 3D map, and it should be noted that this structure is obtained based on the content contained in the 3D map data. However, the structure does not constitute a limitation on the encoding device 60. The encoding device 60 may include a wider variety of modules than those in the embodiment shown in Figure 6f. For example, referring to the embodiment shown in Figure 6a or Figure 6b, another structure may be obtained based on the content contained in the 3D map data. In the embodiment shown in Figure 6d, the quantization module and the binarization module are in an "and / or" relationship, i.e., the compression module may include either the quantization module or the binarization module, or both the quantization module and the binarization module. When a distinction is made between the first module, the second module and the third module, the processing method may independently configure a module for processing region descriptors, a module for processing 3D map point descriptors, and a module for processing 3D map point spatial locations, and they do not need to be exactly the same. In the embodiments shown in Figures 6a to 6c, the compactification module and the prediction module are in an "and / or" relationship, that is, the compression module may include either the compactification module or the prediction module, or it may include both the compactification module and the prediction module. When a distinction is made between the first module, the second module and the third module, the processing method may independently configure a module for processing region descriptors, a module for processing 3D map point descriptors, and a module for processing 3D map point spatial locations, and they do not need to be exactly the same.

[0310] As described above regarding 3D map data, 3D map data may include data for multiple region descriptors and multiple 3D map points.

[0311] In a possible implementation, Figure 7a shows the structure of a device 70-1 for encoding a 3D map according to an embodiment of the present application. As shown in Figure 7a, the encoding device 70-1 may use a server or electronic device in the embodiments described above, in particular a device that needs to compress and transmit the 3D map, for example, a server in the embodiment shown in Figure 4a, or a first electronic device in the embodiments shown in Figures 4d to 4f.

[0312] Referring to the embodiment shown in Figure 5, in this embodiment of the present application, the encoding device 70-1 includes a compression module 71-1 and a transmission module 72-1, wherein the compression module 71-1 includes a first compression submodule 711-1 and a second compression submodule 712-1. Further details are as follows.

[0313] The first compression submodule 711-1 is configured to perform compression on the input 21st data and output a bitstream of the 21st data. The second compression submodule 712-1 is configured to perform compression on the input 22nd data and output a bitstream of the 22nd data.

[0314] Data 21 is one of several region descriptors, and data 22 is data for one of several 3D map points. It can be seen that the input data for the first compression submodule 711-1 is a region descriptor, and the output data is a bitstream of the region descriptors. The input data for the second compression submodule 712-1 is data for a 3D map point, and the output data is a bitstream of the 3D map point. The 3D map bitstream includes the bitstreams of the multiple region descriptors and the bitstreams of the multiple 3D map points.

[0315] For the first compression submodule 711-1 and the second compression submodule 712-1, please refer to the structure of the compression module in the embodiment shown in Figures 6a to 6f. Note that the first compression submodule 711-1 and the second compression submodule 712-1 are independent of each other and may use the same structure or different structures. That is, the first compression submodule 711-1 configured to process region descriptors and the second compression submodule 712-1 configured to process 3D map point data may have the same structure or different structures. Accordingly, the compression stages performed on the region descriptors may be the same as or different from the compression stages performed on the 3D map point data.

[0316] For example, the first compression submodule includes a first compactification module and a first encapsulation module. Thus, after a region descriptor is input to the first compression submodule, the first compactification module first processes the region descriptor and obtains the compacted data of the region descriptor, and then the first encapsulation module processes the compacted data and obtains the bitstream of the region descriptor. The second compression submodule includes a second compactification module, a second prediction module, and a second encapsulation module. Thus, after data of 3D map points is input to the second compression submodule, the second compactification module first processes the data of the 3D map points and obtains the compacted data of the 3D map points, and then the second prediction module processes the compacted data and obtains the residual data of the 3D map points, and then the second encapsulation module processes the residual data and obtains the bitstream of the 3D map points.

[0317] In another example, the first compression submodule includes a first quantization module, a first binarization module, a first prediction module, and a first encapsulation module. Thus, after the region descriptor is input to the first compression submodule, the first quantization module processes the region descriptor and obtains the quantized data of the region descriptor, the first binarization module processes the region descriptor and obtains the binary data of the region descriptor, then the first prediction module processes the quantized and binary data and obtains the residual data of the region descriptor, then the first encapsulation module processes the residual data and obtains the bitstream of the region descriptor. The second compression submodule includes a second quantization module, a second prediction module, and a second encapsulation module. Thus, after the 3D map point data is input to the second compression submodule, the second quantization module first processes the 3D map point data and obtains the quantized data of the 3D map points, then the second prediction module processes the quantized data and obtains the residual data of the 3D map points, and then the second encapsulation module processes the residual data and obtains the bitstream of the 3D map points.

[0318] It should be understood that the structures of the first and second compression submodules are described above as examples. However, this does not constitute a limitation on the structures of the first and second compression submodules. The two submodules may include more or fewer modules than those in the examples. For details, please refer to the structures of the compression modules in the embodiments shown in Figures 6a to 6f. This is not limited to the embodiments of the present application.

[0319] In a possible implementation, Figure 7b shows the structure of a device 70-2 for encoding a 3D map according to an embodiment of the present application. As shown in Figure 7b, the encoding device 70-2 may use a server or electronic device in the embodiments described above, in particular a device that needs to compress and transmit the 3D map, for example, a server in the embodiment shown in Figure 4a, or a first electronic device in the embodiments shown in Figures 4d to 4f.

[0320] Referring to the embodiment shown in Figure 7a, in this embodiment of the present application, the encoding device 70-2 includes a compression module 71-2 and a transmission module 72-2. The compression module 71-2 includes a first compression submodule 711-2 and a second compression submodule 712-2. The first compression submodule 711-2 includes a first quantization module 7111, a first binarization module 7112, a first prediction module 7113, and a first encapsulation module 7114. The second compression submodule 712-2 includes a second quantization module 7121, a second binarization module 7122, a second prediction module 7123, and a second encapsulation module 7124. Further details are as follows.

[0321] The first quantization module 7111 is configured to perform quantization on the input region descriptor and obtain the quantized data of the region descriptor. The first binarization module 7112 is configured to perform binarization on the input quantized data and obtain the binary data of the region descriptor. The first prediction module 7113 is configured to perform prediction on the input binary data and obtain the residual data of the region descriptor. The first encapsulation module 7114 is configured to encapsulate the input residual data and obtain the bitstream of the region descriptor. The second quantization module 7121 is configured to perform quantization on the input data of the 3D map point and obtain the quantized data of the 3D map point. The second binarization module 7122 is configured to perform binarization on the input quantized data and obtain the binary data of the 3D map point. The second prediction module 7123 is configured to perform prediction on the input binary data and obtain the residual data of the 3D map point. The second encapsulation module 7124 is configured to encapsulate the input residual data and obtain a bitstream of 3D map points.

[0322] It should be understood that the structures of the first and second compression submodules are described as examples in the embodiment shown in Figure 7b. However, this does not constitute a limitation on the structures of the first and second compression submodules. The two submodules may include more or fewer modules than those in the example. For details, please refer to the structure of the compression module 61 in the embodiments shown in Figures 6a to 6f. This is not specifically limited to the embodiments of the present application.

[0323] As described above regarding 3D map data, 3D map data may include multiple region descriptors, multiple 3D map point descriptors, and spatial locations.

[0324] In a possible implementation, Figure 8a shows the structure of an apparatus 80-1 for encoding a 3D map according to an embodiment of the present application. As shown in Figure 8a, the encoding apparatus 80-1 may use a server or electronic device in the embodiments described above, in particular a device that needs to compress and transmit the 3D map, for example, a server in the embodiment shown in Figure 4a, or a first electronic device in the embodiments shown in Figures 4d to 4f.

[0325] Referring to the embodiment shown in Figure 5, in this embodiment of the present application, the encoding device 80-1 includes a compression module 81-1 and a transmission module 82-1. The compression module 81-1 includes a first compression submodule 811-1, a second compression submodule 812-1, and a third compression submodule 813-1. Details are as follows.

[0326] The first compression submodule 811-1 is configured to compress the input 29th data and output a bitstream of the 29th data. The second compression submodule 812-1 is configured to compress the input 30th data and output a bitstream of the 30th data. The third compression submodule 813-1 is configured to compress the input 31st data and output a bitstream of the 31st data.

[0327] Data 29 is one of several region descriptors, data 30 is a 3D map point descriptor of one of several 3D map points, and data 31 is a spatial location of one of several 3D map points. It can be seen that the input data of the first compression submodule 811-1 is a region descriptor and the output data is a bitstream of region descriptors; the input data of the second compression submodule 812-1 is a 3D map point descriptor and the output data is a bitstream of 3D map point descriptors; and the input data of the third compression submodule 813-1 is a 3D map point spatial location and the output data is a bitstream of 3D map point spatial locations. The 3D map bitstream includes bitstreams of multiple region descriptors, bitstreams of multiple 3D map point descriptors, and bitstreams of multiple 3D map point spatial locations.

[0328] In a possible implementation, Figure 8b shows the structure of an apparatus 80-2 for encoding a 3D map according to an embodiment of the present application. As shown in Figure 8b, the encoding apparatus 80-2 may use a server or electronic device in the embodiments described above, in particular a device that needs to compress and transmit the 3D map, for example, the server in the embodiment shown in Figure 4a, or the first electronic device in the embodiments shown in Figures 4d to 4f.

[0329] Referring to the embodiment shown in Figure 8a, in this embodiment of the present application, the encoding device 80-2 includes a compression module 81-2 and a transmission module 82-2. The compression module 81-2 includes a first compression submodule 811-2, a second compression submodule 812-2, and a third compression submodule 813-2. The first compression submodule 811-2 includes a first quantization module 8111, a first binarization module 8112, a first prediction module 8113, and a first encapsulation module 8114. The second compression submodule 812-2 includes a second quantization module 8121, a second binarization module 8122, a second prediction module 8123, and a second encapsulation module 8124. The third compression submodule 813-2 includes a third quantization module 8131, a third binarization module 8132, a third prediction module 8133, and a third encapsulation module 8134. Further details are as follows.

[0330] The first quantization module 8111 is configured to perform quantization on the input region descriptor and obtain the quantized data of the region descriptor. The first binarization module 8112 is configured to perform binarization on the input quantized data and obtain the binary data of the region descriptor. The first prediction module 8113 is configured to perform prediction on the input binary data and obtain the residual data of the region descriptor. The first encapsulation module 8114 is configured to encapsulate the input residual data and obtain the bitstream of the region descriptor. The second quantization module 8121 is configured to perform quantization on the input 3D map point descriptor and obtain the quantized data of the 3D map point descriptor. The second binarization module 8122 is configured to perform binarization on the input quantized data and obtain the binary data of the 3D map point descriptor. The second prediction module 8123 is configured to perform prediction on the input binary data and obtain the residual data of the 3D map point descriptor. The second encapsulation module 8124 is configured to encapsulate the input residual data and obtain a bitstream of 3D map point descriptors. The third quantization module 8131 is configured to perform quantization on the input 3D map point spatial locations and obtain quantized data of the 3D map point spatial locations. The third binarization module 8132 is configured to perform binarization on the input quantized data and obtain binary data of the 3D map point spatial locations. The third prediction module 8133 is configured to perform prediction on the input binary data and obtain residual data of the 3D map point spatial locations. The third encapsulation module 8134 is configured to encapsulate the input residual data and obtain a bitstream of the 3D map point spatial locations.

[0331] It should be understood that the structures of the first, second, and third compression submodules are described as examples in the embodiment shown in Figure 8b. However, this does not constitute a limitation on the structures of the first, second, and third compression submodules. The three submodules may include more or fewer modules than those in the example. For details, please refer to the structure of the compression module 61 in the embodiment shown in Figures 6a to 6f. This is not specifically limited to the embodiments of the present application.

[0332] In this embodiment of the present application, a reordering function may be implemented in the prediction module described above, that is, multiple data to be processed may be reordered before prediction, improving the correlation between adjacent data to be processed, thereby further reducing the amount of residual data. The data to be processed includes at least one of a region descriptor, 3D map point data, 3D map point descriptor, or 3D map point spatial location.

[0333] Figure 9 is a flowchart of process 900 of a method for encoding a 3D map according to an embodiment of the present application. As shown in Figure 9, process 900 may be performed by the encoding device in the embodiments described above. Process 900 is described as a series of steps or operations. It should be understood that the steps or operations of process 900 may be performed in various sequences and / or simultaneously and are not limited to the execution sequence shown in Figure 9. Assume that the data of the 3D map is compressed in the encoding device to obtain a bitstream of the 3D map, and then the bitstream of the 3D map is transmitted by the encoding device and process 900 is performed, including the following steps, to process the data of the 3D map currently being processed.

[0334] Step 901: Compress the 3D map data to obtain the 3D map bitstream.

[0335] For details regarding 3D maps and 3D map data, please refer to the above description. Further details will not be provided again in this specification.

[0336] In this embodiment of the present application, the compression performed on the 3D map may include compactification and / or prediction, and encapsulation, and the compactification may include quantization and / or binarization. For details of the aforementioned processes, please refer to the description in the above embodiments. Further details are not described again herein.

[0337] Stage 902: Transmit the bitstream of the 3D map.

[0338] The encoding device can transmit the bitstream of the 3D map to an external source using a communication link.

[0339] Figure 10 is a diagram showing the structure of an apparatus 100 for encoding a 3D map according to an embodiment of the present application. As shown in Figure 10, the encoding apparatus 100 may be used in a server or electronic device in the embodiments described above, in particular a device that needs to compress and store a 3D map, for example, a server in the embodiment shown in Figure 4b, or an electronic device in the embodiment shown in Figure 4c.

[0340] The apparatus 100 for encoding a 3D map in this embodiment of the present application includes an encoder 101 and a memory 102. Encoder 1 01 is configured to compress the 3D map data to obtain the 3D map in a compressed and bitstream format, where the 3D map includes multiple 3D map points, and the 3D map data includes the data of multiple 3D map points. Ri102 isIt is configured to store a 3D map acquired by compression and in bitstream format. It can be seen that the input data of the compressor 101 is 3D map data, the output data is a 3D map acquired by compression and in bitstream format, and the input data of the memory 101 is a 3D map acquired by compression and output by the compressor 101 and in bitstream format.

[0341] The compressor 101 can perform compression on the 3D map data, reducing the amount of data in the 3D map. In scenarios where the 3D map needs to be stored, storing the compressed data of the 3D map instead of the original data of the 3D map can reduce the space occupied by the storage of the 3D map data.

[0342] In this embodiment of the present application, the compressor 101 is shown with reference to the compression module in the embodiment shown in Figures 5 to 8b. The difference is that the compressor does not include an encapsulation module.

[0343] In the implementation process, the steps in the embodiments of the method described above may be completed by an integrated logic circuit in the form of hardware within a processor or by instructions in the form of software. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or another programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc. The steps of the method disclosed in the embodiments of this application may be presented directly as being performed and completed by a hardware coding processor, or may be performed and completed by a combination of hardware and software modules within the coding processor. The software modules may be located in mature storage media of the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage media is located in memory. The processor reads the information from memory and, in combination with the processor hardware, completes the steps of the method described above.

[0344] The memory mentioned in the embodiments described above may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM) used as an external buffer. Rather than providing a restrictive description, many forms of RAM may be used through examples, such as static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM), and direct rambus dynamic random access memory (direct rambus RAM, DR RAM). Note that the memory in the systems and methods described herein is not limited to these and any other suitable type of memory.

[0345] Those skilled in the art will recognize, in combination with the examples described in the embodiments disclosed herein, that the unit and algorithmic stages can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software will depend on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementations should not be considered to be beyond the scope of the application.

[0346] For the purpose of providing a simple and concise explanation, it will be readily apparent to those skilled in the art that the detailed operation of the aforementioned systems, apparatus, and units can be described by referring to the corresponding processes in the embodiments of the methods described above. Further details are not described again herein.

[0347] It should be understood that in some embodiments provided herein, the disclosed systems, devices, and methods may be implemented in other ways. For example, the embodiments of the devices described above are merely examples. For example, the division of units is merely a logical functional division, and other divisions may be used in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some functions may be ignored or not performed. In addition, the mutual coupling, direct coupling, or communication connection shown or discussed may be implemented by using some interfaces. Indirect coupling or communication connection between devices or units may be implemented in electronic, mechanical, or other forms.

[0348] Units described as separate parts may or may not be physically separate, and parts shown as units may or may not be physical units, and may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to the actual requirements in order to achieve the objectives of the solution in the embodiment.

[0349] In addition, the functional units in the embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically in isolation, and two or more units may be integrated into a single unit.

[0350] When these functions are implemented in the form of software function units and sold or used as independent products, such functions may be stored on a computer-readable storage medium. Based on this understanding, the technical solutions of the present application, either in essence, in part with respect to the prior art, or in part with respect to the technical solutions, may be implemented in the form of a software product. A computer software product is stored on a storage medium and contains several instructions for instructing a computer device (such as a personal computer, server, or network device) to perform all or some of the steps of the method described in the embodiments of the present application. The storage medium includes any medium capable of storing program code, such as a USB flash drive, removable hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0351] The foregoing description merely represents a specific implementation of the present application, and the scope of protection of the present application is not limited thereto. Any modifications or substitutions that are readily conceivable to a person skilled in the art within the scope of the technical scope disclosed herein shall be included in the scope of protection of the present application. Accordingly, the scope of protection of the present application shall be subject to the scope of protection of the claims. 。 [Other possible items] [Item 1] A codec system for a 3D map comprising an encoding device and a decoding device, wherein the encoding device is communicatively connected to the decoding device; The encoding device is configured to: compress the data of the 3D map to obtain a bitstream of the 3D map, and transmit the bitstream of the 3D map to the decoding device, wherein the 3D map includes a plurality of 3D map points, and the data of the 3D map includes the data of the plurality of 3D map points; The decoding device is configured to receive the bitstream of the 3D map and to decompress the bitstream of the 3D map to obtain reconstructed data of the 3D map. system. [Item 2] The encoding device is a cloud server, and the decoding device is an electronic device; or the encoding device is a first electronic device, and the decoding device is a second electronic device; The decoding device is further configured to send a 3D map download request to the decoding device, wherein the 3D map download request includes location information; The system according to item 1, wherein the encoding device is further configured to receive the 3D map download request and to transmit to the decoding device a bitstream of the 3D map corresponding to the position indication information. [Item 3] The encoding device is an electronic device, and the decoding device is a cloud server; The system according to item 1 or 2, wherein the encoding device is specifically configured to transmit the bitstream of the 3D map to the decoding device after the 3D map has been created. [Item 4] The data of the 3D map further includes a plurality of region descriptors, any one of which describes the characteristics of some or all of the plurality of 3D map points, according to any one of items 1 to 3. [Item 5] The system described in item 1 or 4, wherein the data for any one of the plurality of 3D map points includes a 3D map point descriptor and a 3D map point spatial location. [Item 6] The encoding device is further configured to create the 3D map, according to any one of items 1 to 5. [Item 7] The decoding device is further configured to perform positioning based on the 3D map, according to any one of items 1 to 6. [Item 8] A device for encoding a 3D map, comprising a compression module and a transmission module, The compression module is configured to compress the data of a 3D map to obtain a bitstream of the 3D map, wherein the 3D map includes a plurality of 3D map points, and the data of the 3D map includes the data of the plurality of 3D map points; The transmission module is configured to transmit the bitstream of the 3D map, in the apparatus. [Item 9] The apparatus according to item 8, wherein the data of the 3D map further includes a plurality of region descriptors, any one of the plurality of region descriptors describing the characteristics of some or all of the plurality of 3D map points. [Item 10] The apparatus according to item 8 or 9, wherein the data of any one of the plurality of 3D map points includes a 3D map point descriptor and a 3D map point spatial location. [Item 11] The apparatus for encoding the 3D map is a cloud server or an electronic device; The transmission module is further configured to: receive a 3D map download request, wherein the 3D map download request includes location information; and transmit a bitstream of the 3D map corresponding to the location information in accordance with the 3D map download request. The apparatus described in any one of items 8 through 10. [Item 12] The apparatus for encoding a 3D map is an electronic device; The apparatus according to any one of items 8 to 10, wherein the transmission module is specifically configured to transmit the bitstream of the 3D map after the 3D map has been created. [Item 13] The compression module includes a compactification module and / or a prediction module, and an encapsulation module; The compactification module is configured to perform compactification on the input first data and output the compactified data of the first data; The prediction module is configured to perform a prediction on the input second data and output residual data for the second data; The encapsulation module is configured to process the input third data and output the bitstream of the 3D map; The apparatus according to any one of items 8 to 12, wherein the first data is the data of the 3D map, the second data is the compactified data of the data of the 3D map or the first data, and the third data is the compactified data of the first data or the residual data of the second data. [Item 14] The apparatus according to item 13, wherein the prediction module is further configured to: rearrange a plurality of the second data, and, based on the results of the rearrangement, perform a prediction on at least a portion of the second data and obtain residual data on at least a portion of the second data. [Item 15] The compactification module includes a quantization module and / or a binarization module; The quantization module is configured to perform quantization on the input fourth data and output the quantized data of the fourth data; The binarization module is configured to perform binarization on the input fifth data and output the binary data of the fifth data, where The apparatus according to item 13 or 14, wherein the fourth data is the data of the 3D map, and the fifth data is the quantized data of the data of the 3D map or the fourth data. [Item 16] The quantization module includes a first quantization module and a second quantization module, and / or the binarization module includes a first binarization module and a second binarization module, and the prediction module includes a first prediction module and a second prediction module; The first quantization module is configured to perform quantization on the input sixth data and output the quantized data of the sixth data; The first binarization module is configured to perform binarization on the input seventh data and output the binary data of the seventh data; The first prediction module is configured to perform a prediction on the input eighth data and output residual data for the eighth data; The sixth data is one of the plurality of region descriptors; the seventh data is one of the plurality of region descriptors or the quantized data of the sixth data; and the eighth data is one of the plurality of region descriptors, the quantized data of the sixth data, or the binary data of the seventh data; The second quantization module is configured to perform quantization on the input ninth data and output the quantized data of the ninth data; The second binarization module is configured to perform binarization on the input tenth data and output the binary data of the tenth data; The first prediction module is configured to perform a prediction on the input 11th data and output residual data for the 11th data; The apparatus according to item 15, wherein the 9th data is data of one of the plurality of 3D map points, the 10th data is data of one of the plurality of 3D map points or the quantized data of the 9th data, and the 11th data is data of one of the plurality of 3D map points, the quantized data of the 9th data, or the binary data of the 10th data. [Item 17] The quantization module includes a first quantization module, a second quantization module, and a third quantization module, and / or the binarization module includes a first binarization module, a second binarization module, and a third binarization module, and the prediction module includes a first prediction module, a second prediction module, and a third prediction module; The first quantization module is configured to perform quantization on the input 12th data and output the quantized data of the 12th data; The first binarization module is configured to perform binarization on the input 13th data and output the binary data of the 13th data; The first prediction module is configured to perform a prediction on the input 14th data and output residual data for the 14th data; The 12th data is one of the plurality of region descriptors, the 13th data is one of the plurality of region descriptors or the quantized data of the 12th data, and the 14th data is one of the plurality of region descriptors, the quantized data of the 12th data, or the binary data of the 13th data; The second quantization module is configured to perform quantization on the input 15th data and output the quantized data of the 15th data; The second binarization module is configured to perform binarization on the input 16th data and output the binary data of the 16th data; The second prediction module is configured to perform a prediction on the input 17th data and output residual data for the 17th data; The 15th data is a 3D map point descriptor for one of the plurality of 3D map points; the 16th data is the 3D map point descriptor for one of the plurality of 3D map points or the quantized data of the 15th data; and the 17th data is the 3D map point descriptor for one of the plurality of 3D map points, the quantized data of the 15th data, or the binary data of the 16th data; The third quantization module is configured to perform quantization on the input 18th data and output the quantized data of the 18th data; The third binarization module is configured to perform binarization on the input 19th data and output the binary data of the 19th data; The third prediction module is configured to perform a prediction on the input 20th data and output residual data for the 20th data; The apparatus according to item 15, wherein the 18th data is the spatial position of one of the plurality of 3D map points, the 19th data is the spatial position of one of the plurality of 3D map points or the quantized data of the 18th data, and the 20th data is the spatial position of one of the plurality of 3D map points, the quantized data of the 18th data, or the binary data of the 19th data. [Item 18] The compression module includes a first compression submodule and a second compression submodule; The first compression submodule is configured to compress the input 21 data and output a bitstream of the 21 data; The second compression submodule is configured to compress the input 22 data and output a bitstream of the 22 data; The apparatus according to any one of items 8 to 12, wherein the 21st data is one of the plurality of region descriptors, and the 22nd data is data of one of the plurality of 3D map points. [Item 19] The first compression submodule includes a first compactification module and / or a first prediction module, and a first encapsulation module; the second compression submodule includes a second compactification module and / or a second prediction module, and a second encapsulation module; The first compactification module is configured to perform compactification on the input 23 data and output the compactified data of the 23 data; The first prediction module is configured to perform a prediction on the input 24 data points and output residual data for the 24 data points; The first encapsulation module is configured to process the input 25th data and output a bitstream of the 25th data; The 23rd data is one of the plurality of region descriptors; the 24th data is one of the plurality of region descriptors or the compacted data of the 23rd data; and the 25th data is the compacted data of the 23rd data or the residual data of the 24th data; The second compactification module is configured to perform compactification on the input 26th data and output the compactified data of the 26th data; The second prediction module is configured to perform a prediction on the input 27th data and output residual data for the 27th data; The second encapsulation module is configured to process the input 28th data and output a bitstream of the 28th data; The apparatus according to item 18, wherein the 26th data is data of one of the plurality of 3D map points, the 27th data is the data of one of the plurality of 3D map points or the compacted data of the 26th data, and the 28th data is the compacted data of the 26th data or the residual data of the 27th data. [Item 20] The first compactification module includes a first quantization module and / or a first binarization module, and the second compactification module includes a second quantization module and / or a second binarization module; The first quantization module is configured to perform quantization on the input 29th data and output the quantized data of the 29th data; The first binarization module is configured to perform binarization on the input 30th data and output the binary data of the 30th data; The 29th data is one of the plurality of region descriptors, and the 30th data is one of the plurality of region descriptors or the quantized data of the 29th data; The second quantization module is configured to perform quantization on the input 31st data and output the quantized data of the 31st data; The second binarization module is configured to perform binarization on the input 32 data and output the binary data of the 32 data; The apparatus according to item 19, wherein the 31st data is one of the plurality of 3D map points, and the 30th data is either one of the plurality of 3D map points or the quantized data of the 31st data. [Item 21] The compression module includes a first compression submodule, a second compression submodule, and a third compression submodule; The first compression submodule is configured to compress the input 33 data and output a bitstream of the 33 data; The second compression submodule is configured to compress the input 34 data and output a bitstream of the 34 data; The third compression submodule is configured to compress the input 35th data and output a bitstream of the 35th data; The apparatus according to any one of items 8 to 12, wherein the 33rd data is one of the plurality of region descriptors, the 34th data is a 3D map point descriptor of one of the plurality of 3D map points, and the 35th data is the spatial position of one of the plurality of 3D map points. [Item 22] The first compression submodule includes a first compactification module and / or a first prediction module, and a first encapsulation module; the second compression submodule includes a second compactification module and / or a second prediction module, and a second encapsulation module; the third compression submodule includes a third compactification module and / or a third prediction module, and a third encapsulation module; The first compactification module is configured to perform compactification on the input 36th data and output the compactified data of the 36th data; The first prediction module is configured to perform a prediction on the input 37th data and to obtain residual data for the 37th data; The first encapsulation module is configured to process the input 38th data and obtain a bitstream of the 38th data; The 36th data is one of the plurality of region descriptors; the 37th data is one of the plurality of region descriptors or the compacted data of the 36th data; and the 38th data is the compacted data of the 36th data or the residual data of the 37th data; The second compactification module is configured to perform compactification on the input 39th data and output the compactified data of the 39th data; The second prediction module is configured to perform a prediction on the input 40th data and to obtain residual data for the 40th data; The second encapsulation module is configured to process the input 41st data and obtain a bitstream of the 41st data; The 39th data is a 3D map point descriptor for one of the plurality of 3D map points; the 40th data is the compacted data of the 39th data or the compacted data of the 39th data; and the 41st data is the residual data of the 40th data or the compacted data of the 39th data; The third compactification module is configured to perform compactification on the input 42 data and output the compactified data of the 42 data; The third prediction module is configured to perform a prediction on the input 43 data and to obtain residual data for the 43 data; The third encapsulation module is configured to process the input 44th data and obtain a bitstream of the 44th data; The apparatus according to item 21, wherein the 42nd data is the spatial position of one of the plurality of 3D map points, the 43rd data is the compactified data of the 42nd data or the spatial position of one of the plurality of 3D map points, and the 44th data is the compactified data of the 42nd data or the residual data of the 43rd data. [Item 23] The first compactification module includes a first quantization module and / or a first binarization module; the second compactification module includes a second quantization module and / or a second binarization module; the third compactification module includes a third quantization module and / or a third binarization module; The first quantization module is configured to perform quantization on the input 45th data and output the quantized data of the 45th data; The first binarization module is configured to perform binarization on the input 46th data and output the binary data of the 46th data; The 45th data is one of the plurality of region descriptors, and the 46th data is one of the plurality of region descriptors or the quantized data of the 45th data; The second quantization module is configured to perform quantization on the input 47th data and output the quantized data of the 47th data; The second binarization module is configured to perform binarization on the input 48 data and output the binary data of the 48 data; The 47th data is a 3D map point descriptor for one of the plurality of 3D map points, and the 48th data is the quantized data of the 3D map point descriptor for one of the plurality of 3D map points or the 47th data; The third quantization module is configured to perform quantization on the input 49th data and output the quantized data of the 49th data; The third binarization module is configured to perform binarization on the input 50th data and output the binary data of the 50th data; The apparatus according to item 22, wherein the 49th data is the spatial position of one of the plurality of 3D map points, and the 50th data is the spatial position of one of the plurality of 3D map points or the quantized data of the 49th data. [Item 24] A method for encoding 3D maps, The steps include: compressing the data of a 3D map to obtain a bitstream of the 3D map, wherein the 3D map includes a plurality of 3D map points, and the data of the 3D map includes the data of the plurality of 3D map points; and The step of transmitting the bitstream of the 3D map. A method that includes [a certain feature]. [Item 25] The method according to item 24, wherein the data of the 3D map further includes a plurality of region descriptors, any one of the plurality of region descriptors describes the characteristics of some or all of the plurality of 3D map points. [Item 26] The data for any one of the aforementioned multiple 3D map points includes a 3D map point descriptor and a 3D map point spatial location, as described in item 24 or 25. [Item 27] The process involves receiving a 3D map download request, where the 3D map download request includes location information; The step further comprising transmitting the bitstream of the 3D map is: The step of transmitting a bitstream of the 3D map corresponding to the position information in accordance with the 3D map download request. The method described in any one of items 24 to 26, including the method described in item 24 to 26. [Item 28] The step of transmitting the bitstream of the 3D map is: After the 3D map is created, the bitstream of the 3D map is transmitted. The method described in any one of items 24 to 26, including the method described in item 24 to 26. [Item 29] The above step of compressing the 3D map data to obtain the bitstream of the 3D map is: A step of performing compaction on the first data and obtaining the compacted data of the first data, and / or a step of performing prediction on the second data and obtaining the residual data of the second data; and The third step involves processing the data and obtaining the bitstream of the 3D map. This includes, here The first data is the data of the 3D map, the second data is the compacted data of the data of the 3D map or the first data, and the third data is the compacted data of the first data or the residual data of the second data. The method described in any one of items 24 to 28. [Item 30] Prior to the step of performing a prediction on the second data and obtaining residual data for the second data, the method further: The step of rearranging multiple sets of the aforementioned second data. Equipped with, The step of performing a prediction on the second data and obtaining residual data for the second data is: Based on the results of the reordering, a prediction is made on at least a portion of the second data, and residual data of at least a portion of the second data is obtained. The method described in item 29, including the method described in item 29. [Item 31] The step of performing compaction on the first data and obtaining the compacted data of the first data is: A step of performing quantization on the fourth data and obtaining the quantized data of the fourth data, and / or a step of performing binarization on the fifth data and obtaining the binary data of the fifth data. This includes, here The fourth data is the first data, and the fifth data is the quantized data of the first data or the fourth data, and accordingly, the compactified data of the first data includes the quantized data of the fourth data and / or the binary data of the fifth data. The method described in item 29 or 30. [Item 32] One or more processors; and Memory configured to store one or more programs Equipped with, here A device for encoding a 3D map, wherein, when the one or more programs are executed by the one or more processors, the one or more processors are capable of implementing the method described in any one of items 24 to 31. [Item 33] An encoder configured to compress 3D map data to obtain a 3D map obtained by compression and in bitstream form, wherein the 3D map includes a plurality of 3D map points, and the data of the 3D map includes the data of the plurality of 3D map points; and A memory configured to store the 3D map, which is obtained by compression and is in bitstream format. A device for encoding 3D maps, equipped with [specific features / equipment]. [Item 34] The apparatus according to item 33, wherein the data of the 3D map further includes a plurality of region descriptors, any one of the plurality of region descriptors describing the characteristics of some or all of the plurality of 3D map points. [Item 35] The apparatus according to item 33 or 34, wherein the data of any one of the plurality of 3D map points includes a 3D map point descriptor and a 3D map point spatial location. [Item 36] The apparatus for encoding a 3D map is a cloud server or an electronic device, as described in any one of items 33 to 35. [Item 37] A computer-readable storage medium containing a computer program, wherein when the computer program is executed on the computer, the computer is able to perform the method described in any one of items 24 to 31. [Item 38] A computer program, which, when executed on a computer, enables the computer to perform the method described in any one of items 24 to 31. [Item 39] A non-temporary storage medium containing a bitstream encoded by using the method described in any one of items 24 through 31.

Claims

1. A codec system for a 3D map comprising an encoding device and a decoding device, wherein the encoding device is communicatively connected to the decoding device; The encoding device is configured to: compress the data of a 3D map to obtain a bitstream of the 3D map, and transmit the bitstream of the 3D map to the decoding device, wherein the 3D map includes a plurality of 3D map points, the data of the 3D map includes the data of the plurality of 3D map points, the data of any one of the plurality of 3D map points includes a 3D map point descriptor and a 3D map point spatial location, the compression of the data of the 3D map includes compressing the 3D map point descriptor and the 3D map point spatial location of any one of the plurality of 3D map points, and the 3D map point descriptor is a vector used to represent the local function of the 3D map point; The decoding device is configured to receive the bitstream of the 3D map and to decompress the bitstream of the 3D map to obtain reconstructed data of the 3D map. system.

2. The encoding device is a cloud server, and the decoding device is an electronic device; or the encoding device is a first electronic device, and the decoding device is a second electronic device; The decoding device is further configured to transmit a 3D map download request to the encoding device, wherein the 3D map download request includes location information; The system according to claim 1, wherein the encoding device is further configured to receive the 3D map download request and to transmit to the decoding device a bitstream of the 3D map corresponding to the position indication information.

3. The encoding device is an electronic device, and the decoding device is a cloud server; The system according to claim 1 or 2, wherein the encoding device is specifically configured to transmit the bitstream of the 3D map to the decoding device after the 3D map has been created.

4. The system according to any one of claims 1 to 3, wherein the data of the 3D map further includes a plurality of region descriptors, and any one of the plurality of region descriptors describes the characteristics of some or all of the plurality of 3D map points.

5. The system according to any one of claims 1 to 4, wherein the encoding device is further configured to create the 3D map.

6. The decoding device is further configured to perform positioning based on the 3D map, and the 3D map point descriptors are used for matching between the 3D map points, according to any one of claims 1 to 5.

7. A device for encoding a 3D map, comprising a compression module and a transmission module, The compression module is configured to compress the data of a 3D map to obtain a bitstream of the 3D map, wherein the 3D map includes a plurality of 3D map points, the data of the 3D map includes the data of the plurality of 3D map points, the data of any one of the plurality of 3D map points includes a 3D map point descriptor and a 3D map point spatial location, the compression of the data of the 3D map includes compressing the 3D map point descriptor and the 3D map point spatial location of any one of the plurality of 3D map points, the 3D map point descriptor is a vector used to represent the local functionality of the 3D map point; The transmission module is configured to transmit the bitstream of the 3D map, in the apparatus.

8. The apparatus according to claim 7, wherein the data of the 3D map further includes a plurality of region descriptors, any one of the plurality of region descriptors describes the characteristics of some or all of the plurality of 3D map points.

9. The apparatus for encoding the 3D map is a cloud server or an electronic device; The transmission module is further configured to: receive a 3D map download request, wherein the 3D map download request includes location information; and transmit a bitstream of the 3D map corresponding to the location information in accordance with the 3D map download request. The apparatus according to claim 7 or 8.

10. The apparatus for encoding a 3D map is an electronic device; The apparatus according to claim 7 or 8, wherein the transmission module is specifically configured to transmit the bitstream of the 3D map after the 3D map has been created, and the 3D map point descriptors are used for matching between the 3D map points.

11. The compression module includes a compactification module and / or a prediction module, and an encapsulation module; The compactification module is configured to perform compactification on the input first data and output the compactified data of the first data; The prediction module is configured to perform a prediction on the input second data and output residual data for the second data; The encapsulation module is configured to process the input third data and output the bitstream of the 3D map; The apparatus according to any one of claims 7 to 10, wherein the first data is the data of the 3D map, the second data is the compactified data of the data of the 3D map or the first data, and the third data is the compactified data of the first data or the residual data of the second data.

12. The apparatus according to claim 11, wherein the prediction module is further configured to: rearrange a plurality of the second data, and, based on the results of the rearrangement, perform a prediction on at least a portion of the second data and obtain residual data on at least a portion of the second data.

13. The compactification module includes a quantization module and / or a binarization module; The quantization module is configured to perform quantization on the input fourth data and output the quantized data of the fourth data; The binarization module is configured to perform binarization on the input fifth data and output the binary data of the fifth data, where The apparatus according to claim 11 or 12, wherein the fourth data is the data of the 3D map, and the fifth data is the quantized data of the data of the 3D map or the fourth data.

14. The quantization module includes a first quantization module and a second quantization module, and / or the binarization module includes a first binarization module and a second binarization module, and the prediction module includes a first prediction module and a second prediction module; The first quantization module is configured to perform quantization on the input sixth data and output the quantized data of the sixth data; The first binarization module is configured to perform binarization on the input seventh data and output the binary data of the seventh data; The first prediction module is configured to perform a prediction on the input eighth data and output residual data for the eighth data; The sixth data is one of a plurality of region descriptors; the seventh data is one of the plurality of region descriptors or the quantized data of the sixth data; and the eighth data is one of the plurality of region descriptors, the quantized data of the sixth data, or the binary data of the seventh data; The second quantization module is configured to perform quantization on the input ninth data and output the quantized data of the ninth data; The second binarization module is configured to perform binarization on the input tenth data and output the binary data of the tenth data; The second prediction module is configured to perform a prediction on the input 11th data and output residual data for the 11th data; The apparatus according to claim 13, wherein the ninth data is data of one of the plurality of 3D map points, the tenth data is data of one of the plurality of 3D map points or the quantized data of the ninth data, and the eleventh data is data of one of the plurality of 3D map points, the quantized data of the ninth data, or the binary data of the tenth data.

15. A device for encoding a 3D map, comprising a compression module and a transmission module, The compression module is configured to compress the data of a 3D map to obtain a bitstream of the 3D map, wherein the 3D map includes a plurality of 3D map points, and the data of the 3D map includes the data of the plurality of 3D map points; The transmission module is configured to transmit the bitstream of the 3D map, The compression module includes a compactification module and / or a prediction module, and an encapsulation module; The compactification module is configured to perform compactification on the input first data and output the compactified data of the first data; The prediction module is configured to perform a prediction on the input second data and output residual data for the second data; The encapsulation module is configured to process the input third data and output the bitstream of the 3D map; The first data is the data of the 3D map; the second data is the compactified data of the data of the 3D map or the first data; and the third data is the compactified data of the first data or the residual data of the second data; The compactification module includes a quantization module and / or a binarization module; The quantization module includes a first quantization module and a second quantization module, and / or the binarization module includes a first binarization module and a second binarization module, and the prediction module includes a first prediction module and a second prediction module; The first quantization module is configured to perform quantization on the input sixth data and output the quantized data of the sixth data; The first binarization module is configured to perform binarization on the input seventh data and output the binary data of the seventh data; The first prediction module is configured to perform a prediction on the input eighth data and output residual data for the eighth data; The sixth data is one of a plurality of region descriptors; the seventh data is one of the plurality of region descriptors or the quantized data of the sixth data; and the eighth data is one of the plurality of region descriptors, the quantized data of the sixth data, or the binary data of the seventh data; The second quantization module is configured to perform quantization on the input ninth data and output the quantized data of the ninth data; The second binarization module is configured to perform binarization on the input tenth data and output the binary data of the tenth data; The second prediction module is configured to perform a prediction on the input 11th data and output residual data for the 11th data; The apparatus wherein the ninth data is data of one of the plurality of 3D map points, the tenth data is data of one of the plurality of 3D map points or the quantized data of the ninth data, and the eleventh data is data of one of the plurality of 3D map points, the quantized data of the ninth data, or the binary data of the tenth data.

16. The quantization module includes a first quantization module, a second quantization module, and a third quantization module, and / or the binarization module includes a first binarization module, a second binarization module, and a third binarization module, and the prediction module includes a first prediction module, a second prediction module, and a third prediction module; The first quantization module is configured to perform quantization on the input 12th data and output the quantized data of the 12th data; The first binarization module is configured to perform binarization on the input 13th data and output the binary data of the 13th data; The first prediction module is configured to perform a prediction on the input 14th data and output residual data for the 14th data; The 12th data is one of a plurality of region descriptors, the 13th data is one of the plurality of region descriptors or the quantized data of the 12th data, and the 14th data is one of the plurality of region descriptors, the quantized data of the 12th data, or the binary data of the 13th data; The second quantization module is configured to perform quantization on the input 15th data and output the quantized data of the 15th data; The second binarization module is configured to perform binarization on the input 16th data and output binary data of the 16th data; The second prediction module is configured to perform a prediction on the input 17th data and output residual data for the 17th data; The 15th data is a 3D map point descriptor for one of the plurality of 3D map points; the 16th data is the 3D map point descriptor for one of the plurality of 3D map points or the quantized data of the 15th data; and the 17th data is the 3D map point descriptor for one of the plurality of 3D map points, the quantized data of the 15th data, or the binary data of the 16th data; The third quantization module is configured to perform quantization on the input 18th data and output the quantized data of the 18th data; The third binarization module is configured to perform binarization on the input 19th data and output the binary data of the 19th data; The third prediction module is configured to perform a prediction on the input 20th data and output residual data for the 20th data; The apparatus according to claim 13, wherein the 18th data is the spatial position of one of the plurality of 3D map points, the 19th data is the spatial position of one of the plurality of 3D map points or the quantized data of the 18th data, and the 20th data is the spatial position of one of the plurality of 3D map points, the quantized data of the 18th data, or the binary data of the 19th data.

17. A device for encoding a 3D map, comprising a compression module and a transmission module, The compression module is configured to compress the data of a 3D map to obtain a bitstream of the 3D map, wherein the 3D map includes a plurality of 3D map points, and the data of the 3D map includes the data of the plurality of 3D map points; The transmission module is configured to transmit the bitstream of the 3D map, The compression module includes a compactification module and / or a prediction module, and an encapsulation module; The compactification module is configured to perform compactification on the input first data and output the compactified data of the first data; The prediction module is configured to perform a prediction on the input second data and output residual data for the second data; The encapsulation module is configured to process the input third data and output the bitstream of the 3D map; The first data is the data of the 3D map; the second data is the compactified data of the data of the 3D map or the first data; and the third data is the compactified data of the first data or the residual data of the second data; The compactification module includes a quantization module and / or a binarization module; The quantization module includes a first quantization module, a second quantization module, and a third quantization module, and / or the binarization module includes a first binarization module, a second binarization module, and a third binarization module, and the prediction module includes a first prediction module, a second prediction module, and a third prediction module; The first quantization module is configured to perform quantization on the input 12th data and output the quantized data of the 12th data; The first binarization module is configured to perform binarization on the input 13th data and output the binary data of the 13th data; The first prediction module is configured to perform a prediction on the input 14th data and output residual data for the 14th data; The 12th data is one of a plurality of region descriptors, the 13th data is one of the plurality of region descriptors or the quantized data of the 12th data, and the 14th data is one of the plurality of region descriptors, the quantized data of the 12th data, or the binary data of the 13th data; The second quantization module is configured to perform quantization on the input 15th data and output the quantized data of the 15th data; The second binarization module is configured to perform binarization on the input 16th data and output binary data of the 16th data; The second prediction module is configured to perform a prediction on the input 17th data and output residual data for the 17th data; The 15th data is a 3D map point descriptor for one of the plurality of 3D map points; the 16th data is the 3D map point descriptor for one of the plurality of 3D map points or the quantized data of the 15th data; and the 17th data is the 3D map point descriptor for one of the plurality of 3D map points, the quantized data of the 15th data, or the binary data of the 16th data; The third quantization module is configured to perform quantization on the input 18th data and output the quantized data of the 18th data; The third binarization module is configured to perform binarization on the input 19th data and output the binary data of the 19th data; The third prediction module is configured to perform a prediction on the input 20th data and output residual data for the 20th data; The apparatus wherein the 18th data is the spatial position of one of the plurality of 3D map points, the 19th data is the spatial position of one of the plurality of 3D map points or the quantized data of the 18th data, and the 20th data is the spatial position of one of the plurality of 3D map points, the quantized data of the 18th data, or the binary data of the 19th data.

18. The compression module includes a first compression submodule and a second compression submodule; The first compression submodule is configured to compress the input 21st data and output a bitstream of the 21st data; The second compression submodule is configured to compress the input 22 data and output a bitstream of the 22 data; The apparatus according to any one of claims 7 to 10, wherein the 21st data is one of a plurality of region descriptors, and the 22nd data is data of one of the plurality of 3D map points.

19. The first compression submodule includes a first compactification module and / or a first prediction module, and a first encapsulation module; the second compression submodule includes a second compactification module and / or a second prediction module, and a second encapsulation module; The first compactification module is configured to perform compactification on the input 23 data and output the compactified data of the 23 data; The first prediction module is configured to perform a prediction on the input 24 data and output residual data for the 24 data; The first encapsulation module is configured to process the input 25th data and output a bitstream of the 25th data; The 23rd data is one of the plurality of region descriptors, the 24th data is one of the plurality of region descriptors or the compacted data of the 23rd data, and the 25th data is the compacted data of the 23rd data or the residual data of the 24th data; The second compactification module is configured to perform compactification on the input 26th data and output the compactified data of the 26th data; The second prediction module is configured to perform a prediction on the input 27th data and output residual data for the 27th data; The second encapsulation module is configured to process the input 28 data and output a bitstream of the 28 data; The apparatus according to claim 18, wherein the 26th data is data of one of the plurality of 3D map points, the 27th data is data of one of the plurality of 3D map points or the compacted data of the 26th data, and the 28th data is the compacted data of the 26th data or the residual data of the 27th data.

20. A device for encoding a 3D map, comprising a compression module and a transmission module, The compression module is configured to compress the data of a 3D map to obtain a bitstream of the 3D map, wherein the 3D map includes a plurality of 3D map points, and the data of the 3D map includes the data of the plurality of 3D map points; The transmission module is configured to transmit the bitstream of the 3D map, The compression module includes a first compression submodule and a second compression submodule; The first compression submodule includes a first compactification module and / or a first prediction module, and a first encapsulation module; the second compression submodule includes a second compactification module and / or a second prediction module, and a second encapsulation module; The first compactification module is configured to perform compactification on the input 23 data and output the compactified data of the 23 data; The first prediction module is configured to perform a prediction on the input 24 data and output residual data for the 24 data; The first encapsulation module is configured to process the input 25th data and output a bitstream of the 25th data; The 23rd data is one of a plurality of region descriptors, the 24th data is one of the plurality of region descriptors or the compacted data of the 23rd data, and the 25th data is the compacted data of the 23rd data or the residual data of the 24th data; The second compactification module is configured to perform compactification on the input 26th data and output the compactified data of the 26th data; The second prediction module is configured to perform a prediction on the input 27th data and output residual data for the 27th data; The second encapsulation module is configured to process the input 28 data and output a bitstream of the 28 data; The apparatus wherein the 26th data is data of one of the plurality of 3D map points, the 27th data is the data of one of the plurality of 3D map points or the compacted data of the 26th data, and the 28th data is the compacted data of the 26th data or the residual data of the 27th data.

21. The first compactification module includes a first quantization module and / or a first binarization module, and the second compactification module includes a second quantization module and / or a second binarization module; The first quantization module is configured to perform quantization on the input 29th data and output the quantized data of the 29th data; The first binarization module is configured to perform binarization on the input 30th data and output binary data of the 30th data; The 29th data is one of the plurality of region descriptors, and the 30th data is one of the plurality of region descriptors or the quantized data of the 29th data; The second quantization module is configured to perform quantization on the input 31st data and output the quantized data of the 31st data; The second binarization module is configured to perform binarization on the input 32 data and output the binary data of the 32 data; The apparatus according to claim 19 or 20, wherein the 31st data is data of one of the plurality of 3D map points, and the 30th data is data of one of the plurality of 3D map points or the quantized data of the 31st data.

22. The compression module includes a first compression submodule, a second compression submodule, and a third compression submodule; The first compression submodule is configured to compress the input 33 data and output a bitstream of the 33 data; The second compression submodule is configured to compress the input 34 data and output a bitstream of the 34 data; The third compression submodule is configured to compress the input 35th data and output a bitstream of the 35th data; The apparatus according to any one of claims 7 to 10, wherein the 33rd data is one of a plurality of region descriptors, the 34th data is a 3D map point descriptor of one of the plurality of 3D map points, and the 35th data is the spatial position of one of the plurality of 3D map points.

23. The first compression submodule includes a first compactification module and / or a first prediction module, and a first encapsulation module; the second compression submodule includes a second compactification module and / or a second prediction module, and a second encapsulation module; the third compression submodule includes a third compactification module and / or a third prediction module, and a third encapsulation module; The first compactification module is configured to perform compactification on the input 36th data and output the compactified data of the 36th data; The first prediction module is configured to perform a prediction on the input 37th data and to obtain residual data for the 37th data; The first encapsulation module is configured to process the input 38 data and obtain a bitstream of the 38 data; The 36th data is one of the plurality of region descriptors, the 37th data is one of the plurality of region descriptors or the compacted data of the 36th data, and the 38th data is the compacted data of the 36th data or the residual data of the 37th data; The second compactification module is configured to perform compactification on the input 39th data and output the compactified data of the 39th data; The second prediction module is configured to perform a prediction on the input 40th data and to obtain residual data for the 40th data; The second encapsulation module is configured to process the input 41 data and obtain a bitstream of the 41 data; The 39th data is a 3D map point descriptor for one of the plurality of 3D map points; the 40th data is the compactified data of the 39th data or the compactified data of the 39th data; and the 41st data is the residual data of the 39th data or the 40th data; The third compactification module is configured to perform compactification on the input 42 data and output the compactified data of the 42 data; The third prediction module is configured to perform a prediction on the input 43 data and to obtain residual data for the 43 data; The third encapsulation module is configured to process the input 44th data and obtain a bitstream of the 44th data; The apparatus according to claim 22, wherein the 42nd data is the spatial position of one of the plurality of 3D map points, the 43rd data is the spatial position of one of the plurality of 3D map points or the compactified data of the 42nd data, and the 44th data is the compactified data of the 42nd data or the residual data of the 43rd data.

24. A device for encoding a 3D map, comprising a compression module and a transmission module, The compression module is configured to compress the data of a 3D map to obtain a bitstream of the 3D map, wherein the 3D map includes a plurality of 3D map points, and the data of the 3D map includes the data of the plurality of 3D map points; The transmission module is configured to transmit the bitstream of the 3D map; The compression module includes a first compression submodule, a second compression submodule, and a third compression submodule; The first compression submodule includes a first compactification module and / or a first prediction module, and a first encapsulation module; the second compression submodule includes a second compactification module and / or a second prediction module, and a second encapsulation module; the third compression submodule includes a third compactification module and / or a third prediction module, and a third encapsulation module; The first compactification module is configured to perform compactification on the input 36th data and output the compactified data of the 36th data; The first prediction module is configured to perform a prediction on the input 37th data and to obtain residual data for the 37th data; The first encapsulation module is configured to process the input 38 data and obtain a bitstream of the 38 data; The 36th data is one of a plurality of region descriptors, the 37th data is one of the plurality of region descriptors or the compacted data of the 36th data, and the 38th data is the compacted data of the 36th data or the residual data of the 37th data; The second compactification module is configured to perform compactification on the input 39th data and output the compactified data of the 39th data; The second prediction module is configured to perform a prediction on the input 40th data and to obtain residual data for the 40th data; The second encapsulation module is configured to process the input 41 data and obtain a bitstream of the 41 data; The 39th data is a 3D map point descriptor for one of the plurality of 3D map points; the 40th data is the compactified data of the 39th data or the compactified data of the 39th data; and the 41st data is the residual data of the 39th data or the 40th data; The third compactification module is configured to perform compactification on the input 42 data and output the compactified data of the 42 data; The third prediction module is configured to perform a prediction on the input 43 data and to obtain residual data for the 43 data; The third encapsulation module is configured to process the input 44th data and obtain a bitstream of the 44th data; The apparatus wherein the 42nd data is the spatial position of one of the plurality of 3D map points, the 43rd data is the spatial position of one of the plurality of 3D map points or the compactified data of the 42nd data, and the 44th data is the compactified data of the 42nd data or the residual data of the 43rd data.

25. The first compactification module includes a first quantization module and / or a first binarization module; the second compactification module includes a second quantization module and / or a second binarization module; the third compactification module includes a third quantization module and / or a third binarization module; The first quantization module is configured to perform quantization on the input 45th data and output the quantized data of the 45th data; The first binarization module is configured to perform binarization on the input 46th data and output the binary data of the 46th data; The 45th data is one of the plurality of region descriptors, and the 46th data is one of the plurality of region descriptors or the quantized data of the 45th data; The second quantization module is configured to perform quantization on the input 47th data and output the quantized data of the 47th data; The second binarization module is configured to perform binarization on the input 48 data and output the binary data of the 48 data; The 47th data is a 3D map point descriptor for one of the plurality of 3D map points, and the 48th data is the quantized data of the 3D map point descriptor for one of the plurality of 3D map points or the 47th data; The third quantization module is configured to perform quantization on the input 49th data and output the quantized data of the 49th data; The third binarization module is configured to perform binarization on the input 50th data and output the binary data of the 50th data; The apparatus according to claim 23 or 24, wherein the 49th data is a spatial position of one of the plurality of 3D map points, and the 50th data is a spatial position of one of the plurality of 3D map points or the quantized data of the 49th data.

26. A method for encoding a 3D map, The step of compressing the data of a 3D map to obtain a bitstream of the 3D map, wherein the 3D map includes a plurality of 3D map points, the data of the 3D map includes the data of the plurality of 3D map points, the data of any one of the plurality of 3D map points includes a 3D map point descriptor and a 3D map point spatial location, compressing the data of the 3D map includes compressing the 3D map point descriptor and a 3D map point spatial location of any one of the plurality of 3D map points, the 3D map point descriptor is a vector used to represent the local functionality of the 3D map point; and The step of transmitting the bitstream of the 3D map. A method that includes [a certain feature].

27. The method according to claim 26, wherein the data of the 3D map further includes a plurality of region descriptors, any one of the plurality of region descriptors describes the characteristics of some or all of the plurality of 3D map points.

28. The process involves receiving a 3D map download request, where the 3D map download request includes location information; The step further comprising transmitting the bitstream of the 3D map includes: The step of transmitting a bitstream of the 3D map corresponding to the position indication information in accordance with the 3D map download request. The method according to claim 26 or 27, including the method described in claim 26 or 27.

29. The step of transmitting the bitstream of the 3D map is: After the 3D map is created, the bitstream of the 3D map is transmitted. Includes, The method according to claim 26 or 27, wherein the 3D map point descriptor is used for matching between the 3D map points.

30. The step of compressing the 3D map data to obtain the bitstream of the 3D map is: A step of performing compaction on the first data and obtaining the compacted data of the first data, and / or a step of performing prediction on the second data and obtaining the residual data of the second data; and The third step involves processing the data and obtaining the bitstream of the 3D map. This includes, here The first data is the data of the 3D map, the second data is the compacted data of the data of the 3D map or the first data, and the third data is the compacted data of the first data or the residual data of the second data. The method according to any one of claims 26 to 29.

31. Prior to the step of performing a prediction on the second data and obtaining residual data for the second data, the method further: Step of rearranging the order of multiple second data items. Equipped with, The step of performing a prediction on the second data and obtaining residual data for the second data is: Based on the results of the reordering, a prediction is performed on at least a portion of the second data, and residual data of at least a portion of the second data is obtained. The method according to claim 30, including the method described in claim 30.

32. The step of performing compaction on the first data and obtaining the compacted data of the first data is: A step of performing quantization on the fourth data and obtaining the quantized data of the fourth data, and / or a step of performing binarization on the fifth data and obtaining the binary data of the fifth data. This includes, here The fourth data is the first data, and the fifth data is the quantized data of the first data or the fourth data, and accordingly, the compactified data of the first data includes the quantized data of the fourth data and / or the binary data of the fifth data. The method according to claim 30 or 31.

33. One or more processors; and Memory configured to store one or more programs Equipped with, here An apparatus for encoding a 3D map, wherein, when the one or more programs are executed by the one or more processors, the one or more processors are capable of implementing the method according to any one of claims 26 to 32.

34. An encoder configured to compress data of a 3D map to obtain a 3D map obtained by compression and in bitstream form, wherein the 3D map comprises a plurality of 3D map points, the data of the 3D map comprises data of the plurality of 3D map points, the data of any one of the plurality of 3D map points comprises a 3D map point descriptor and a 3D map point spatial position, the compression of the data of the 3D map comprises compressing the 3D map point descriptor and the 3D map point spatial position of any one of the plurality of 3D map points, the 3D map point descriptor is a vector used to represent the local function of the 3D map point; and A memory configured to store the 3D map, which is obtained by compression and is in bitstream format. A device for encoding 3D maps, equipped with the following features.

35. The apparatus according to claim 34, wherein the data of the 3D map further includes a plurality of region descriptors, any one of the plurality of region descriptors describes the characteristics of some or all of the plurality of 3D map points.

36. The apparatus for encoding a 3D map is a cloud server or an electronic device, and the 3D map point descriptors are used for matching the 3D map points, according to claim 34 or 35.

37. A computer-readable storage medium containing a computer program, wherein when the computer program is executed on a computer, the computer is able to perform the method according to any one of claims 26 to 32.

38. A computer program, wherein when the computer program is executed on a computer, the computer is able to perform the method described in any one of claims 26 to 32.

39. A method for storing a bitstream of a 3D map, The steps include generating the bitstream and transmitting the bitstream through a communication interface, A step of storing the bitstream in one or more storage media, wherein the 3D map includes a plurality of 3D map points, the data of the 3D map includes data of the plurality of 3D map points, the data of any one of the plurality of 3D map points includes a 3D map point descriptor and a 3D map point spatial location, the bitstream has compressed data of the 3D map point descriptor of any one of the plurality of 3D map points and compressed data of the 3D map point spatial location, the compressed data of the 3D map point descriptor and the compressed data of the 3D map point spatial location are obtained by compressing the 3D map point descriptor and the 3D map point spatial location of any one of the plurality of 3D map points, the 3D map point descriptor is a vector used to represent the local functionality of the 3D map point, and A method that includes [a certain feature].

40. A device for storing a bitstream of a 3D map, the device comprising generating the bitstream, at least one storage medium, and at least one communication interface, The at least one communication interface is configured to transmit the bitstream, The device comprises at least one storage medium configured to store the bitstream, wherein the 3D map comprises a plurality of 3D map points, the data of the 3D map comprises data of the plurality of 3D map points, the data of any one of the plurality of 3D map points comprises a 3D map point descriptor and a 3D map point spatial location, the bitstream has compressed data of the 3D map point descriptor of any one of the plurality of 3D map points and compressed data of the 3D map point spatial location, the compressed data of the 3D map point descriptor and the compressed data of the 3D map point spatial location are obtained by compressing the 3D map point descriptor and the 3D map point spatial location of any one of the plurality of 3D map points, the 3D map point descriptor is a vector used to represent the local functionality of the 3D map point.

41. A method for transmitting a bitstream of a 3D map, A step of generating the bitstream and storing the bitstream in one of at least one storage mediums, and retrieving the bitstream of the 3D map from one of the at least one storage mediums, wherein the 3D map includes a plurality of 3D map points, the data of the 3D map includes data of the plurality of 3D map points, the data of any one of the plurality of 3D map points includes a 3D map point descriptor and a 3D map point spatial location, the bitstream has compressed data of the 3D map point descriptor of any one of the plurality of 3D map points and compressed data of the 3D map point spatial location, the compressed data of the 3D map point descriptor and the compressed data of the 3D map point spatial location are obtained by compressing the 3D map point descriptor and the 3D map point spatial location of any one of the plurality of 3D map points, the 3D map point descriptor is a vector used to represent the local functionality of the 3D map point, and The step of transmitting the bitstream A method that includes [a certain feature].

42. A device for transmitting a bitstream of a 3D map, wherein the device generates the bitstream, At least one storage medium configured to store the bitstream, wherein the 3D map includes a plurality of 3D map points, the data of the 3D map includes data of the plurality of 3D map points, the data of any one of the plurality of 3D map points includes a 3D map point descriptor and a 3D map point spatial location, the bitstream has compressed data of the 3D map point descriptor of any one of the plurality of 3D map points and compressed data of the 3D map point spatial location, the compressed data of the 3D map point descriptor and the compressed data of the 3D map point spatial location are obtained by compressing the 3D map point descriptor and the 3D map point spatial location of any one of the plurality of 3D map points, and the 3D map point descriptor is a vector used to represent the local functionality of the 3D map point, A processor configured to acquire one or more bitstreams from one of the at least one storage mediums and to transmit the one or more bitstreams. A device equipped with the following features.

Images