Apparatus and method for decoding 3D maps and encoded bitstreams of 3D maps

Abstract

The present application provides an apparatus and method for decoding a 3D map, and an encoded bitstream of the 3D map. The apparatus for decoding the 3D map in the present application includes a transmission module and a decompression module. The transmission module is configured to receive a bitstream of the 3D map, where the 3D map includes a plurality of 3D map points. The decompression module is configured to decompress the bitstream of the 3D map to obtain reconstruction data of the 3D map, where the reconstruction data of the 3D map includes reconstruction data of a plurality of 3D map points. In the present application, the decompressor end may support decompression of the compressed data of the 3D map to support compression / decompression of the data of the 3D map. This can reduce the number of bits occupied by the data of the 3D map, thereby reducing the transmission bandwidth and improving the transmission efficiency.

Description

[Technical Field]

[0001] The present application relates to 3D map technology, and in particular to an apparatus and method for decoding a 3D map and an encoded bitstream of the 3D map. [Background technology]

[0002] Virtual reality (VR), augmented reality (AR), and mixed reality (MR) are emerging multimedia virtual scene technologies. These technologies can be used to generate virtual reality and overlay it on the real world to create new visual environments and interactive experiences. In such applications, electronic devices need to determine pose information of the electronic device in the current environment to accurately perform blending between virtual objects and real scenes.

[0003] Furthermore, in applications such as autonomous driving, autonomous navigation, unmanned aerial vehicle automatic inspection, and industrial robotics, in order to perform accurate route planning, navigation, detection, and control, a transport device such as a vehicle, unmanned aerial vehicle, or robot needs to determine the pose of an electronic device being transported by the transport device, thereby determining the pose of the transport device in the current environment.

[0004] In the aforementioned applications, with regard to the problem of needing to determine the pose of the electronic device in the current environment, a typical solution is as follows: the electronic device receives a 3D map of the environment in which the electronic device is located from a server or another device, collects visual information in the environment by using local sensors, and determines the current pose of the electronic device based on the collected visual information and the downloaded 3D map.

[0005] However, the original 3D map usually contains a large data volume, and map transmission requires a large amount of bandwidth and time consumption, which severely limits application performance and affects user experience. Summary of the Invention

[0006] The present application provides an apparatus and method for decoding a 3D map and an encoded bitstream of the 3D map, so as to reduce the data volume of the 3D map, thereby reducing the transmission bandwidth and improving transmission efficiency.

[0007] According to a first aspect, the present application provides a codec system for a 3D map, comprising an encoding device and a decoding device. The encoding device is communicably connected to a decoding device. The encoding device is configured to compress data of the 3D map to obtain a 3D map bitstream and transmit the 3D map bitstream to the decoding device, where the 3D map includes a plurality of 3D map points and the 3D map data includes data of the plurality of 3D map points. The decoding device is configured to receive the 3D map bitstream and decompress the 3D map bitstream to obtain the 3D map data.

[0008] The 3D map may include a plurality of 3D map points, and accordingly the data for the 3D map may include data for a plurality of 3D map points, which are points of interest or points having significant features in the environment.

[0009] In this embodiment of the present application, the compression module compresses the data of the 3D map to reduce the data volume of the 3D map, for example, to reduce the data volume of the 3D map from the Terabyte (TB) level to the Gigabyte (GB) level. Therefore, in a scenario where a 3D map needs to be transmitted, by transmitting compressed data of the 3D map instead of transmitting the original data of the 3D map, the data volume for transmission can be reduced and the bandwidth occupied by the transmission can be further reduced, thereby improving the transmission efficiency of the 3D map.

[0010] In a possible implementation, the encoding device is a cloud server and the decoding device is an electronic device; or the encoding device is a first electronic device and the decoding device is a second electronic device. The decoding device is further configured to send a 3D map download request to the decoding device, the 3D map download request including location indication information. The encoding device is further configured to receive the 3D map download request and send a bitstream of the 3D map corresponding to the location indication information to the decoding device according to the 3D map download request.

[0011] The aforementioned electronic device may be a user terminal device.

[0012] In a possible implementation, the encoding device is an electronic device and the decoding device is a cloud server, the encoding device being particularly configured to transmit a bitstream of the 3D map to the decoding device after the 3D map has been generated.

[0013] Optionally, in this embodiment of the present application, the electronic device may collect visual information by using a sensor and determine the current pose of the electronic device by referring to the visual information and a 3D map from the server.

[0014] The 3D map is provided by a server. Specifically, the server generates a 3D map, then compresses the 3D map, and sends the compressed 3D map data to the electronic device. After receiving the compressed 3D map data, the electronic device performs decompression to obtain reconstructed data of the 3D map, and determines the current pose of the electronic device by referring to the collected visual information and the 3D map. The pose refers to the position and orientation information of the electronic device, and may be an absolute pose in a world coordinate system or a pose relative to a point in the environment.

[0015] In this embodiment of the present application, the server may generate a 3D map in advance, compress the 3D map, and then store the compressed data of the 3D map locally, thereby saving storage space. Furthermore, the server may send the compressed data of the 3D map to another device, such as cloud storage.

[0016] 1. The server generates a 3D map, compresses the 3D map to obtain compressed data of the 3D map, and stores the compressed data locally.

[0017] The server compresses the 3D map to save local storage space.

[0018] 2. The electronic device sends a map download request to the server. The map download request can be triggered in two ways.

[0019] (1) A user starts a 3D map application installed on an electronic device, and the application uploads location information obtained based on GPS positioning or Wi-Fi positioning to a server corresponding to the application. The upload operation may trigger a map download request. Since the uploaded content includes location information, the server may perform a preliminary estimation based on the location information and send compressed data of a 3D map of the area to which the positioning point indicated by the location information belongs to the electronic device. The range of the area to which the positioning point indicated by the location information belongs may be preset. For example, the area to which the positioning point belongs may be an administrative region of any level (including a county, city, country, or administrative region) where the positioning point is located, or may be a circular area centered on the positioning point and using a specified distance as a radius.

[0020] (2) A user starts a 3D map application installed on an electronic device and actively enters or selects an area on the application. For example, the user actively enters "xx business center" or selects "Street A" from a list of "Street A, Street B, and Street C." The user's aforementioned operation may trigger a map download request. Whether the user enters or selects a geographic location, the server accordingly transmits compressed data of a 3D map of the geographic location to the electronic device.

[0021] It should be understood that in this embodiment of the present application, in addition to the two methods described above, other methods may be used to trigger a map download request. For example, the electronic device automatically detects whether a condition for downloading a 3D map or starting a 3D map download is met, or the electronic device starts a 3D map download and requests to download a 3D map of an area range from a server when it detects a change in ambient light or an environment change. The size of the area range is not particularly limited.

[0022] 3. The server sends the compressed data of the 3D map to the electronic device.

[0023] 4. Electronic devices collect visual information.

[0024] It should be noted that steps 3 and 4 are independent of each other and the order in which they are performed is not limited.

[0025] 5. The electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.

[0026] 6. The electronic device performs positioning in a 3D map based on the visual information to obtain a pose corresponding to the visual information.

[0027] After receiving the compressed data of the 3D map, the electronic device does not need to immediately decompress the compressed data, but only needs to decompress the compressed data to obtain reconstructed data of the 3D map before performing positioning based on visual information. For example, a user may pre-download compressed data of the 3D map of an area range by downloading an "offline map," and decompress the compressed data of the 3D map only when positioning is required.

[0028] Optionally, in this embodiment of the present application, the electronic device may collect visual information by using a sensor, and the server determines the current pose of the electronic device by referring to the visual information from the electronic device and the 3D map.

[0029] The 3D map is provided by a server. Specifically, the server generates the 3D map, then compresses the 3D map, and locally stores the compressed data of the 3D map. When receiving visual information from the electronic device, the server performs decompression to obtain reconstructed data of the 3D map, and references the visual information and the 3D map to determine the current pose of the electronic device.

[0030] 1. The server generates a 3D map, compresses the 3D map to obtain compressed data of the 3D map, and stores the compressed data locally.

[0031] 2. Electronic devices collect visual information.

[0032] 3. The electronic device transmits the visual information to the server.

[0033] 4. The server decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.

[0034] It should be understood that the server compresses the 3D map to save storage space.

[0035] 5. The server performs positioning in the 3D map based on the visual information to obtain a pose corresponding to the visual information.

[0036] 6. The server sends the pause to the electronic device.

[0037] Optionally, in this embodiment of the present application, the electronic device may collect visual information by using a sensor and determine the current pose of the electronic device by referring to the visual information and the 3D map.

[0038] The 3D map is provided by the electronic device. Specifically, the electronic device generates the 3D map, then compresses the 3D map, and locally stores the compressed data of the 3D map. When visual information is collected, the electronic device performs decompression to obtain reconstructed data of the 3D map, and determines the current pose of the electronic device by referring to the collected visual information and the 3D map.

[0039] 1. The electronic device generates a 3D map, compresses the 3D map to obtain compressed data of the 3D map, and locally stores the compressed data.

[0040] It should be understood that the electronic device may compress the 3D map to save storage space.

[0041] 2. Electronic devices collect visual information by using sensors.

[0042] 3. The electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.

[0043] 4. The electronic device performs positioning in a 3D map based on the visual information to obtain a pose corresponding to the visual information.

[0044] Optionally, in this embodiment of the present application, the second electronic device may collect visual information by using a sensor and determine the current pose of the second electronic device by referring to the visual information and a 3D map from the server.

[0045] The 3D map is generated by a first electronic device. Specifically, the first electronic device generates the 3D map, compresses the 3D map, and then transmits the compressed 3D map data to a server. The server then transmits the compressed 3D map data to a second electronic device. The second electronic device performs decompression to obtain reconstructed data of the 3D map and determines the current pose of the second electronic device by referring to the collected visual information and the 3D map.

[0046] In this embodiment of the present application, the first electronic device may generate a 3D map in advance, compress the 3D map, and then transmit the compressed data of the 3D map to the server, thereby reducing the transmission bandwidth.

[0047] 1. A first electronic device generates a 3D map and compresses the 3D map to obtain compressed data of the 3D map.

[0048] 2. The first electronic device sends compressed data of the 3D map to the server.

[0049] The first electronic device compresses the 3D map and then transmits the compressed data of the 3D map, thereby reducing transmission bandwidth and improving transmission efficiency.

[0050] 3. The second electronic device sends a map download request to the server.

[0051] The second electronic device may send a map download request based on the trigger scheme shown in FIG. 4a.

[0052] 4. The server sends the compressed data of the 3D map to the second electronic device.

[0053] 5. The second electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.

[0054] 6. The second electronic device collects visual information by using a sensor.

[0055] 7. The second electronic device performs positioning in the 3D map based on the visual information and obtains a pose corresponding to the visual information.

[0056] Optionally, in this embodiment of the present application, the second electronic device may collect visual information by using a sensor, and the server determines the current pose of the second electronic device by referring to the visual information from the second electronic device and the 3D map from the first electronic device.

[0057] The 3D map is generated by a first electronic device. Specifically, the first electronic device generates a 3D map, compresses the 3D map, and then transmits the compressed data of the 3D map to a server. The server performs decompression to obtain reconstructed data of the 3D map, and determines the current pose of the second electronic device by referring to the visual information from the second electronic device and the 3D map.

[0058] 1. A first electronic device generates a 3D map and compresses the 3D map to obtain compressed data of the 3D map.

[0059] 2. The first electronic device sends compressed data of the 3D map to the server.

[0060] 3. The second electronic device collects visual information by using a sensor.

[0061] 4. The second electronic device sends a positioning request to the server, where the positioning request carries the visual information.

[0062] 5. The server decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.

[0063] 6. The server performs positioning in the 3D map based on the visual information to obtain a pose corresponding to the visual information.

[0064] 7. The server transmits the pose obtained through positioning to the second electronic device.

[0065] Optionally, in this embodiment of the present application, the second electronic device may collect visual information by using a sensor and determine the current pose of the second electronic device by referring to the visual information and the 3D map from the first electronic device.

[0066] The 3D map is generated by a first electronic device. Specifically, the first electronic device generates the 3D map, compresses the 3D map, and then transmits the compressed data of the 3D map to a second electronic device. The second electronic device performs decompression to obtain reconstructed data of the 3D map and determines a current pose of the second electronic device by referring to the collected visual information and the 3D map from the first electronic device.

[0067] 1. A first electronic device generates a 3D map, compresses the 3D map to obtain compressed data of the 3D map, and locally stores the compressed data.

[0068] 2. The second electronic device sends a map download request to the first electronic device.

[0069] 3. The first electronic device transmits the compressed data of the 3D map to the second electronic device.

[0070] 4. The second electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.

[0071] 5. The second electronic device collects visual information by using a sensor.

[0072] 6. The second electronic device performs positioning in a 3D map based on the visual information to obtain a pose corresponding to the visual information.

[0073] In a possible implementation, the 3D map data further includes a plurality of area descriptors, any one of which describes characteristics of some or all of the plurality of 3D map points.

[0074] For any one of the plurality of area descriptors, the area descriptor may describe characteristics of some or all of the plurality of 3D map points. In this case, the area descriptor and the 3D map point have a one-to-many relationship. Each characteristic of the plurality of 3D map points may be described by some or all of the area descriptors of the plurality of area descriptors. In this case, the 3D map point and the area descriptor have a one-to-many relationship. It can be seen that the plurality of area descriptors and the plurality of 3D map points have a many-to-many relationship. Methods for generating area descriptors include, but are not limited to, traditional methods such as bag of words (BOW) and vector of locally aggregated descriptors (VLAD), as well as novel methods based on NetVLAD or artificial intelligence (AI). Similarly, the plurality of area descriptors may be identified by numbers to distinguish the plurality of area descriptors. However, the numbers are similarly not intended to limit the order of the plurality of area descriptors.

[0075] In a possible implementation, the data for any one of the plurality of 3D map points includes a 3D map point descriptor and a 3D map point spatial location.

[0076] A 3D map point descriptor is a vector used to represent the local features of a 3D map point.

[0077] The 3D map point spatial position may be represented by using X, Y, and Z on three-dimensional spatial axes, or by using longitude, latitude, and altitude, or by using polar coordinates, etc. The method for representing the 3D map point spatial position is not particularly limited in the embodiments of the present application. The 3D map point spatial position may be the absolute position of the 3D map point, or the relative position of the 3D map point.

[0078] In a possible implementation, the encoding device is further configured to generate a 3D map.

[0079] In a possible implementation, the decoding device is further configured to perform positioning based on a 3D map.

[0080] According to a second aspect, the present application provides an apparatus for decoding a 3D map, comprising: a transmission module configured to receive a bitstream of a 3D map, where the 3D map includes a plurality of 3D map points; and a decompression module configured to decompress the bitstream of the 3D map to obtain reconstruction data of the 3D map, where the reconstruction data of the 3D map includes reconstruction data of the plurality of 3D map points.

[0081] In this embodiment of the present application, the decompression module may support decompressing the compressed data of the 3D map. In other words, the encoding and decoding system may support compressing / decompressing the data of the 3D map so as to reduce the data volume of the 3D map. In a scenario where a 3D map needs to be transmitted, transmitting compressed data of the 3D map instead of transmitting the original data of the 3D map can reduce the data volume for transmission and further reduce the bandwidth occupied by the transmission, thereby improving the transmission efficiency of the 3D map.

[0082] In this embodiment of the present application, decompression may include at least one of dequantization and prediction.

[0083] Quantization refers to mapping data to be processed to one or more quantization indexes, where each quantization index corresponds to a quantization center. The number of bits of a quantization index is usually significantly smaller than the number of bits of the original data, thereby saving storage or transmission bandwidth. Quantization methods include, but are not limited to, scalar quantization, vector quantization, and product quantization. Based on this, dequantization is the inverse of the aforementioned quantization. Specifically, reconstructed data corresponding to the original data is restored based on one or more quantization indexes, and the number of bits of the reconstructed data is greater than the number of bits of the quantization indexes.

[0084] Prediction means performing prediction by using residual data of the processed data and the data to be processed to obtain the data to be processed. The selection of reference data may be agreed upon in advance. For example, previously processed data is fixedly used as reference data, in which case the reference data does not need to be identified in the bitstream. For another example, any processed data may be used, in which case the identification information of the reference data needs to be written into the bitstream, and includes the number of the reference data or other information that can be used to infer the reference data.

[0085] In the above description of the 3D map data, it can be seen that the order of the 3D map points included in the 3D map is meaningless. Therefore, when a decoder end is involved in decompressing or decoding the 3D map points, the order of the 3D map points is not limited. Decompression or decoding may be performed on the 3D map points based on the order of the bitstream.

[0086] Additionally, at the encoder end, processing the data of the 3D map includes encapsulation, which encapsulates the data to be encoded into a bitstream. The encapsulation may use any encoding algorithm, for example, entropy encoding. Entropy encoding is a lossless data compression method. Entropy encoding algorithms include, but are not limited to, Huffman encoding, arithmetic encoding, the LZ77 compression algorithm (Lempel-Ziv-Markov chain algorithm, LZMA), improved compression / decompression algorithms based on the Function Library algorithm for data compression (zlib), etc. Accordingly, at the decoder end, decompression may further include decapsulation, which deencapsulates the bitstream into data to be decompressed. The decapsulation may use a decoding algorithm, for example, entropy decoding, corresponding to the encoding algorithm used by the encoder end.

[0087] In a possible implementation, the data of the 3D map further includes reconstruction data of a plurality of area descriptors, any one of the plurality of area descriptors describing characteristics of some or all of the plurality of 3D map points.

[0088] In a possible implementation, the reconstruction data for any one of the plurality of 3D map points includes reconstruction data of a 3D map point descriptor and reconstruction data of a 3D map point spatial position.

[0089] In a possible implementation, the apparatus for decoding the 3D map is a cloud server or an electronic device. The transmission module is further configured to send a 3D map download request, where the 3D map download request includes location indication information; and receive a bitstream of the 3D map corresponding to the location indication information.

[0090] In a possible implementation, the apparatus for decoding the 3D map is a cloud server. The transmission module is particularly configured to receive a bitstream of the 3D map generated by the electronic device.

[0091] Decompression mode Several implementations of the module are as follows:

[0092] In a possible implementation, the decompression module includes a decapsulation module and a prediction module and / or a dequantization module. The decapsulation module is configured to process an input bitstream of the 3D map to output first data. The prediction module is configured to perform prediction on input residual data of the second data to output the second data. The dequantization module is configured to perform dequantization on input third data to output dequantized data of the third data. The first data is residual data of the second data or the third data, the second data is reconstruction data of the 3D map or the third data, and the dequantized data of the third data is reconstruction data of the 3D map.

[0093] Optionally, the decompression module comprises only a decapsulation module and a dequantization module or a prediction module.

[0094] Optionally, the decompression module comprises a prediction module, a dequantization module, and an encapsulation module.

[0095] In a possible implementation, the prediction module includes a first prediction module and a second prediction module; and / or the dequantization module includes a first dequantization module and a second dequantization module. The first prediction module is configured to perform prediction on input residual data of fourth data to output fourth data. The first dequantization module is configured to perform dequantization on input fifth data to output dequantized data of the fifth data. The fourth data is reconstructed data of one of the plurality of area descriptors or the fifth data, and the dequantized data of the fifth data is reconstructed data of one of the plurality of area descriptors. The second prediction module is configured to perform prediction on input residual data of sixth data to output sixth data. The second quantization module is configured to perform dequantization on input seventh data to output dequantized data of the seventh data. The sixth data is reconstructed data of one of the plurality of 3D map points or the seventh data, and the dequantized data of the seventh data is reconstructed data of one of the plurality of 3D map points.

[0096] Optionally, the decompression module comprises only a first prediction module and a second prediction module, or only a first dequantization module and a second dequantization module.

[0097] Optionally, the decompression module comprises a first prediction module, a second prediction module, a first dequantization module, and a second dequantization module.

[0098] In a possible implementation, the prediction module includes a first prediction module, a second prediction module, and a third prediction module; and / or the dequantization module includes a first dequantization module, a second dequantization module, and a third dequantization module. The first prediction module is configured to perform prediction on input residual data of eighth data and output the eighth data. The first dequantization module is configured to perform dequantization on input ninth data and output dequantized data of the ninth data. The eighth data is reconstructed data of one of the plurality of area descriptors or ninth data, and the dequantized data of the ninth data is reconstructed data of one of the plurality of area descriptors. The second prediction module is configured to perform prediction on input residual data of tenth data and output the tenth data. The second quantization module is configured to perform dequantization on input eleventh data and output dequantized data of the eleventh data. The tenth data is reconstructed data of a 3D map point descriptor of one of the plurality of 3D map points or eleventh data, and the dequantized data of the eleventh data is reconstructed data of a 3D map point descriptor of one of the plurality of 3D map points. The third prediction module is configured to perform prediction on input residual data of the twelfth data, and output the twelfth data. The third dequantization module is configured to perform dequantization on the input thirteenth data, and output the dequantized data of the thirteenth data. The twelfth data is reconstructed data of a 3D map point space location of one of the plurality of 3D map points or thirteenth data, and the dequantized data of the thirteenth data is reconstructed data of a 3D map point space location of one of the plurality of 3D map points.

[0099] Optionally, the decompression module has only the first prediction module, the second prediction module, and the third prediction module, or has only the first dequantization module, the second dequantization module, and the third dequantization module.

[0100] Optionally, the decompression module comprises a first prediction module, a second prediction module, a third prediction module, a first dequantization module, a second dequantization module, and a third dequantization module.

[0101] In a possible implementation, the decompression module has a first decompression sub-module and a second decompression sub-module. The first decompression sub-module is configured to decompress an input bitstream of fourteenth data to output fourteenth data. The second decompression sub-module is configured to decompress an input bitstream of fifteenth data to output fifteenth data. The fourteenth data is reconstructed data of one of the plurality of area descriptors, and the fifteenth data is reconstructed data of one of the plurality of 3D map points.

[0102] In a possible implementation, the first decompression submodule includes a first decapsulation module and a first prediction module and / or a first quantization module. The second decompression submodule includes a second decapsulation module and a second prediction module and / or a second quantization module. The first decapsulation module is configured to process an input bitstream of the 3D map to output sixteenth data. The first prediction module is configured to perform prediction on input residual data of the seventeenth data to output the seventeenth data. The first quantization module is configured to perform dequantization on input eighteenth data to output dequantized data of the eighteenth data. The sixteenth data is residual data of the seventeenth data or the eighteenth data, the seventeenth data is reconstructed data of one of the plurality of area descriptors or the eighteenth data, and the dequantized data of the eighteenth data is reconstructed data of one of the plurality of area descriptors. The second de-encapsulation module is configured to process the input bitstream of the 3D map to output nineteenth data. The second prediction module is configured to perform prediction on the input residual data of the twentieth data to output the twentieth data. The second de-quantization module is configured to perform de-quantization on the input twenty-first data to output de-quantized data of the twenty-first data. The nineteenth data is the residual data of the twenty-first data or the twenty-first data, the twenty-first data is reconstructed data of one of the plurality of 3D map points or the twenty-first data, and the de-quantized data of the twenty-first data is reconstructed data of one of the plurality of 3D map points.

[0103] In a possible implementation, the decompression module has a first decompression sub-module, a second decompression sub-module, and a third decompression sub-module. The first decompression sub-module is configured to decompress an input bitstream of 22 data to output 22 data. The second decompression sub-module is configured to decompress an input bitstream of 23 data to output 23 data. The third decompression sub-module is configured to decompress an input bitstream of 24 data to output 24 data. Uncompress the and outputting twenty-fourth data, wherein the twenty-second data is reconstruction data of one of the plurality of area descriptors, the twenty-third data is reconstruction data of a 3D map point descriptor of one of the plurality of 3D map points, and the twenty-fourth data is reconstruction data of a 3D map point spatial position of one of the plurality of 3D map points.

[0104] In a possible implementation, the first decompression submodule includes a first decapsulation module and a first prediction module and / or a first quantization module. The second decompression submodule includes a second decapsulation module and a second prediction module and / or a second quantization module. The third decompression submodule includes a third decapsulation module and a third prediction module and / or a third quantization module. The first decapsulation module is configured to process an input bitstream of the 3D map to output 25th data. The first prediction module is configured to perform prediction on input residual data of 26th data to output 26th data. The first quantization module is configured to perform dequantization on input 27th data to output dequantized data of 27th data. The 25th data is residual data of the 26th data or 27th data, the 26th data is reconstructed data of one of the plurality of area descriptors or the 27th data, and the dequantized data of the 27th data is reconstructed data of one of the plurality of area descriptors. The second de-encapsulation module is configured to process the input bitstream of 3D maps to output 28th data. The second prediction module is configured to perform prediction on the input residual data of the 29th data to output the 29th data. The second de-quantization module is configured to perform de-quantization on the input 30th data to output the dequantized data of the 30th data. The 28th data is residual data of the 29th data or the 30th data, the 29th data is reconstructed data of one 3D map point descriptor of the plurality of 3D map points or the 30th data, and the dequantized data of the 30th data is reconstructed data of one 3D map point descriptor of the plurality of 3D map points. The third de-encapsulation module is configured to process the input bitstream of the 3D map and output 31st data.The third prediction module is configured to perform prediction on input residual data of the 32nd data, and output the 32nd data. The third dequantization module is configured to perform dequantization on input 33rd data, and output dequantized data of the 33rd data. The 31st data is residual data of the 32nd data or the 33rd data, the 32nd data is reconstructed data of a 3D map point space position of one of the plurality of 3D map points, or the 33rd data, and the dequantized data of the 33rd data is reconstructed data of a 3D map point space position of one of the plurality of 3D map points.

[0105] According to a third aspect, the present application provides a method for decoding a 3D map, comprising: receiving a bitstream of a 3D map, wherein the 3D map includes a plurality of 3D map points; and decompressing the bitstream of the 3D map to obtain reconstruction data of the 3D map, wherein the reconstruction data of the 3D map includes reconstruction data of the plurality of 3D map points.

[0106] In this embodiment of the present application, the decompression module may support decompressing the compressed data of the 3D map. In other words, the encoding and decoding system may support compressing / decompressing the data of the 3D map so as to reduce the data volume of the 3D map. In a scenario where a 3D map needs to be transmitted, transmitting compressed data of the 3D map instead of transmitting the original data of the 3D map can reduce the data volume for transmission and further reduce the bandwidth occupied by the transmission, thereby improving the transmission efficiency of the 3D map.

[0107] In a possible implementation, the reconstruction data of the 3D map further includes reconstruction data of a plurality of area descriptors, any one of the plurality of area descriptors describing characteristics of some or all of the plurality of 3D map points.

[0108] In a possible implementation, the reconstruction data for any one of the plurality of 3D map points includes reconstruction data of a 3D map point descriptor and reconstruction data of a 3D map point spatial position.

[0109] In a possible implementation, the method further comprises transmitting a 3D map download request, wherein the 3D map download request includes location indication information, and receiving a bitstream of the 3D map includes receiving a bitstream of the 3D map corresponding to the location indication information.

[0110] In a possible implementation, receiving a bitstream of the 3D map comprises receiving a bitstream of the 3D map generated by an electronic device.

[0111] In a possible implementation, decompressing the bitstream of the 3D map to obtain reconstructed data of the 3D map includes processing the bitstream of the 3D map to obtain first data; and performing prediction on residual data of the second data to obtain the second data and / or performing dequantization on the third data to obtain dequantized data of the third data, where the first data is the residual data of the second data or the third data, the second data is the reconstructed data of the 3D map or the third data, and the dequantized data of the third data is the reconstructed data of the 3D map.

[0112] According to a fourth aspect, the present application provides an apparatus for decoding a 3D map, the apparatus comprising: a memory configured to store a 3D map in a bitstream format obtained through compression, where the 3D map includes a plurality of 3D map points, and data of the 3D map includes data of the plurality of 3D map points; and a decoder configured to decompress the 3D map in the bitstream format obtained through compression to obtain a reconstructed 3D map, where the reconstructed 3D map includes a plurality of 3D map points, and data of the reconstructed 3D map includes reconstructed data of the plurality of 3D map points.

[0113] In a possible implementation, the data of the reconstructed 3D map further includes reconstruction data of a plurality of area descriptors, any one of the plurality of area descriptors describing characteristics of some or all of the plurality of 3D map points.

[0114] In a possible implementation, the reconstruction data for any one of the plurality of 3D map points includes reconstruction data of a 3D map point descriptor and reconstruction data of a 3D map point spatial position.

[0115] In a possible implementation, 3D maps Decoding The apparatus for performing the process is a cloud server or an electronic device.

[0116] According to a fifth aspect, the present application provides a computer-readable storage medium comprising a computer program, which, when run on a computer, enables the computer to perform a method according to any one of the implementations of the third aspect.

[0117] According to a sixth aspect, the present application provides a computer program product comprising computer program code that, when executed on a computer, enables the computer to perform a method according to any one of the implementations of the third aspect.

[0118] According to a seventh aspect, the present application provides an encoded bitstream of a 3D map, wherein the encoded bitstream of the 3D map comprises a bitstream of a plurality of 3D map points, the 3D map comprising a plurality of 3D map points.

[0119] In a possible implementation, the encoded bitstream of the 3D map further comprises a bitstream of a plurality of area descriptors, any one of the plurality of area descriptors corresponding to at least one 3D map point of the plurality of 3D map points.

[0120] In a possible implementation, the bitstream for any one of the plurality of 3D map points includes a bitstream of 3D map point descriptors and a bitstream of 3D map point spatial locations.

[0121] In a possible implementation, the bitstream for any one of the plurality of 3D map points includes residual data for the 3D map point.

[0122] In a possible implementation, the bitstream of 3D map point descriptors includes residual data of 3D map point descriptors; and / or the bitstream of 3D map point space locations includes residual data of 3D map point space locations.

[0123] In a possible implementation, the bitstream of any one of the plurality of 3D map points further includes indication information of a reference 3D map point, and the plurality of 3D map points includes a reference 3D map point, which is a 3D map point that is encoded before the 3D map point is encoded.

[0124] In a possible implementation, the bitstream for any one of the plurality of 3D map points includes quantized data for the 3D map point.

[0125] In a possible implementation, the bitstream of 3D map point descriptors includes quantized data of 3D map point descriptors; and / or the bitstream of 3D map point space locations includes quantized data of 3D map point space locations.

[0126] In a possible implementation, the bitstream of any one of the multiple area descriptors includes the residual data of the area descriptor.

[0127] In a possible implementation, the bitstream of any one of the plurality of area descriptors further includes indication information of a reference area descriptor, and the plurality of area descriptors includes the reference area descriptor.

[0128] In a possible implementation, the bitstream for any one of the multiple area descriptors includes quantized data for the area descriptor. [Brief explanation of the drawings]

[0129] [Figure 1] 1 is a schematic diagram of an application architecture according to an embodiment of the present application;

[0130] [Figure 2] 1 is a schematic diagram of the structure of an electronic device 20 according to an embodiment of the present application;

[0131] [Figure 3] 1 is a schematic diagram of the structure of a server 30 according to an embodiment of the present application;

[0132] [Figure 4a] 1 is a schematic diagram of an application scenario according to an embodiment of the present application;

[0133] [Figure 4b] 1 is a schematic diagram of an application scenario according to an embodiment of the present application;

[0134] [Figure 4c] 1 is a schematic diagram of an application scenario according to an embodiment of the present application;

[0135] [Figure 4d] 1 is a schematic diagram of an application scenario according to an embodiment of the present application;

[0136] [Figure 4e] 1 is a schematic diagram of an application scenario according to an embodiment of the present application;

[0137] [Figure 4f]1 is a schematic diagram of an application scenario according to an embodiment of the present application;

[0138] [Figure 4g] 1 is a schematic diagram of a user interface displayed by an electronic device according to an embodiment of the present application;

[0139] [Figure 5] 1 is a diagram of the structure of an apparatus 50 for decoding 3D maps according to an embodiment of the present application;

[0140] [Figure 6a] 1 is a diagram of the structure of an apparatus 60 for decoding 3D maps according to an embodiment of the present application;

[0141] [Figure 6b] 1 is a diagram of the structure of an apparatus 60 for decoding 3D maps according to an embodiment of the present application;

[0142] [Figure 6c] 1 is a diagram of the structure of an apparatus 60 for decoding 3D maps according to an embodiment of the present application;

[0143] [Figure 6d] 1 is a diagram of the structure of an apparatus 60 for decoding 3D maps according to an embodiment of the present application;

[0144] [Figure 6e] 1 is a diagram of the structure of an apparatus 60 for decoding 3D maps according to an embodiment of the present application;

[0145] [Figure 7a] 1 is a diagram of the structure of an apparatus 70 for decoding 3D maps according to an embodiment of the present application;

[0146] [Figure 7b] 1 is a diagram of the structure of an apparatus 70 for decoding 3D maps according to an embodiment of the present application;

[0147] [Figure 8a] 1 is a diagram of the structure of an apparatus 80 for decoding 3D maps according to an embodiment of the present application;

[0148] [Figure 8b] 1 is a diagram of the structure of an apparatus 80 for decoding 3D maps according to an embodiment of the present application;

[0149] [Figure 9] 9 is a flowchart of a process 900 of a method for decoding a 3D map according to an embodiment of the present application;

[0150] [Figure 10] 1 is a diagram of the structure of an apparatus 100 for decoding 3D maps according to an embodiment of the present application;

[0151] [Figure 11a] 1 is a diagram of an example of the structure of an encoded bitstream of a 3D map according to an embodiment of the present application;

[0152] [Figure 11b] 1 is a diagram of an example of the structure of an encoded bitstream of a 3D map according to an embodiment of the present application;

[0153] [Figure 11c] 1 is a diagram of an example of the structure of an encoded bitstream of a 3D map according to an embodiment of the present application;

[0154] [Figure 11d] 1 is a diagram of an example of the structure of an encoded bitstream of a 3D map according to an embodiment of the present application; and

[0155] [Figure 11e] FIG. 2 is a diagram of an example of the structure of an encoded bitstream of a 3D map according to an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0156] In order to clarify the objectives, technical solutions and advantages of the present application, the technical solutions of the present application will be described clearly and completely below with reference to the accompanying drawings. It is clear that the described embodiments are only a part, not all, of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present application.

[0157] In the embodiments, claims, and accompanying drawings of this specification, terms such as "first," "second," etc. are intended to distinguish and describe only and are not to be understood as indicating or suggesting relative importance or sequence. Furthermore, the terms "comprise," "have," and any variations thereof are intended to cover non-exclusive inclusions, e.g., the inclusion of a series of steps or units. A method, system, product, or device is not necessarily limited to explicitly recited steps or units, but may include other steps or units that are not explicitly recited and are inherent to the process, method, product, or device.

[0158] In this application, "at least one (item)" should be understood to mean one or more, and "multiple" should be understood to mean two or more. The term "and / or" describes an association relationship between associated objects and indicates that three relationships may exist. For example, "A and / or B" may indicate three cases: only A is present, only B is present, and both A and B are present. A and B may be singular or plural. The symbol " / " typically indicates an "or" relationship between associated objects. "At least one of the following items" or similar expressions indicates any combination of the items, including any combination of one of the items or more than one of the items. For example, "at least one of a, b, or c" may refer to a, b, c, a and B, a and c, b and c, or a, b, and c, where a, b, and c may be singular or plural.

[0159] FIG. 1 is a schematic diagram of an application architecture according to an embodiment of the present application. As shown in FIG. 1, the application architecture includes multiple electronic devices and a server. The multiple electronic devices may include a first electronic device and one or more second electronic devices (two second electronic devices are used as an example in FIG. 1). The one or more second electronic devices are electronic devices other than the first electronic device. Communication may be performed between the multiple electronic devices and the server, and between the multiple electronic devices. For example, any device in the application architecture may communicate with another device via wireless fidelity (Wi-Fi®) communication, Bluetooth® communication, or cellular second / third / fourth / fifth generation (2G / 3G / 4G / 5G) communication, etc. It should be understood that other communication methods, including future communication methods, may also be used between the server and the electronic devices. This is not particularly limited herein. It should be noted that the term "one or more second electronic devices" in this embodiment of the present application is merely used to refer to electronic devices other than the first electronic device, and is not limited to whether the multiple electronic devices are of the same type.

[0160] The electronic device may be various types of devices provided with a camera and a display component. For example, the electronic device may be a terminal device such as a mobile phone, a tablet computer, a notebook computer, or a video recorder (FIG. 1 shows a mobile phone as an example of an electronic device). Alternatively, the electronic device may be a device used for interaction in a virtual scenario, including VR glasses, an AR device, an MR interaction device, etc. Alternatively, the electronic device may be a wearable electronic device such as a smart watch or a smart band. Alternatively, the electronic device may be a device carried in a transport device such as a vehicle, an unmanned vehicle, an unmanned aerial vehicle, or an industrial robot. The specific form of the electronic device is not particularly limited in the embodiments of the present application.

[0161] Additionally, an electronic device may also be referred to as user equipment (UE), a subscriber station, mobile unit, subscriber unit, radio unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, terminal device, access terminal, mobile terminal, wireless terminal, smart terminal, remote terminal, handheld terminal, user agent, mobile client, client, or another suitable terminology.

[0162] The server may be one or more physical servers (in FIG. 1, one physical server is used as an example), or may be a computer cluster, or may be a virtual machine or cloud server in a cloud computing scenario, etc.

[0163] In this embodiment of the present application, a virtual scenario application (APP), such as a VR application, an AR application, or an MR application, may be installed on an electronic device, and the VR application, the AR application, or the MR application may be activated based on a user operation (e.g., tap, touch, slide, shake, or voice control). The electronic device may collect visual information of any object in the environment by using a sensor, and then display a virtual object on a display component based on the collected visual information. The virtual object may be a virtual object (i.e., an object in a virtual environment) in a VR scenario, an AR scenario, or an MR scenario.

[0164] In this embodiment of the present application, a navigation, detection, or control application may be installed on the electronic device, and the corresponding application is activated based on the operation and control of the user or a pre-set program. The electronic device may execute applications such as route planning, object detection, and transport device operation and control based on the pose and other status information of the electronic device in the current environment.

[0165] Visual information in embodiments of the present application includes, but is not limited to, image video collected by a camera (without depth information), image video collected by a depth sensor (with depth information), data collected by a LiDAR, and data collected by a millimeter wave radar (RaDAR).

[0166] It should be noted that in this embodiment of the present application, the virtual scenario application in the electronic device may be an application built into the electronic device, or may be an application provided by a third-party service provider and installed by the user, although this is not particularly limited herein.

[0167] In this embodiment of the present application, a simultaneous localization and mapping (SLAM) system may be further configured for the electronic device. The SLAM system generates a map of a completely unknown environment and can use this map to perform positioning, pose (position and posture) determination, navigation, etc. In this embodiment of the present application, the map generated by the SLAM system is referred to as a SLAM map. The SLAM map may be understood as a map drawn by the SLAM system based on environmental information collected by a collection device. The collection device may include a visual information collection device and an inertial measurement unit (IMU) in the electronic device. The visual information collection device may include, for example, a camera, a depth camera, a lidar, and a millimeter-wave radar. The IMU may include, for example, sensors such as a gyroscope and an accelerometer.

[0168] In the embodiment of the present application, the SLAM map is also referred to as a 3D map. Note that the 3D map includes, but is not limited to, the SLAM map, and may further include a three-dimensional map generated by using another technique. This is not particularly limited in the embodiment of the present application.

[0169] In a possible implementation, the 3D map may include a plurality of 3D map points, and accordingly, the data of the 3D map may include data of a plurality of 3D map points, which may be points of interest or points having significant features in the environment.

[0170] Possible methods for acquiring 3D map points include performing photography using multiple devices such as lidar, aerial photography from the viewing angle of an unmanned aerial vehicle (tilt photography), a high-definition panoramic camera, and a high-definition industrial camera, and extracting features from an oriented accelerated segment test (FAST) and rotational binary robust independent elementary features (BRIEF) from the data acquired through photography by the above-mentioned devices. ), notes The goal is to extract 3D map points by using methods such as oriented FAST and rotated BRIEF (ORB), scale invariant feature transform (SIFT), and speeded-up robust feature (SURF), BRIEF, binary robust invariant scalable keypoints (BRISK), fast retina keypoint (FREAK), or repeatable and reliable detector and descriptor (R2D2).

[0171] The data for a 3D map point may include:

[0172] (1) 3D map point descriptor

[0173] A 3D map point descriptor is a vector used to represent the local features of a 3D map point. In visual positioning algorithms, the 3D map point descriptor is used for matching between 3D map points. One possible method is to calculate the distance (which may be Euclidean distance, dot product distance, Hamming distance, etc.) between two 3D map point descriptors; and if the distance is smaller than a threshold, the two 3D map points are considered to match.

[0174] (2) 3D map point space position

[0175] The 3D map point spatial position may be represented by using X, Y, and Z on three-dimensional spatial axes, or by using longitude, latitude, and altitude, or by using polar coordinates, etc. The method for representing the 3D map point spatial position is not particularly limited in the embodiments of the present application. The 3D map point spatial position may be the absolute position of the 3D map point, or the relative position of the 3D map point. For example, the center position of the entire area is used as the origin, and all 3D map point spatial positions are positions shifted with respect to the spatial position of the origin.

[0176] In an embodiment of the present application, a number may be assigned to each 3D map point and written into the 3D map data, or the storage order of multiple 3D map points in memory may be used to implicitly indicate the number of the 3D map point. Note that the order of multiple 3D map points included in a 3D map is meaningless. Therefore, the numbers may be considered as identifiers used to identify the 3D map points in order to distinguish them. However, the numbers are not intended to limit the order of the multiple 3D map points. For example, if a 3D map includes three 3D map points numbered 1, 2, and 3, respectively, the three 3D map points may be processed in the order 1, 2, and 3, or 3, 2, and 1, or 2, 1, and 3, etc.

[0177] In a possible implementation, the 3D map data further includes a plurality of area descriptors, any one of which describes characteristics of some or all of the plurality of 3D map points. Specifically, for any one of the plurality of area descriptors, the area descriptor may describe characteristics of some or all of the plurality of 3D map points. In this case, the area descriptor and the 3D map point have a one-to-many relationship. Each characteristic of the plurality of 3D map points may be described by some or all of the area descriptors of the plurality of area descriptors. In this case, the 3D map point and the area descriptor have a one-to-many relationship. It can be seen that the plurality of area descriptors and the plurality of 3D map points have a many-to-many relationship. Methods for generating area descriptors include, but are not limited to, traditional methods such as bag of words (BOW) and vector of locally aggregated descriptors (VLAD), and novel methods based on NetVLAD or artificial intelligence (AI). Similarly, multiple area descriptors may be identified by numbers to distinguish between the multiple area descriptors, but similarly, the numbers are not intended to limit the order of the multiple area descriptors.

[0178] In a possible implementation, the 3D map data further includes correspondences between 3D map points and area descriptors, which clearly describe which 3D map point any area descriptor corresponds to, and which area descriptor any 3D map point corresponds to.

[0179] Optionally, the aforementioned correspondence may be explicitly described by using a correspondence table between area descriptor numbers and 3D map point numbers. For example, a 3D map includes three area descriptors numbered T1 to T3 and six 3D map points numbered P1 to P6. The correspondence table is shown in Table 1 to is shown. Table 1 [Table 1]

[0180] It should be noted that Table 1 is an example of a correspondence table between area descriptor numbers and 3D map point numbers. The correspondence table may alternatively be presented in another format or manner, which is not particularly limited in this application.

[0181] Optionally, the aforementioned correspondences may alternatively be described implicitly by using the storage locations of the area descriptors and 3D map points, for example, T1 is stored in memory first, then the data of P1, P2, and P3 are stored; then T2 is stored, then the data of P2 and P3 are stored; finally, T3 is stored, then the data of P3, P4, P5, and P6 are stored.

[0182] FIG. 2 is a schematic diagram of the structure of an electronic device 20 according to one embodiment of the present application. As shown in FIG. 2, the electronic device 20 may be at least one of the first electronic device and one or more second electronic devices in the embodiment shown in FIG. 1. It should be understood that the structure shown in FIG. 2 does not constitute a specific limitation on the electronic device 20. In some other embodiments of the present application, the electronic device 20 may include more or fewer components than those shown in FIG. 2, combine some components, separate some components, or have a different component configuration. The components shown in FIG. 2 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0183] The electronic device 20 may include a chip 21, a memory 22 (one or more computer-readable storage media), a user interface 23, a display component 24, a camera 25, sensors 26, a positioning module 27 configured to perform device positioning, and a transceiver 28 configured to perform communications. These components may communicate with each other by using one or more buses 29.

[0184] One or more processors 211, a clock module 212, and a power management module 213 may be integrated into the chip 21. The clock module 212 integrated into the chip 21 is mainly configured to provide timers required for data transmission and timing control of the processor 211. The timers may implement clock functions for data transmission and timing control. The processor 211 may perform operations based on instruction operation codes and timing signals and generate operation control signals to complete control of instruction fetching and instruction execution. The power management module 213 integrated into the chip 21 is mainly configured to provide stable and highly accurate voltages for the chip 21 and other components of the electronic device 20.

[0185] The processor 211 may also be referred to as a central processing unit (CPU). Specifically, the processor 211 may include one or more processing units. For example, the processor 211 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU), etc. The different processing units may be independent components or may be integrated into one or more processors.

[0186] In possible implementations, the processor 211 may include one or more interfaces, which may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, a universal serial bus (USB) port, etc.

[0187] The memory 22 may be connected to the processor 211 through a bus 29 or may be Sa 211 The memory 22 may be configured to store multiple groups of various software programs and / or instructions, which may be combined together. The memory 22 may include high-speed random access memory (e.g., cache) or may include non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 22 may store an operating system, such as an embedded operating system such as Android (Android), Apple (iOS), Microsoft Windows operating system (Windows), or a UNIX-like operating system (Linux). The memory 22 may further store data, such as image data, point cloud data, 3D map data, pose data, coordinate system transformation information, and map update information. The memory 22 may further store computer-executable program code. The computer-executable program code includes instructions, such as SLAM system communication program instructions and related program instructions. The memory 22 may further store one or more applications, such as a virtual scenario application for AR / VR / MR, a 3D map application, an image management application, and a navigation and control application. The memory 22 may further store a user interface program. The user interface program may vividly display the content of the application, such as virtual objects in a virtual scenario for AR / VR / MR, using a graphical operation interface, present the content using the display component 24, and receive control operations performed by a user on the application using input controls such as menus, dialog boxes, or buttons.

[0188] The user interface 23 may be, for example, a touch panel that detects a user's operation command on the touch panel, or a keypad, physical buttons, or a mouse.

[0189] The electronic device 20 may include one or more display components 24. The electronic device 20 may implement display functions by using the display component 24, a graphics processing unit (GPU), an application processor (AP), and the like within the chip 21 in cooperation with each other. The GPU is a microprocessor for implementing image processing and is connected to the display component 24 and the application processor. The GPU performs mathematical and geometric calculations for graphics rendering. The display component 24 may display interface content output by the electronic device 20, such as images, videos, and the like in virtual scenarios such as AR / VR / MR. The interface content may include interfaces such as running applications and system-level menus, and specifically may include the following interface elements: input interface elements such as buttons, text entry boxes, scrollbars, and menus; and output interface elements such as windows, labels, images, videos, and animations.

[0190] The display component 24 may be a display panel, a lens (e.g., VR glasses), a projection screen, etc. The display panel may also be referred to as a display screen, and may be, for example, a touch screen, a flexible screen, a curved screen, etc., or may be another optical component. It should be understood that the display screen of the electronic device in the embodiments of the present application may be a touch screen, a flexible screen, a curved screen, or a screen of another form. In other words, the display screen of the electronic device has the function of displaying images, and the specific material and shape of the display screen are not particularly limited.

[0191] For example, if display component 24 includes a display panel, the display panel may use a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLed, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. Furthermore, in a possible implementation, a touch panel in user interface 23 may be coupled to the display panel in display component 24. For example, the touch panel may be disposed below the display panel, and the touch panel may be configured to detect tactile pressure acting on the display panel when a user inputs a touch operation (e.g., a tap, a slide, or a touch) using the display panel, and the display panel is configured to display content.

[0192] The camera 25 may be a monocular camera, a binocular camera, or a depth camera and is configured to photograph / record the environment and obtain images / video images. The images / video images collected by the camera 25 may be used as input data for a SLAM system, for example, or the images / video may be displayed using the display component 24.

[0193] In a possible implementation, the camera 25 may be considered as a sensor. The images collected by the camera 25 may be in IMG format or another format type, which is not particularly limited in the embodiment of the present application.

[0194] The sensor 26 may be configured to collect data related to status changes (e.g., rotation, shaking, movement, or vibration) of the electronic device 20. The data collected by the sensor 26 may be used as input data for the SLAM system. The sensor 26 may include one or more sensors, such as an inertial measurement unit (IMU) and a time-of-flight (TOF) sensor. The IMU may include sensors such as a gyroscope and an accelerometer. The gyroscope is configured to measure the angular velocity of the electronic device when the electronic device moves, and the accelerometer is configured to measure the acceleration of the electronic device when the electronic device moves. The TOF sensor may include a light emitter and a light receiver. The light emitter is configured to emit light, such as laser light, infrared light, or radar waves, outward. The light receiver is configured to detect reflected light, such as reflected laser light, infrared light, or radar waves.

[0195] It should be noted that the sensor 26 may further include many other sensors, such as an inertial sensor, a barometer, a magnetometer, and a wheel speedometer, etc. This is not particularly limited in the embodiment of the present application.

[0196] The positioning module 27 is configured to implement physical positioning of the electronic device 20, for example, to obtain an initial position of the electronic device 20. The positioning module 27 may include one or more of a Wi-Fi positioning module, a Bluetooth positioning module, a base station positioning module, and a satellite positioning module. To assist in the positioning, a global navigation satellite system (GNSS) may be disposed in the satellite positioning module. The GNSS includes, but is not limited to, the BeiDou system, the global positioning system (GPS) system, the global navigation satellite system (GLONASS) system, and the Galileo navigation satellite system (Galileo) system.

[0197] The transceiver 28 is configured to facilitate communication between the electronic device 20 and another device (e.g., a server or another electronic device). The transceiver 28 integrates a transmitter and a receiver configured to transmit and receive radio frequency signals, respectively. In a specific implementation, the transceiver 28 includes, but is not limited to, an antenna system, a radio frequency (RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec (CODEC) chip, a subscriber identity module (SIM) card, a storage medium, etc. In a possible implementation, the transceiver 28 may alternatively be implemented on a separate chip. The transceiver 28 supports at least one data network communication in 2G / 3G / 4G / 5G, etc., and / or supports at least one of the following short-range wireless communication methods: Bluetooth (BT) communication, wireless fidelity (Wi-Fi) communication, near-field communication (NFC), infrared (IR) wireless communication, ultra wideband (UWB) communication, and ZigBee (registered trademark) protocol communication.

[0198] In this embodiment of the present application, the processor 211 operates program code stored in the memory 22 to perform various functional applications and data processing of the electronic device 20 .

[0199] Figure 3 is a schematic diagram of the structure of a server 30 according to one embodiment of the present application. As shown in Figure 3, the server 30 may be the server in the embodiment shown in Figure 1. The server 30 includes a processor 301, a memory 302 (one or more computer-readable storage media), and a transceiver 303. These components may communicate with each other by using one or more buses 304.

[0200] The processor 301 may be one or more CPUs. When the processor 301 is a CPU, the CPU may be a single-core CPU or a multi-core CPU.

[0201] The memory 302 may be connected to or coupled to the processor 301 via a bus 304 and configured to store various groups of program codes and / or instructions and data (e.g., map data and pose data). In a specific implementation, the memory 302 includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a compact disc read-only memory (CD-ROM), etc.

[0202] The transceiver 303 is mainly an integration of a receiver and a transmitter, where the receiver is configured to receive data (e.g., requests or images) sent by the electronic device, and the transmitter is configured to send data (e.g., map data or pose data) to the electronic device.

[0203] 3 is only an example provided in this embodiment of the present application, and it should be understood that the server 30 may further include more components than those shown in the figure, which is not particularly limited in this embodiment of the present application.

[0204] In this embodiment of the present application, the processor 301 runs program code stored in the memory 302 to perform various functional applications and data processing of the server 30 .

[0205] The term "coupled" as used in the present application means a direct connection or a connection through one or more intermediate components or circuits.

[0206] 4a is a schematic diagram of an application scenario according to one embodiment of the present application. As shown in FIG. 4a, in this application scenario, an electronic device collects visual information by using a sensor, and determines the current pose of the electronic device by referring to the visual information and a 3D map from a server.

[0207] The 3D map is provided by a server. Specifically, the server generates a 3D map, then compresses the 3D map, and sends the compressed data of the 3D map to the electronic device. After receiving the compressed data of the 3D map, the electronic device performs decompression to obtain reconstructed data of the 3D map, and determines the current pose of the electronic device by referring to the collected visual information and the 3D map. The pose is position information of the electronic device, and may be an absolute pose in a world coordinate system or a pose relative to a point in the environment.

[0208] In this embodiment of the present application, the server may generate a 3D map in advance, compress the 3D map, and then store the compressed data of the 3D map locally, thereby saving storage space. Furthermore, the server may send the compressed data of the 3D map to another device, such as cloud storage.

[0209] 1. The server generates a 3D map, compresses the 3D map to obtain compressed data of the 3D map, and stores the compressed data locally.

[0210] The server compresses the 3D map to save local storage space.

[0211] 2. The electronic device sends a map download request to the server. The map download request can be triggered in two ways.

[0212] (1) A user starts a 3D map application installed on an electronic device, and the application uploads location information obtained based on GPS positioning or Wi-Fi positioning to a server corresponding to the application. The upload operation may trigger a map download request. Since the uploaded content includes location information, the server may perform a preliminary estimation based on the location information and send compressed data of a 3D map of the area to which the positioning point indicated by the location information belongs to the electronic device. The range of the area to which the positioning point indicated by the location information belongs may be preset. For example, the area to which the positioning point belongs may be an administrative region of any level (including a county, city, country, or administrative region) in which the positioning point is located, or may be a circular area centered on the positioning point and using a specified distance as a radius.

[0213] (2) A user starts a 3D map application installed on an electronic device and actively enters or selects an area on the application. For example, the user actively enters "xx business center" or selects "Street A" from a list of "Street A, Street B, and Street C." The user's aforementioned operation may trigger a map download request. Whether the user enters or selects a geographic location, the server accordingly transmits compressed data of a 3D map of the geographic location to the electronic device.

[0214] It should be understood that in this embodiment of the present application, in addition to the two methods described above, other methods may be used to trigger a map download request. For example, the electronic device automatically detects whether a condition for downloading a 3D map or starting a 3D map download is met, or the electronic device starts a 3D map download and requests to download a 3D map of an area range from a server when it detects a change in ambient light or an environment change. The size of the area range is not particularly limited.

[0215] 3. The server sends the compressed data of the 3D map to the electronic device.

[0216] 4. Electronic devices collect visual information.

[0217] It should be noted that steps 3 and 4 are independent of each other and the order in which they are performed is not limited.

[0218] 5. The electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.

[0219] 6. The electronic device performs positioning in a 3D map based on the visual information to obtain a pose corresponding to the visual information.

[0220] After receiving the compressed data of the 3D map, the electronic device does not need to immediately decompress the compressed data, but only needs to decompress the compressed data to obtain reconstructed data of the 3D map before performing positioning based on visual information. For example, a user may pre-download compressed data of the 3D map of an area range by downloading an "offline map," and decompress the compressed data of the 3D map only when positioning is required.

[0221] 4b is a schematic diagram of an application scenario according to an embodiment of the present application. As shown in FIG. 4b, in this application scenario, an electronic device collects visual information by using a sensor, and a server determines the current pose of the electronic device by referring to the visual information from the electronic device and a 3D map.

[0222] The 3D map is provided by a server. Specifically, the server generates the 3D map, then compresses the 3D map, and locally stores the compressed data of the 3D map. When receiving visual information from the electronic device, the server performs decompression to obtain reconstructed data of the 3D map, and references the visual information and the 3D map to determine the current pose of the electronic device.

[0223] 1. The server generates a 3D map, compresses the 3D map to obtain compressed data of the 3D map, and stores the compressed data locally.

[0224] 2. Electronic devices collect visual information.

[0225] 3. The electronic device transmits the visual information to the server.

[0226] 4. The server decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.

[0227] It should be understood that the server compresses the 3D map to save storage space.

[0228] 5. The server performs positioning in the 3D map based on the visual information to obtain a pose corresponding to the visual information.

[0229] 6. The server sends the pause to the electronic device.

[0230] 4c is a schematic diagram of an application scenario according to an embodiment of the present application. As shown in FIG. 4c, in this application scenario, an electronic device collects visual information by using a sensor, and determines a current pose of the electronic device by referring to the visual information and a 3D map.

[0231] The 3D map is provided by the electronic device. Specifically, the electronic device generates the 3D map, then compresses the 3D map, and locally stores the compressed data of the 3D map. When visual information is collected, the electronic device performs decompression to obtain reconstructed data of the 3D map, and determines the current pose of the electronic device by referring to the collected visual information and the 3D map.

[0232] 1. The electronic device generates a 3D map, compresses the 3D map to obtain compressed data of the 3D map, and locally stores the compressed data.

[0233] It should be understood that the electronic device may compress the 3D map to save storage space.

[0234] 2. Electronic devices collect visual information by using sensors.

[0235] 3. The electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.

[0236] 4. The electronic device performs positioning in a 3D map based on the visual information to obtain a pose corresponding to the visual information.

[0237] FIG. 4d is a schematic diagram of an application scenario according to an embodiment of the present application. Figure 4d As shown, in this application scenario, the second electronic device collects visual information by using a sensor and determines the current pose of the second electronic device by referencing the visual information and a 3D map from the server.

[0238] The 3D map is generated by a first electronic device. Specifically, the first electronic device generates the 3D map, compresses the 3D map, and then transmits the compressed 3D map data to a server. The server then transmits the compressed 3D map data to a second electronic device. The second electronic device performs decompression to obtain reconstructed data of the 3D map and determines the current pose of the second electronic device by referring to the collected visual information and the 3D map.

[0239] In this embodiment of the present application, the first electronic device may generate a 3D map in advance, compress the 3D map, and then transmit the compressed data of the 3D map to the server, thereby reducing the transmission bandwidth.

[0240] 1. A first electronic device generates a 3D map and compresses the 3D map to obtain compressed data of the 3D map.

[0241] 2. The first electronic device sends compressed data of the 3D map to the server.

[0242] The first electronic device compresses the 3D map and then transmits the compressed data of the 3D map, thereby reducing transmission bandwidth and improving transmission efficiency.

[0243] 3. The second electronic device sends a map download request to the server.

[0244] The second electronic device may send a map download request based on the trigger scheme shown in FIG. 4a.

[0245] 4. The server sends the compressed data of the 3D map to the second electronic device.

[0246] 5. The second electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.

[0247] 6. The second electronic device collects visual information by using a sensor.

[0248] 7. The second electronic device performs positioning in the 3D map based on the visual information and obtains a pose corresponding to the visual information.

[0249] 4e is a schematic diagram of an application scenario according to an embodiment of the present application. As shown in FIG. 4e, in this application scenario, the second electronic device collects visual information by using a sensor, and the server determines the current pose of the second electronic device by referring to the visual information from the second electronic device and the 3D map from the first electronic device.

[0250] The 3D map is generated by a first electronic device. Specifically, the first electronic device generates a 3D map, compresses the 3D map, and then transmits the compressed data of the 3D map to a server. The server performs decompression to obtain reconstructed data of the 3D map, and determines the current pose of the second electronic device by referring to the visual information from the second electronic device and the 3D map.

[0251] 1. A first electronic device generates a 3D map and compresses the 3D map to obtain compressed data of the 3D map.

[0252] 2. The first electronic device sends compressed data of the 3D map to the server.

[0253] 3. The second electronic device collects visual information by using a sensor.

[0254] 4. The second electronic device sends a positioning request to the server, where the positioning request carries the visual information.

[0255] 5. The server decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.

[0256] 6. The server performs positioning in the 3D map based on the visual information to obtain a pose corresponding to the visual information.

[0257] 7. The server transmits the pose obtained through positioning to the second electronic device.

[0258] 4f is a schematic diagram of an application scenario according to an embodiment of the present application. As shown in FIG. 4f, in this application scenario, the second electronic device collects visual information by using a sensor, and determines the current pose of the second electronic device by referring to the visual information and the 3D map from the first electronic device.

[0259] The 3D map is generated by a first electronic device. Specifically, the first electronic device generates the 3D map, compresses the 3D map, and then transmits the compressed data of the 3D map to a second electronic device. The second electronic device performs decompression to obtain reconstructed data of the 3D map and determines a current pose of the second electronic device by referring to the collected visual information and the 3D map from the first electronic device.

[0260] 1. A first electronic device generates a 3D map, compresses the 3D map to obtain compressed data of the 3D map, and locally stores the compressed data.

[0261] 2. The second electronic device sends a map download request to the first electronic device.

[0262] 3. The first electronic device transmits the compressed data of the 3D map to the second electronic device.

[0263] 4. The second electronic device decompresses the compressed data of the 3D map to obtain reconstructed data of the 3D map.

[0264] 5. The second electronic device collects visual information by using a sensor.

[0265] 6. The second electronic device performs positioning in a 3D map based on the visual information to obtain a pose corresponding to the visual information.

[0266] The positioning algorithm used in the embodiment shown in Figures 4a-4f may include:

[0267] (1) Area descriptors to be retrieved are extracted from the visual information, where the algorithm used to extract the area descriptors to be retrieved is consistent with the algorithm for extracting area descriptors from 3D maps.

[0268] (2) From the visual information, the 3D map points to be extracted are extracted, and the 3D map point spatial positions to be extracted and the 3D map point descriptors to be extracted are obtained, where the algorithm for extracting the 3D map point descriptors to be extracted is consistent with the algorithm for extracting the 3D map point descriptors from the 3D map.

[0269] (3) Based on the area descriptor to be extracted, extraction is performed on a plurality of area descriptors contained in the data of the 3D map to obtain a plurality of candidate area descriptors.

[0270] In an embodiment of the present application, the distance between the area descriptor to be retrieved and each area descriptor in the plurality of area descriptors may be calculated. The distance may include Hamming distance, Manhattan distance, Euclidean distance, etc. Then, at least one area descriptor that satisfies a condition (e.g., the distance is smaller than a threshold) is selected as a candidate area descriptor.

[0271] (4) Matching between the 3D map point descriptor to be retrieved and the 3D map point descriptors corresponding to the multiple candidate area descriptors is performed separately, which means calculating the similarity between the 3D map point descriptor to be retrieved and the 3D map point descriptors corresponding to the multiple candidate area descriptors separately to find the most similar 3D map point.

[0272] (5) The pose of the electronic device is calculated based on the discovered 3D map points by using a pose solving algorithm such as perspective-n-point (PnP) camera pose estimation and efficient perspective-n-point camera pose estimation (EPnP).

[0273] The embodiments shown in Figures 4a-4f are all related to compressing 3D maps. The embodiments of the present application provide multiple device frameworks for performing the aforementioned compression. The multiple device frameworks are described below.

[0274] In any one of the application scenarios in FIGS. 4a to 4f, positioning is performed based on the 3D map in an embodiment of the present application to obtain the current pose of the electronic device. The pose may be applied to fields such as AR navigation, AR human-computer interaction, driving assistance, and autonomous driving. For example, pose-based AR navigation is used as an example. FIG. 4g is a schematic diagram of a user interface displayed by an electronic device according to an embodiment of the present application. The electronic device may display the user interface shown in FIG. 4g based on the pose. The user interface may include a navigation arrow indication pointing toward the conference room 2, which may be a virtual object obtained from a server based on the pose or obtained locally based on the pose. The user interface may further include visual information collected by a sensor, such as the building shown in FIG. 4g. The user goes to the conference room 2 by referring to the user interface of the electronic device shown in FIG. 4g.

[0275] It should be noted that the reconstructed data of the 3D map in the embodiment of the present application obtained through decompression may also be referred to as reconstructed data of the 3D map.

[0276] Fig. 5 is a diagram of the structure of an apparatus 50 for decoding a 3D map according to one embodiment of the present application. As shown in Fig. 5, the decoding apparatus 50 may be used in a server or electronic device in the above-mentioned embodiments, in particular in a device that needs to receive and decompress the 3D map, such as the electronic device in the embodiment shown in Fig. 4a, the second electronic device in the embodiments shown in Fig. 4d and Fig. 4f, or the server in the embodiment shown in Fig. 4e.

[0277] In this embodiment of the present application, the apparatus 50 for decoding a 3D map comprises a transmission module 51 and a decompression module 52. The transmission module 51 is configured to receive a bitstream of a 3D map, where the 3D map includes a plurality of 3D map points. The decompression module 52 is configured to decompress the bitstream of the 3D map to obtain reconstruction data of the 3D map, where the reconstruction data of the 3D map includes reconstruction data of a plurality of 3D map points. It can be seen that the data of the transmission module 51 is the bitstream of the 3D map, and the decompression module 52 is configured to decompress the bitstream of the 3D map to obtain reconstruction data of the 3D map.

[0278] The decompression module 52 may support decompression of compressed data of the 3D map. In other words, the encoding and decoding system may support compression / decompression of data of the 3D map so as to reduce the data volume of the 3D map. In a scenario where a 3D map needs to be transmitted, transmitting compressed data of the 3D map instead of transmitting the original data of the 3D map can reduce the data volume for transmission, for example, compress the data volume from TB level to GB level, and further reduce the bandwidth occupied by the transmission, thereby improving the transmission efficiency of the 3D map.

[0279] It should be noted that in this embodiment of the present application, decompression may include at least one of dequantization and prediction.

[0280] Quantization refers to mapping data to be processed to one or more quantization indexes, where each quantization index corresponds to a quantization center. The number of bits of a quantization index is usually significantly smaller than the number of bits of the original data, thereby saving storage or transmission bandwidth. Quantization methods include, but are not limited to, scalar quantization, vector quantization, and product quantization. Based on this, dequantization is the inverse of the aforementioned quantization. Specifically, reconstructed data corresponding to the original data is restored based on one or more quantization indexes, and the number of bits of the reconstructed data is greater than the number of bits of the quantization indexes.

[0281] Prediction means performing prediction by using residual data of the processed data and the data to be processed to obtain the data to be processed. The selection of reference data may be agreed upon in advance. For example, previously processed data is fixedly used as reference data, in which case the reference data does not need to be identified in the bitstream. For another example, any processed data may be used, in which case the identification information of the reference data needs to be written into the bitstream, and includes the number of the reference data or other information that can be used to infer the reference data.

[0282] In the above description of the 3D map data, it can be seen that the order of the 3D map points included in the 3D map is meaningless. Therefore, when a decoder end is involved in decompressing or decoding the 3D map points, the order of the 3D map points is not limited. Decompression or decoding may be performed on the 3D map points based on the order of the bitstream.

[0283] Additionally, at the encoder end, processing the data of the 3D map includes encapsulation, which encapsulates the data to be encoded into a bitstream. The encapsulation may use any encoding algorithm, for example, entropy encoding. Entropy encoding is a lossless data compression method. Entropy encoding algorithms include, but are not limited to, Huffman encoding, arithmetic encoding, the LZ77 compression algorithm (Lempel-Ziv-Markov chain algorithm, LZMA), improved compression / decompression algorithms based on the Function Library algorithm for data compression (zlib), etc. Accordingly, at the decoder end, decompression may further include decapsulation, which deencapsulates the bitstream into data to be decompressed. The decapsulation may use a decoding algorithm, for example, entropy decoding, corresponding to the encoding algorithm used by the encoder end.

[0284] It should be understood that quantization is a lossy compression method, so the data of the 3D map obtained after decompression by the decoder end is not completely the same as the original data of the 3D map before compression by the encoder end. Based on this, in this embodiment of the present application, the data of the 3D map output by the decompression module 52 may be referred to as reconstructed data of the 3D map. For the reconstructed data of the 3D map, please refer to the above description of the data of the 3D map. The details will not be described again here.

[0285] Based on the embodiment shown in FIG. 5, the embodiments of the present application provide examples of several possible structures of the decompression module, and the following embodiments are used for illustration.

[0286] In a possible implementation, Fig. 6a is a diagram of the structure of an apparatus 60-1 for decoding a 3D map according to one embodiment of the present application. As shown in Fig. 6a, the decoding apparatus 60-1 may be used in a server or electronic device in the above-mentioned embodiments, in particular in a device that needs to receive and decompress the 3D map, such as the electronic device in the embodiment shown in Fig. 4a, the second electronic device in the embodiments shown in Fig. 4d and Fig. 4f, or the server in the embodiment shown in Fig. 4e.

[0287] 5, in this embodiment of the present application, the decoding device 60-1 includes a transmission module 61-1 and a decompression module 62-1. The decompression module 62-1 includes a decapsulation module 621-1 and a dequantization module 623-1. The details are as follows:

[0288] The de-encapsulation module 621-1 is configured to process an input bitstream of the 3D map to output first data, and the de-quantization module 623-1 is configured to perform de-quantization on the input third data to output de-quantized data of the third data.

[0289] The aforementioned first data is the third data, and the dequantized data of the third data is the reconstructed data of the 3D map. It can be seen that the input data of the decapsulation module 621-1 is the bitstream of the 3D map, and the output data is the quantized data of the 3D map; and the input data of the dequantization module 623-1 is the quantized data of the 3D map output by the decapsulation module 621-1, and the output data is the reconstructed data of the 3D map.

[0290] In a possible implementation, Fig. 6b is a diagram of the structure of an apparatus 60-2 for decoding a 3D map according to one embodiment of the present application. As shown in Fig. 6b, the decoding apparatus 60-2 may be used in a server or electronic device in the above-mentioned embodiments, in particular in a device that needs to receive and decompress the 3D map, such as the electronic device in the embodiment shown in Fig. 4a, the second electronic device in the embodiments shown in Fig. 4d and Fig. 4f, or the server in the embodiment shown in Fig. 4e.

[0291] 5, in this embodiment of the present application, the decoding device 60-2 includes a transmission module 61-2 and a decompression module 62-2. The decompression module 62-2 includes a decapsulation module 621-2 and a prediction module 622-1. Details are as follows:

[0292] The decapsulation module 621-2 is configured to process an input bitstream of the 3D map to output first data, and the prediction module 622-1 is configured to perform prediction on residual data of the input second data to output the second data.

[0293] The first data is residual data of the second data, and the second data is reconstruction data of the 3D map. It can be seen that the input data of the decapsulation module 621-2 is the bitstream of the 3D map, and the output data is the residual data of the 3D map; and the input data of the prediction module 622-1 is the residual data of the 3D map output by the decapsulation module 621-2, and the output data is reconstruction data of the 3D map.

[0294] In a possible implementation, Fig. 6c is a diagram of the structure of an apparatus 60-3 for decoding a 3D map according to one embodiment of the present application. As shown in Fig. 6c, the decoding apparatus 60-3 may be used in a server or electronic device in the above-mentioned embodiments, in particular in a device that needs to receive and decompress the 3D map, such as the electronic device in the embodiment shown in Fig. 4a, the second electronic device in the embodiments shown in Fig. 4d and Fig. 4f, or the server in the embodiment shown in Fig. 4e.

[0295] 5, in this embodiment of the present application, the decoding device 60-3 includes a transmission module 61-3 and a decompression module 62-3. The decompression module 62-3 includes a decapsulation module 621-3, a prediction module 622-2, and a dequantization module 623-2, and the details are as follows:

[0296] The de-encapsulation module 621-3 is configured to process an input bitstream of the 3D map to output first data. The prediction module 622-2 is configured to perform prediction on residual data of the input second data to output the second data. The de-quantization module 623-2 is configured to perform de-quantization on the input third data to output de-quantized data of the third data.

[0297] The first data is residual data of the second data or third data, the second data is reconstructed data of the 3D map or third data, and the dequantized data of the third data is reconstructed data of the 3D map. It can be seen that the input data of the decapsulation module 621-3 is the bitstream of the 3D map, and the output data is residual data of the 3D map; the input data of the prediction module 622-2 is the residual data of the 3D map output by the decapsulation module 621-3, and the output data is quantized data of the 3D map; and the input data of the dequantization module 623-2 is the quantized data of the 3D map output by the prediction module 622-2, and the output data is reconstructed data of the 3D map.

[0298] According to the above description of the 3D map data, the reconstruction data of the 3D map may include reconstruction data of a plurality of area descriptors and reconstruction data of a plurality of 3D map points. Therefore, in the embodiment shown in FIGS. 6A to 6C, a separate module in the decompression module, different from the decapsulation module, may be divided into a first module configured to process the area descriptors and a second module configured to process the 3D map point data. For example, the dequantization module may include a first dequantization module and a second dequantization module, and the prediction module may include a first prediction module and a second prediction module. The difference between the first module and the second module obtained after differentiating from the above embodiment lies in that the input data of the first module and the input data of the second module correspond to the area descriptors and the 3D map point data, respectively.

[0299] Fig. 6d is a diagram of the structure of an apparatus 60-4 for decoding a 3D map according to one embodiment of the present application. As shown in Fig. 6d, the decoding apparatus 60-4 may be used in a server or electronic device in the above-mentioned embodiments, in particular in a device that needs to receive and decompress the 3D map, such as the electronic device in the embodiment shown in Fig. 4a, the second electronic device in the embodiments shown in Fig. 4d and Fig. 4f, or the server in the embodiment shown in Fig. 4e.

[0300] 6c, in this embodiment of the present application, the decoding device 60-4 includes a transmission module 61-4 and a decompression module 62-4. The decompression module 62-4 includes a decapsulation module 621-4, a prediction module 622-3, and a dequantization module 623-3. The prediction module 622-3 includes a first prediction module 622a and a second prediction module 622b. The dequantization module 623 includes a first dequantization module 623a and a second dequantization module 623b.

[0301] The transmission module 61-4 is configured to receive a bitstream of the 3D map.

[0302] The input data of the decapsulation module 621-4 is a bitstream of 3D maps, and the decapsulation module 621-4 performs decapsulation on the bitstream of 3D maps to obtain residual data of area descriptors and residual data of 3D map points. The input data of the first prediction module 622a is the residual data of area descriptors, and the output data is quantized data of the area descriptors. The input data of the first quantization module 623a is quantized data of the area descriptors, and the output data is reconstructed data of the area descriptors. The input data of the second prediction module 622b is residual data of 3D map points, and the output data is quantized data of 3D map points. The input data of the second quantization module 623b is quantized data of 3D map points, and the output data is reconstructed data of 3D map points.

[0303] According to the above description of the 3D map data, the 3D map reconstruction data may include reconstruction data of a plurality of area descriptors, reconstruction data of a 3D map point descriptor, and reconstruction data of 3D map point space positions of a plurality of 3D map points. Therefore, in the embodiment shown in Figures 6a to 6c, a separate module in the decompression module, different from the decapsulation module, may be divided into a first module configured to process the area descriptors, a second module configured to process the 3D map point descriptors, and a third module configured to process the 3D map point space positions. For example, the dequantization module may include a first dequantization module, a second dequantization module, and a third dequantization module, and the prediction module may include a first prediction module, a second prediction module, and a third prediction module. The difference between the first module, the second module, and the third module obtained after differentiation from the above-mentioned embodiments is that the input data of the first module, the input data of the second module, and the input data of the third module correspond to area descriptors, 3D map point descriptors, and 3D map point space locations, respectively.

[0304] Fig. 6e is a diagram of the structure of an apparatus 60-5 for decoding a 3D map according to one embodiment of the present application. As shown in Fig. 6e, the decoding apparatus 60-5 may be used in a server or electronic device in the above-mentioned embodiments, in particular in a device that needs to receive and decompress the 3D map, such as the electronic device in the embodiment shown in Fig. 4a, the second electronic device in the embodiments shown in Fig. 4d and Fig. 4f, or the server in the embodiment shown in Fig. 4e.

[0305] 6c, in this embodiment of the present application, the decoding device 60-5 includes a transmission module 61-5 and a decompression module 62-5. The decompression module 62-5 includes a decapsulation module 621-5, a prediction module 622-4, and a dequantization module 623-4. The prediction module 622-4 includes a first prediction module 622a, a second prediction module 622b, and a third prediction module 622c. The dequantization module 623-4 includes a first dequantization module 623a, a second dequantization module 623b, and a third dequantization module 623c.

[0306] The transmission module 61-5 is configured to receive a bitstream of the 3D map.

[0307] The input data of the decapsulation module 621-5 is a bitstream of a 3D map, and the decapsulation module 621-5 performs decapsulation on the bitstream of the 3D map to obtain residual data of the area descriptor, residual data of the 3D map point descriptor, and residual data of the 3D map point spatial position. The input data of the first prediction module 622a is the residual data of the area descriptor, and the output data is quantized data of the area descriptor. The input data of the first quantization module 623a is the quantized data of the area descriptor, and the output data is reconstructed data of the area descriptor. The input data of the second prediction module 622b is the residual data of the 3D map point descriptor, and the output data is quantized data of the 3D map point descriptor. The input data of the second quantization module 623b is the quantized data of the 3D map point descriptor, and the output data is reconstructed data of the 3D map point descriptor. The third prediction module Le 622c The input data is the residual data of the 3D map point space positions, and the output data is the quantized data of the 3D map point space positions. Le 623c The input data is quantized data of 3D map point space positions, and the output data is reconstructed data of 3D map point space positions.

[0308] 6d and 6e respectively show exemplary structures of an apparatus for decoding a 3D map, which are obtained based on the content included in the 3D map bitstream. However, these structures do not constitute a limitation on the decoding apparatus. The decoding apparatus may have more diverse modules than those in the embodiment shown in FIG. 6d or 6e. For example, in relation to the embodiment shown in FIG. 6a or 6b, a different structure is obtained based on the content included in the 3D map bitstream. In the embodiment shown in FIG. 6a to 6c, the prediction module and the dequantization module are in an "and / or" relationship, i.e., the decompression module may include one of the prediction module and the dequantization module, or may include both the prediction module and the dequantization module. Therefore, when distinguishing between the first module and the second module, the processing methods for the module for processing the area descriptor and the module for processing the 3D map point data may be set independently and do not need to be completely consistent. For example, the area descriptor may be processed by using a first prediction module and a first dequantization module, and the 3D map point data may be processed by using a second prediction module. When distinguishing between the first module, the second module, and the third module, the processing methods for the module for processing the area descriptor, the module for processing the 3D map point descriptor, and the module for processing the 3D map point spatial position may be set independently and do not need to be completely consistent. For example, the area descriptor may be processed by using a first prediction module and a first dequantization module, the 3D map point descriptor may be processed by using a second prediction module, and the 3D map point spatial position may be processed by using a third dequantization module. The specific implementation of the decompression module is not particularly limited in this application.

[0309] According to the above description of the 3D map data, the 3D map reconstruction data may include reconstruction data of a plurality of area descriptors and reconstruction data of a plurality of 3D map points.

[0310] In a possible implementation, Fig. 7a is a diagram of the structure of an apparatus 70-1 for decoding a 3D map according to one embodiment of the present application. As shown in Fig. 7a, the decoding apparatus 70-1 may be used in a server or electronic device in the above-mentioned embodiments, in particular in a device that needs to receive and decompress the 3D map, such as the electronic device in the embodiment shown in Fig. 4a, the second electronic device in the embodiments shown in Fig. 4d and Fig. 4f, or the server in the embodiment shown in Fig. 4e.

[0311] 5, in this embodiment of the present application, the decoding device 70-1 includes a transmission module 71-1 and a decompression module 72-1. The decompression module 72-1 includes a first decompression sub-module 721-1 and a second decompression sub-module 722-1. The details are as follows:

[0312] The first decompression sub-module 721-1 is configured to decompress an input bitstream of fourteenth data and output fourteenth data. The second decompression sub-module 722-1 is configured to decompress an input bitstream of fifteenth data and output fifteenth data.

[0313] The fourteenth data is reconstructed data of one of the plurality of area descriptors, and the fifteenth data is reconstructed data of one of the plurality of 3D map points. It can be seen that the input data of the first decompression sub-module 721-1 is a bit stream of area descriptors, and the output data is reconstructed data of the area descriptors; the input data of the second decompression sub-module 722-1 is a bit stream of 3D map points, and the output data is reconstructed data of the 3D map points. The 3D map bit stream includes a bit stream of the plurality of area descriptors and a bit stream of the plurality of 3D map points.

[0314] For the first decompression sub-module 721-1 and the second decompression sub-module 722-1, please refer to the structures of the decompression modules in the embodiments shown in Figures 6a to 6e. It should be noted that the first decompression sub-module 721-1 and the second decompression sub-module 722-1 are independent of each other and may have the same or different structures. That is, the first decompression sub-module 721-1 configured to process the area descriptor and the second decompression sub-module 722-1 configured to process the data of the 3D map points may have the same or different structures. Accordingly, the decompression step performed on the area descriptor may be the same as or different from the decompression step performed on the data of the 3D map points.

[0315] For example, the first decompression sub-module includes a first decapsulation module and a first prediction module. In this way, after a bitstream of an area descriptor is input to the first decompression sub-module, the first decapsulation module first decapsulates the bitstream to obtain residual data of the area descriptor, and then the first prediction module processes the residual data to obtain reconstructed data of the area descriptor. The second decompression sub-module includes a second decapsulation module, a second prediction module, and a second dequantization module. In this way, after a bitstream of 3D map points is input to the second decompression sub-module, the second decapsulation module first processes the bitstream to obtain residual data of the 3D map points, then the second prediction module processes the residual data to obtain quantized data of the 3D map points, and then the second quantization module processes the quantized data to obtain reconstructed data of the 3D map points.

[0316] In another example, the first decompression sub-module includes a first decapsulation module, a first prediction module, and a first dequantization module. In this way, after a bitstream of area descriptors is input to the first decompression sub-module, the first decapsulation module first processes the bitstream to obtain residual data of the area descriptor, the first prediction module processes the residual data to obtain quantized data of the area descriptor, and the first quantization module processes the quantized data to obtain reconstructed data of the area descriptor. The second decompression sub-module includes a second decapsulation module and a second prediction module. In this way, after a bitstream of 3D map points is input to the second decompression sub-module, the second decapsulation module processes the description to obtain residual data of the 3D map points, and the second prediction module processes the residual data to obtain reconstructed data of the 3D map points.

[0317] It should be understood that the structures of the first decompression sub-module and the second decompression sub-module are described above as an example. However, this does not constitute a limitation on the structures of the first decompression sub-module and the second decompression sub-module. These two sub-modules may include more or fewer modules than those in this example. For details, please refer to the structure of the decompression module in the embodiment shown in Figures 6a to 6e. This is not particularly limited in the embodiment of the present application.

[0318] In a possible implementation, Fig. 7b is a diagram of the structure of an apparatus 70-2 for decoding a 3D map according to one embodiment of the present application. As shown in Fig. 7b, the decoding apparatus 70-2 may be used in a server or electronic device in the above-mentioned embodiments, in particular in a device that needs to receive and decompress the 3D map, such as the electronic device in the embodiment shown in Fig. 4a, the second electronic device in the embodiments shown in Fig. 4d and Fig. 4f, or the server in the embodiment shown in Fig. 4e.

[0319] 7a, in this embodiment of the present application, the decoding device 70-2 includes a transmission module 71-2 and a decompression module 72-2. The decompression module 72-2 includes a first decompression sub-module 721-2 and a second decompression sub-module 722-2. The first decompression sub-module 721-2 includes a first decapsulation module 7211, a first prediction module 7212, and a first quantization module 7213. The second decompression sub-module 722-2 includes a second decapsulation module 7221, a second prediction module 7222, and a second quantization module 7223. Details are as follows:

[0320] The first decapsulation module 7211 is configured to process an input bitstream of the area descriptor to obtain residual data of the area descriptor. The first prediction module 7212 is configured to perform prediction on the input residual data of the area descriptor to obtain quantized data of the area descriptor. The first dequantization module 7213 is configured to perform dequantization on the input quantized data of the area descriptor to obtain reconstructed data of the area descriptor. The second decapsulation module 7221 is configured to process an input bitstream of the 3D map points to obtain residual data of the 3D map points. The second prediction module 7222 is configured to perform prediction on the input residual data of the 3D map points to obtain quantized data of the 3D map points. The second quantization module 7223 is configured to perform dequantization on the input quantized data of the 3D map points to obtain reconstructed data of the 3D map points.

[0321] It should be understood that the structures of the first and second decompression sub-modules are described as an example in the embodiment shown in FIG. 7b. However, this does not constitute a limitation on the structures of the first and second decompression sub-modules. These two sub-modules may include more or fewer modules than those in this example. For details, please refer to the structure of the decompression module 62 in the embodiment shown in FIGS. 6a to 6e. This is not particularly limited in the embodiment of the present application.

[0322] According to the above description of the 3D map data, the 3D map reconstruction data may include reconstruction data of a plurality of area descriptors, reconstruction data of 3D map point descriptors, and reconstruction data of 3D map point space positions of a plurality of 3D map points.

[0323] In a possible implementation, Fig. 8a is a diagram of the structure of an apparatus 80-1 for decoding a 3D map according to one embodiment of the present application. As shown in Fig. 8a, the decoding apparatus 80-1 may be used in a server or electronic device in the above-mentioned embodiments, in particular in a device that needs to receive and decompress the 3D map, such as the electronic device in the embodiment shown in Fig. 4a, the second electronic device in the embodiments shown in Fig. 4d and Fig. 4f, or the server in the embodiment shown in Fig. 4e.

[0324] 5, in this embodiment of the present application, the decoding device 80-1 includes a transmission module 81-1 and a decompression module 82-1. The decompression module 82-1 includes a first decompression sub-module 821-1, a second decompression sub-module 822-1, and a third decompression sub-module 823-1. Details are as follows:

[0325] The first decompression sub-module 821-1 is configured to decompress an input bitstream of 22 data to output 22 data. The second decompression sub-module 822-1 is configured to decompress an input bitstream of 23 data to output 23 data. The third decompression sub-module 823-1 is configured to decompress an input bitstream of 24 data to output 24 data.

[0326] The 22nd data is reconstructed data of one of the plurality of area descriptors, the 23rd data is reconstructed data of a 3D map point descriptor of one of the plurality of 3D map points, and the 24th data is reconstructed data of a 3D map point space location of one of the plurality of 3D map points. It can be seen that the input data of the first decompression sub-module 821-1 is a bit stream of area descriptors, and the output data is reconstructed data of the area descriptor; the input data of the second decompression sub-module 822-1 is a bit stream of 3D map point descriptors, and the output data is reconstructed data of the 3D map point descriptors; and the input data of the third decompression sub-module 823-1 is a bit stream of 3D map point space locations, and the output data is reconstructed data of the 3D map point space locations. The 3D map bit stream includes a bit stream of multiple area descriptors, a bit stream of multiple 3D map point descriptors, and a bit stream of multiple 3D map point space locations.

[0327] For the first decompression sub-module 821-1, the second decompression sub-module 822-1, and the third decompression sub-module 823-1, please refer to the structures of the decompression modules in the embodiments shown in Figures 6a to 6e. Note that the first decompression sub-module 821-1, the second decompression sub-module 822-1, and the third decompression sub-module 823-1 are independent of each other and may have the same or different structures. That is, the first decompression sub-module 821-1 configured to process the area descriptor, the second decompression sub-module 822-1 configured to process the 3D map point descriptor, and the third decompression sub-module 823-1 configured to process the 3D map point space position may have the same or different structures. Accordingly, the decompression step performed on the area descriptor, the decompression step performed on the 3D map point descriptor, and the decompression step performed on the 3D map point space position may be the same or different.

[0328] In a possible implementation, Fig. 8b is a diagram of the structure of an apparatus 80-2 for decoding a 3D map according to one embodiment of the present application. As shown in Fig. 8b, the decoding apparatus 80-2 may be used in a server or electronic device in the aforementioned embodiments, in particular in a device that needs to receive and decompress the 3D map, such as the electronic device in the embodiment shown in Fig. 4a, the second electronic device in the embodiments shown in Fig. 4d and Fig. 4f, or the server in the embodiment shown in Fig. 4e.

[0329] 8a, in this embodiment of the present application, the decoding device 80-2 includes a transmission module 81-2 and a decompression module 82-2. The decompression module 82-2 includes a first decompression sub-module 821-2, a second decompression sub-module 822-2, and a third decompression sub-module 823-2. The first decompression sub-module 821-2 includes a first decapsulation module 8211, a first prediction module 8212, and a first quantization module 8213. The second decompression sub-module 822-2 includes a second decapsulation module 8221, a second prediction module 8222, and a second quantization module 8223. The third decompression sub-module 823-2 includes a third decapsulation module 8231, a third prediction module 8232, and a third quantization module 8233. Details are as follows.

[0330] The first decapsulation module 8211 is configured to process an input bitstream of the area descriptor to obtain residual data of the area descriptor. The first prediction module 8212 is configured to perform prediction on the input residual data of the area descriptor to obtain quantized data of the area descriptor. The first dequantization module 8213 is configured to perform dequantization on the input quantized data of the area descriptor to obtain reconstructed data of the area descriptor. The second decapsulation module 8221 is configured to process an input bitstream of the 3D map point descriptor to obtain residual data of the 3D map point descriptor. The second prediction module 8222 is configured to perform prediction on the input residual data of the 3D map point descriptor to obtain quantized data of the 3D map point descriptor. The second quantization module 8223 is configured to perform dequantization on the input quantized data of the 3D map point descriptor to obtain reconstructed data of the 3D map point descriptor. The third de-encapsulation module 8231 is configured to process the input bitstream of 3D map point space locations to obtain residual data for the 3D map point space locations. The third prediction module 8232 is configured to perform prediction on the input residual data for the 3D map point space locations to obtain quantized data for the 3D map point space locations. The third de-quantization module 8233 is configured to perform de-quantization on the input quantized data for the 3D map point space locations to obtain reconstructed data for the 3D map point space locations.

[0331] It should be understood that the structures of the first decompression sub-module, the second decompression sub-module, and the third decompression sub-module are described as an example in the embodiment shown in FIG. 8b. However, this does not constitute a limitation on the structures of the first decompression sub-module, the second decompression sub-module, and the third decompression sub-module. These three sub-modules may include more or fewer modules than those in this example. For details, please refer to the structure of the decompression module 62 in the embodiment shown in FIGS. 6a to 6e. This is not particularly limited in the embodiment of the present application.

[0332] 9 is a flowchart of a process 900 of a method for decoding a 3D map according to one embodiment of the present application. As shown in FIG. 9, the process 900 may be performed by a decoding device in the above-described embodiment. The process 900 is described as a series of steps or operations. It should be understood that the steps or operations of the process 900 may be performed in various orders and / or simultaneously, and are not limited to the order of execution shown in FIG. 9. It is assumed that the process 900, including the following steps, is performed to decompress the 3D map bitstream in the decoding device and process the currently processed 3D map bitstream to obtain the 3D map data:

[0333] Step 901: Receive a bitstream of a 3D map.

[0334] The decoding device may receive the bitstream of the 3D map by using a communication link.

[0335] Step 902: Decompress the bitstream of the 3D map to obtain reconstruction data of the 3D map.

[0336] Please refer to the above description of the 3D map and the reconstruction data of the 3D map, and the details will not be described again here.

[0337] In this embodiment of the present application, the decompression performed on the bitstream of the 3D map may include decapsulation, prediction, and / or dequantization. For the aforementioned processes, please refer to the descriptions in the previous embodiments. The details will not be described again here.

[0338] Fig. 10 is a diagram of the structure of an apparatus 100 for decoding a 3D map according to one embodiment of the present application. As shown in Fig. 10, the decoding apparatus 100 may be used in a server or electronic device in the above-mentioned embodiments, in particular in a device that needs to store and decompress a 3D map, such as the server in the embodiment shown in Fig. 4b or the electronic device in the embodiment shown in Fig. 4c.

[0339] In this embodiment of the present application, an apparatus 100 for decoding a 3D map comprises: a memory 101 configured to store a 3D map in a bitstream format obtained through compression, where the 3D map includes a plurality of 3D map points, and data of the 3D map includes data of the plurality of 3D map points; and a decoder 102 configured to decompress the 3D map in the bitstream format obtained through compression to obtain a reconstructed 3D map, where the reconstructed 3D map includes a plurality of 3D map points, and data of the reconstructed 3D map includes reconstructed data of the plurality of 3D map points. It can be seen that the memory 101 stores the 3D map in the bitstream format obtained through compression, and the input data of the decoder 102 is the 3D map in the bitstream format obtained through compression, and the output data is the reconstructed 3D map.

[0340] The decoder 102 may support decompression of the 3D map in a bitstream format obtained through compression. That is, the device for decoding the 3D map may support compression / decompression of the data of the 3D map to reduce the data volume of the 3D map. In a scenario where the 3D map needs to be stored, instead of storing the original data of the 3D map, storing the 3D map in a bitstream format obtained through compression can reduce the area occupied by storing the data of the 3D map.

[0341] In this embodiment of the present application, the decoder 102 refers to the decompression module in the embodiment shown in Figures 5 to 8b, with the difference being that the decoder does not include a decapsulation module.

[0342] 11a is a diagram of an example of the structure of a 3D map encoded bitstream according to one embodiment of the present application. As shown in FIG. 11a, the 3D map encoded bitstream may comprise a file header, area descriptor 1, 3D map point 1, area descriptor 2, 3D map point 2, ..., area descriptor m, 3D map point n, and other information.

[0343] In this embodiment of the present application, it is assumed that the 3D map includes m area descriptors and n 3D map points, and the area descriptors and 3D map points in the encoded bitstream may be arranged in a cross manner.

[0344] 11b is a diagram of an example of the structure of a 3D map encoded bitstream according to one embodiment of the present application. As shown in FIG. 11b, the 3D map encoded bitstream may comprise a file header, area descriptor 1, area descriptor 2, ..., area descriptor m, 3D map point 1, 3D map point 2, ..., 3D map point n, and other information.

[0345] In this embodiment of the present application, it is assumed that the 3D map comprises m area descriptors and n 3D map points. The area descriptors and 3D map points in the encoded bitstream are arranged separately: m area descriptors followed by n 3D map points, or n 3D map points followed by m area descriptors.

[0346] It should be understood that the encoded bitstream of the 3D map in this embodiment of the present application may alternatively use a different structure, with the particular structure being associated with the data content included in the 3D map, for example, the encoded bitstream of the 3D map does not comprise one or more of a file header, an area descriptor, and other information.

[0347] 11c is a diagram of an example of the structure of an encoded bitstream of a 3D map according to one embodiment of the present application. As shown in FIG. 11c, the encoded bitstream of a 3D map may include area descriptor 1, area descriptor 2, and area descriptor 3. This encoded bitstream structure corresponds to the aforementioned case where the decompression module does not include a prediction module. Therefore, no reference information for prediction needs to be written into the bitstream.

[0348] 11d is a diagram of an example of the structure of an encoded bitstream of a 3D map according to one embodiment of the present application. As shown in FIG. 11d, the encoded bitstream of a 3D map may comprise area descriptor 1, an index of a reference area descriptor of area descriptor 2, and residual data of area descriptor 2. This encoded bitstream structure corresponds to the aforementioned case where the decompression module has a prediction module. Therefore, reference information for prediction, for example, an index of a reference area descriptor of area descriptor 2, needs to be written into the bitstream. In addition, the residual data of area descriptor 2 is written into area descriptor 2 instead of the original data.

[0349] 11e is a diagram of an example of the structure of an encoded bitstream of a 3D map according to an embodiment of the present application. As shown in FIG. 11e, the encoded bitstream of the 3D map may comprise 3D map point 1, 3D map point 2, and 3D map point 3. 3D map point 2 is used as an example. The bitstream of 3D map point 2 includes a 3D map point descriptor of 3D map point 2 and a 3D map point space location of 3D map point 2. When the structure of the encoded bitstream corresponds to the above-mentioned case where the decompression module has a prediction module, the bitstream of the 3D map point descriptor includes a reference index of the 3D map point descriptor and residual data of the 3D map point descriptor, and the bitstream of the 3D map point space location includes a reference index of the 3D map point space location and residual data of the 3D map point space location.

[0350] It should be understood that in the 3D map encoded bitstream in this embodiment of the present application, the content included in the 3D map bitstream varies depending on different compression / decompression processes performed on the 3D map data. For example, if the compression / decompression process includes quantization / dequantization, the 3D map bitstream includes quantization data; if the compression / decompression process includes quantization / dequantization and prediction, the 3D map bitstream includes reference indexes and residual data; or if the compression / decompression process includes prediction, the 3D map bitstream includes reference indexes and residual data. Therefore, the structure of the 3D map encoded bitstream is not particularly limited in this embodiment of the present application.

[0351] In the implementation process, the steps in the above-described method embodiments may be completed by an instruction-integrated logic circuit in the form of hardware or software in a processor. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or another programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc. The steps of the disclosed method in the embodiments of the present application may be directly presented as being performed and completed by a hardware encoding processor or by a combination of hardware and software modules in the encoding processor. The software modules may be located in a storage medium well-known in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, or a register. The storage medium is located in the memory. The processor reads the information in the memory and completes the steps of the method described above in combination with the processor hardware.

[0352] The memories mentioned in the above embodiments may be volatile memories or nonvolatile memories, or may include both volatile and nonvolatile memories. The nonvolatile memories may be read-only memories (ROMs), programmable read-only memories (PROMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), or flash memories. The volatile memories may be random access memories (RAMs) used as external buffers. By way of example and not limitation, many types of RAM may be used, such as static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM), and direct rambus dynamic random access memory (direct rambus RAM, DR RAM). It should be noted that memory in the systems and methods described herein includes, but is not limited to, these and any other suitable types of memory.

[0353] Those skilled in the art can recognize that, in combination with the examples described in the embodiments disclosed herein, the units and algorithm steps may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to go beyond the scope of this application.

[0354] For the purpose of simple and easy description, it can be clearly understood by those skilled in the art that the detailed operation processes of the above-mentioned systems, devices and units may refer to the corresponding processes in the above-mentioned method embodiments, and the details will not be described again here.

[0355] In the embodiments provided herein, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the device embodiments described above are merely examples. For example, the division of multiple units is merely a logical division of function, and other divisions may be used in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some functions may be ignored or not performed. In addition, the shown or discussed mutual couplings or direct couplings or communication connections may be implemented by using some interfaces. Indirect couplings or communication connections between devices or units may be implemented in electronic, mechanical, or other forms.

[0356] The units described as separate parts may or may not be physically separate, and the parts shown as units may or may not be physical units, located in one location or distributed over multiple network units, some or all of which may be selected according to actual requirements to achieve the objectives of the solutions in the embodiments.

[0357] Additionally, the functional units in the embodiments of the present application may be integrated into one processing unit, each of the units may exist physically alone, or two or more units may be integrated into one unit.

[0358] When these functions are implemented in the form of software functional units and sold or used as independent products, the functions may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application may be essentially implemented, or a part that contributes to the prior art, or a part of the technical solutions may be implemented in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for instructing a computer device (such as a personal computer, a server, a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The storage medium includes any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0359] The above description is merely a specific implementation of the present application, and the scope of protection of the present application is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application shall be included in the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the scope of protection of the claims. 。 [Other possible items] [Item 1] A codec system for a 3D map, comprising an encoding device and a decoding device, wherein the encoding device is communicatively connected to the decoding device; the encoding device is configured to compress data of a 3D map to obtain a bitstream of the 3D map, and send the bitstream of the 3D map to the decoding device, where the 3D map includes a plurality of 3D map points, and the data of the 3D map includes data of the plurality of 3D map points; and The decoding device is configured to receive the bitstream of the 3D map and decompress the bitstream of the 3D map to obtain reconstruction data of the 3D map. system. [Item 2] The encoding device is a cloud server and the decoding device is an electronic device; or the encoding device is a first electronic device and the decoding device is a second electronic device; the decoding device is further configured to send a 3D map download request to the decoding device, where the 3D map download request includes location indication information; and The encoding device is further configured to receive the 3D map download request and to send a bitstream of the 3D map corresponding to the position indication information to the decoding device according to the 3D map download request. Item 1. The system of item 1. [Item 3] The encoding device is an electronic device and the decoding device is a cloud server; and The encoding device is particularly configured to transmit the bitstream of the 3D map to the decoding device after the 3D map is generated. Item 1 or 2. The system according to item 1 or 2. [Item 4] 4. The system of claim 1, wherein the data of the 3D map further includes a plurality of area descriptors, any one of which describes characteristics of some or all of the plurality of 3D map points. [Item 5] 5. The system of claim 1 or 4, wherein the data for any one of the plurality of 3D map points includes a 3D map point descriptor and a 3D map point spatial location. [Item 6] 6. The system of any one of items 1 to 5, wherein the encoding device is further configured to generate the 3D map. [Item 7] 7. The system of any one of items 1 to 6, wherein the decoding device is further configured to perform positioning based on the 3D map. [Item 8] 1. An apparatus for decoding a 3D map, comprising a transmission module and a decompression module, wherein: the transmission module is configured to receive a bitstream of a 3D map, where the 3D map includes a plurality of 3D map points; and The decompression module is configured to decompress the bitstream of the 3D map to obtain reconstruction data of the 3D map, where the reconstruction data of the 3D map includes reconstruction data of the plurality of 3D map points. Device. [Item 9] The reconstruction data of the 3D map further includes reconstruction data of a plurality of area descriptors, any one of the plurality of area descriptors describing characteristics of some or all of the plurality of 3D map points. Item 9. The device according to item 8. [Item 10] Item 10. The apparatus of item 9, wherein the reconstruction data for any one of the plurality of 3D map points includes reconstruction data for a 3D map point descriptor and reconstruction data for a 3D map point spatial position. [Item 11] The apparatus for decoding the 3D map is a cloud server or an electronic device; and The transmission module is further configured to: send a 3D map download request, where the 3D map download request includes location indication information; and receive a bitstream of the 3D map corresponding to the location indication information. 11. The device according to any one of items 8 to 10. [Item 12] the device for decoding a 3D map is a cloud server; and The transmission module is particularly configured to receive the bitstream of the 3D map generated by an electronic device. 11. The device according to any one of items 8 to 10. [Item 13] The decompression module comprises a decapsulation module, and a prediction module and / or a dequantization module; the de-encapsulation module is configured to process an input bitstream of the 3D map to output first data; The prediction module is configured to perform prediction on input residual data of second data and output the second data; The dequantization module is configured to perform dequantization on the input third data and output dequantized data of the third data; and The first data is the residual data of the second data or the third data, the second data is the reconstructed data of the 3D map or the third data, and the dequantized data of the third data is the reconstructed data of the 3D map. 13. The device according to any one of items 8 to 12. [Item 14] The prediction module includes a first prediction module and a second prediction module; and / or the dequantization module includes a first dequantization module and a second dequantization module; The first prediction module is configured to perform prediction on input residual data of fourth data, and output the fourth data; The first dequantization module is configured to perform dequantization on the input fifth data, and output dequantized data of the fifth data; the fourth data is reconstructed data of one of the plurality of area descriptors or the fifth data, and the dequantized data of the fifth data is reconstructed data of one of the plurality of area descriptors; The second prediction module is configured to perform prediction on input residual data of sixth data, and output the sixth data; The second dequantization module is configured to perform dequantization on the input seventh data and output dequantized data of the seventh data; and The sixth data is reconstructed data of one of the plurality of 3D map points or the seventh data, and the dequantized data of the seventh data is reconstructed data of one of the plurality of 3D map points. Item 14. The device according to item 13. [Item 15] The prediction module includes a first prediction module, a second prediction module, and a third prediction module; and / or the dequantization module includes a first dequantization module, a second dequantization module, and a third dequantization module; The first prediction module is configured to perform prediction on input residual data of eighth data, and output the eighth data; The first dequantization module is configured to perform dequantization on the input ninth data, and output dequantized data of the ninth data; the eighth data is reconstructed data of one of the plurality of area descriptors or the ninth data, and the dequantized data of the ninth data is reconstructed data of one of the plurality of area descriptors; The second prediction module is configured to perform prediction on input residual data of a tenth data item, and output the tenth data item; The second dequantization module is configured to perform dequantization on the input eleventh data, and output dequantized data of the eleventh data; the tenth data is reconstruction data of a 3D map point descriptor of one of the plurality of 3D map points or the eleventh data, and the dequantized data of the eleventh data is reconstruction data of a 3D map point descriptor of one of the plurality of 3D map points; The third prediction module is configured to perform prediction on input residual data of twelfth data, and output the twelfth data; The third dequantization module is configured to perform dequantization on the input thirteenth data and output the dequantized data of the thirteenth data; and the twelfth data is reconstructed data of a 3D map point spatial position of one of the plurality of 3D map points or the thirteenth data, and the dequantized data of the thirteenth data is reconstructed data of a 3D map point spatial position of one of the plurality of 3D map points; Item 14. The device according to item 13. [Item 16] the decompression module having a first decompression sub-module and a second decompression sub-module; the first decompression sub-module is configured to decompress an input bitstream of fourteenth data and output the fourteenth data; the second decompression sub-module is configured to compress an input bitstream of fifteenth data and output the fifteenth data; and The fourteenth data is reconstruction data of one of the plurality of area descriptors, and the fifteenth data is reconstruction data of one of the plurality of 3D map points. 13. The device according to any one of items 8 to 12. [Item 17] The first decompression sub-module includes a first decapsulation module, a first prediction module, and / or a first dequantization module; the second decompression sub-module includes a second decapsulation module, and a second prediction module and / or a second dequantization module; the first de-encapsulation module is configured to process the input bitstream of the 3D map and output sixteenth data; The first prediction module is configured to perform prediction on input residual data of a seventeenth data item, and output the seventeenth data item; The first dequantization module is configured to perform dequantization on the input 18th data, and output dequantized data of the 18th data; the sixteenth data is the residual data of the seventeenth data or the eighteenth data, the seventeenth data is reconstructed data of one of the plurality of area descriptors or the eighteenth data, and the dequantized data of the eighteenth data is reconstructed data of one of the plurality of area descriptors; the second de-encapsulation module is configured to process the input bitstream of the 3D map to output nineteenth data; The second prediction module is configured to perform prediction on input residual data of a twentieth data item, and output the twentieth data item; The second dequantization module is configured to perform dequantization on the input 21st data and output dequantized data of the 21st data; and The 19th data is the residual data of the 20th data or the 21st data, the 20th data is the reconstructed data of one of the plurality of 3D map points or the 21st data, and the dequantized data of the 21st data is the reconstructed data of one of the plurality of 3D map points. Item 17. The device according to item 16. [Item 18] the decompression module includes a first decompression sub-module, a second decompression sub-module, and a third decompression sub-module; the first decompression sub-module is configured to decompress an input bitstream of 22nd data and output the 22nd data; the second decompression sub-module is configured to compress an input bitstream of 23rd data and output the 23rd data; the third decompression sub-module is configured to compress an input bitstream of 24th data and output the 24th data; and The 22nd data is reconstruction data of one of the plurality of area descriptors, the 23rd data is reconstruction data of a 3D map point descriptor of one of the plurality of 3D map points, and the 24th data is reconstruction data of a 3D map point spatial position of one of the plurality of 3D map points. 13. The device according to any one of items 8 to 12. [Item 19] The first decompression sub-module includes a first decapsulation module, and a first prediction module and / or a first dequantization module; the second decompression sub-module includes a second decapsulation module, and a second prediction module and / or a second quantization module; and the third decompression sub-module includes a third decapsulation module, and a third prediction module and / or a third quantization module; the first de-encapsulation module is configured to process an input bitstream of the 3D map and output 25th data; The first prediction module is configured to perform prediction on input residual data of 26th data, and output the 26th data; The first dequantization module is configured to perform dequantization on the input 27th data, and output dequantized data of the 27th data; the 25th data is the residual data of the 26th data or the 27th data, the 26th data is reconstructed data of one of the plurality of area descriptors or the 27th data, and the dequantized data of the 27th data is reconstructed data of one of the plurality of area descriptors; the second de-encapsulation module is configured to process the input bitstream of the 3D map and output 28th data; The second prediction module is configured to perform prediction on the input residual data of the 29th data, and output the 29th data; The second dequantization module is configured to perform dequantization on the input 30th data, and output dequantized data of the 30th data; the 28th data is the residual data of the 29th data or the 30th data, the 29th data is the reconstructed data of a 3D map point descriptor of one of the plurality of 3D map points or the 30th data, and the dequantized data of the 30th data is the reconstructed data of a 3D map point descriptor of one of the plurality of 3D map points; the third de-encapsulation module is configured to process the input bitstream of the 3D map and output 31 data; The third prediction module is configured to perform prediction on input residual data of 32nd data, and output the 32nd data; The third dequantization module is configured to perform dequantization on the input 33rd data, and output dequantized data of the 33rd data; The 31st data is the residual data of the 32nd data or the 33rd data, the 32nd data is reconstructed data of a 3D map point spatial position of one of the plurality of 3D map points or the 33rd data, and the dequantized data of the 33rd data is reconstructed data of a 3D map point spatial position of one of the plurality of 3D map points. Item 19. The device according to item 18. [Item 20] 1. A method for decoding a 3D map, comprising: receiving a bitstream of a 3D map, wherein the 3D map includes a plurality of 3D map points; and decompressing the bitstream of the 3D map to obtain reconstruction data of the 3D map, wherein the reconstruction data of the 3D map includes reconstruction data of the plurality of 3D map points. A method for providing the above. [Item 21] The reconstruction data of the 3D map further includes reconstruction data of a plurality of area descriptors, any one of the plurality of area descriptors describing characteristics of some or all of the plurality of 3D map points. Item 20. The method according to item 20. [Item 22] 22. The method of claim 21, wherein the reconstruction data for any one of the plurality of 3D map points includes reconstruction data for a 3D map point descriptor and reconstruction data for a 3D map point spatial position. [Item 23] sending a 3D map download request, wherein the 3D map download request includes location indication information. and The step of receiving a bitstream of 3D maps includes: receiving a bitstream of the 3D map corresponding to the position indication information; 23. The method according to any one of items 20 to 22, comprising: [Item 24] The step of receiving a bitstream of 3D maps includes: receiving the bitstream of the 3D map generated by an electronic device 23. The method according to any one of items 20 to 22, comprising: [Item 25] decompressing the bitstream of the 3D map to obtain reconstruction data of the 3D map, processing the bitstream of the 3D map to obtain first data; and performing prediction on residual data of second data to obtain the second data, and / or performing dequantization on third data to obtain dequantized data of the third data, wherein: The first data is the residual data of the second data or the third data, the second data is the reconstructed data of the 3D map or the third data, and the dequantized data of the third data is the reconstructed data of the 3D map. 25. The method according to any one of items 20 to 24, comprising: [Item 26] 1. An apparatus for encoding a 3D map, comprising: one or more processors; and a memory configured to store one or more programs; wherein: When the one or more programs are executed by the one or more processors, the one or more processors are enabled to perform the method according to any one of items 20 to 25. Device. [Item 27] 1. An apparatus for decoding a 3D map, comprising: a memory configured to store a 3D map in a bitstream format obtained through compression, wherein the 3D map includes a plurality of 3D map points, and wherein data for the 3D map includes data for the plurality of 3D map points; and a decoder configured to decompress the 3D map in a bitstream format obtained through compression to obtain a reconstructed 3D map, wherein the reconstructed 3D map includes the plurality of 3D map points, and data of the reconstructed 3D map includes the reconstructed data of the plurality of 3D map points. An apparatus comprising: [Item 28] The data of the reconstructed 3D map further includes reconstruction data of a plurality of area descriptors, any one of the plurality of area descriptors describing characteristics of some or all of the plurality of 3D map points. Item 28. The device according to item 27. [Item 29] 29. The apparatus of claim 27 or 28, wherein the reconstruction data for any one of the plurality of 3D map points includes reconstruction data for a 3D map point descriptor and reconstruction data for a 3D map point spatial position. [Item 30] 30. The apparatus of any one of items 27 to 29, wherein the apparatus for encoding a 3D map is a cloud server or an electronic device. [Item 31] 26. A computer-readable storage medium comprising a computer program, which, when run on a computer, enables the computer to carry out the method according to any one of items 20 to 25. [Item 32] 26. A computer program product comprising computer program code, which, when executed on a computer, enables the computer to carry out the method according to any one of items 20 to 25. [Item 33] 1. An encoded bitstream of a 3D map, the encoded bitstream of the 3D map comprising a bitstream of a plurality of 3D map points, the 3D map including the plurality of 3D map points. [Item 34] Item 34. The bitstream of item 33, wherein the encoded bitstream of the 3D map further comprises a bitstream of a plurality of area descriptors, any one of the plurality of area descriptors corresponding to at least one 3D map point of the plurality of 3D map points. [Item 35] Item 35. The bitstream of any one of the plurality of 3D map points, comprising a bitstream of 3D map point descriptors and a bitstream of 3D map point spatial positions. [Item 36] Item 34. The method of item 33, wherein the bitstream of any one of the plurality of 3D map points includes residual data for the 3D map point. [Item 37] Item 36. The bitstream of the 3D map point descriptors includes residual data of the 3D map point descriptors; and / or the bitstream of the 3D map point space locations includes residual data of the 3D map point space locations. [Item 38] Item 38. The bitstream of any one of the plurality of 3D map points further includes indication information of a reference 3D map point, the plurality of 3D map points including the reference 3D map point, the reference 3D map point being a 3D map point that is encoded before the 3D map point is encoded. [Item 39] Item 34. The method of item 33, wherein the bitstream of any one of the plurality of 3D map points includes quantized data of the 3D map point. [Item 40] Item 36. The bitstream of the 3D map point descriptors comprises quantized data of the 3D map point descriptors; and / or the bitstream of the 3D map point space locations comprises quantized data of the 3D map point space locations. [Item 41] Item 35. The bitstream of any one of the plurality of area descriptors, wherein the bitstream includes residual data of the area descriptor. [Item 42] The bitstream of any one of the plurality of area descriptors further includes indication information of a reference area descriptor, and the plurality of area descriptors includes the reference area descriptor. [Item 43] Item 35. The bitstream of any one of the plurality of area descriptors, wherein the bitstream includes quantized data of the area descriptor.

Claims

1. A codec system for a 3D map, comprising an encoding device and a decoding device, wherein the encoding device is communicatively coupled to the decoding device; the encoding device is configured to compress data of a 3D map to obtain a bitstream of the 3D map, and transmit the bitstream of the 3D map to the decoding device, wherein the 3D map includes a plurality of 3D map points, the data of the 3D map includes data of the plurality of 3D map points, the bitstream of the 3D map includes a bitstream of the plurality of 3D map points, the bitstream of any one of the plurality of 3D map points includes residual data of the 3D map point, and the data of any one of the plurality of 3D map points includes a 3D map point descriptor, the 3D map point descriptor being a vector used to represent local features of the 3D map point, and the 3D map point descriptor is used for matching between 3D map points; and The decoding device is configured to receive the bitstream of the 3D map and to decompress the bitstream of the 3D map to obtain reconstruction data of the 3D map. system.

2. The encoding device is a cloud server and the decoding device is an electronic device; or the encoding device is a first electronic device and the decoding device is a second electronic device; the decoding device is further configured to send a 3D map download request to the encoding device, where the 3D map download request includes location indication information; and The encoding device is further configured to receive the 3D map download request and to transmit a bitstream of the 3D map corresponding to the position indication information to the decoding device according to the 3D map download request. The system of claim 1 .

3. The encoding device is an electronic device and the decoding device is a cloud server; and The encoding device is particularly configured to transmit the bitstream of the 3D map to the decoding device after the 3D map is generated.

3. The system according to claim 1 or 2.

4. 4. The system of claim 1, wherein the data for the 3D map further comprises a plurality of area descriptors, any one of the plurality of area descriptors describing characteristics of some or all of the plurality of 3D map points.

5. The system of claim 1 or 4, wherein the data for any one of the plurality of 3D map points further includes a 3D map point spatial position.

6. The system of claim 1 , wherein the encoding device is further configured to generate the 3D map.

7. The system of claim 1 , wherein the decoding device is further configured to perform positioning based on the 3D map.

8. the encoding device is an electronic device, and the decoding device is a cloud server; the electronic device is configured to transmit the bitstream of the 3D map to the cloud server after the 3D map is generated; The cloud server performs decompression to obtain reconstructed data of the 3D map, and determines a current pose of the second electronic device by referring to visual information from a second electronic device and the 3D map from the second electronic device. A system according to any one of claims 1 to 7.

9. 9. The system of claim 1, wherein the bitstream of any one of the plurality of 3D map points comprises a bitstream of a 3D map point descriptor and a bitstream of 3D map point space locations, the bitstream of the 3D map point descriptor comprising residual data of the 3D map point descriptor, and the bitstream of the 3D map point space locations comprising residual data of the 3D map point space locations.

10. 1. An apparatus for decoding a 3D map, comprising a transmission module and a decompression module, wherein: the transmission module is configured to receive a bitstream of a 3D map, where the 3D map includes a plurality of 3D map points, the bitstream of the 3D map includes a bitstream of the plurality of 3D map points, and the bitstream of any one of the plurality of 3D map points includes residual data of the 3D map point; and The decompression module is configured to decompress the bitstream of the 3D map to obtain reconstruction data of the 3D map, wherein the reconstruction data of the 3D map includes reconstruction data of the plurality of 3D map points, and the reconstruction data of any one of the plurality of 3D map points includes reconstruction data of a 3D map point descriptor, the 3D map point descriptor being a vector used to represent local features of the 3D map point, and the reconstruction data of the 3D map point descriptor being used for matching between 3D map points. Device.

11. The reconstruction data of the 3D map further includes reconstruction data of a plurality of area descriptors, any one of the plurality of area descriptors describing characteristics of some or all of the plurality of 3D map points.

11. The apparatus of claim 10.

12. The apparatus of claim 11 , wherein the reconstruction data for any one of the plurality of 3D map points further comprises reconstruction data of 3D map point spatial positions.

13. The apparatus for decoding the 3D map is a cloud server or an electronic device; and The transmission module is further configured to: send a 3D map download request, where the 3D map download request includes location indication information; and receive a bitstream of the 3D map corresponding to the location indication information.

13. Apparatus according to any one of claims 10 to 12.

14. The device for decoding a 3D map is a cloud server; and The transmission module is particularly configured to receive the bitstream of the 3D map generated by an electronic device.

13. Apparatus according to any one of claims 10 to 12.

15. The decompression module comprises a decapsulation module, and a prediction module and / or a dequantization module; the de-encapsulation module is configured to process an input bitstream of the 3D map to output first data; The prediction module is configured to perform prediction on input residual data of second data and output the second data; the dequantization module is configured to perform dequantization on the input third data and output dequantized data of the third data; and The first data is the residual data of the second data or the third data, the second data is the reconstructed data of the 3D map or the third data, and the dequantized data of the third data is the reconstructed data of the 3D map.

15. Apparatus according to any one of claims 10 to 14.

16. The prediction module includes a first prediction module and a second prediction module; and / or the dequantization module includes a first dequantization module and a second dequantization module; The first prediction module is configured to perform prediction on input residual data of fourth data, and output the fourth data; The first dequantization module is configured to perform dequantization on the input fifth data and output dequantized data of the fifth data; the fourth data is reconstructed data of one of a plurality of area descriptors or the fifth data, and the dequantized data of the fifth data is reconstructed data of one of the plurality of area descriptors; The second prediction module is configured to perform prediction on input residual data of sixth data, and output the sixth data; The second dequantization module is configured to perform dequantization on the input seventh data and output dequantized data of the seventh data; and The sixth data is reconstructed data of one of the plurality of 3D map points or the seventh data, and the dequantized data of the seventh data is reconstructed data of one of the plurality of 3D map points.

16. The apparatus of claim 15.

17. 1. An apparatus for decoding a 3D map, comprising a transmission module and a decompression module, wherein: the transmission module is configured to receive a bitstream of a 3D map, where the 3D map includes a plurality of 3D map points; the decompression module is configured to decompress the bitstream of the 3D map to obtain reconstruction data of the 3D map, where the reconstruction data of the 3D map includes reconstruction data of the plurality of 3D map points; The decompression module comprises a decapsulation module, and a prediction module and / or a dequantization module; the de-encapsulation module is configured to process an input bitstream of the 3D map to output first data; The prediction module is configured to perform prediction on input residual data of second data and output the second data; The dequantization module is configured to perform dequantization on the input third data and output dequantized data of the third data; the first data is the residual data of the second data or the third data, the second data is the reconstructed data of the 3D map or the third data, and the dequantized data of the third data is the reconstructed data of the 3D map; The prediction module includes a first prediction module and a second prediction module; and / or the dequantization module includes a first dequantization module and a second dequantization module; The first prediction module is configured to perform prediction on input residual data of fourth data, and output the fourth data; The first dequantization module is configured to perform dequantization on the input fifth data and output dequantized data of the fifth data; the fourth data is reconstructed data of one of a plurality of area descriptors or the fifth data, and the dequantized data of the fifth data is reconstructed data of one of the plurality of area descriptors; The second prediction module is configured to perform prediction on input residual data of sixth data, and output the sixth data; The second dequantization module is configured to perform dequantization on the input seventh data and output dequantized data of the seventh data; the sixth data is reconstructed data of one of the plurality of 3D map points or the seventh data, and the dequantized data of the seventh data is reconstructed data of one of the plurality of 3D map points.

18. The prediction module includes a first prediction module, a second prediction module, and a third prediction module; and / or the dequantization module includes a first dequantization module, a second dequantization module, and a third dequantization module; The first prediction module is configured to perform prediction on input residual data of eighth data, and output the eighth data; The first dequantization module is configured to perform dequantization on the input ninth data, and output dequantized data of the ninth data; the eighth data is reconstructed data of one of a plurality of area descriptors or the ninth data, and the dequantized data of the ninth data is reconstructed data of one of the plurality of area descriptors; The second prediction module is configured to perform prediction on input residual data of a tenth data item, and output the tenth data item; The second dequantization module is configured to perform dequantization on the input eleventh data, and output dequantized data of the eleventh data; the tenth data is reconstruction data of a 3D map point descriptor of one of the plurality of 3D map points or the eleventh data, and the dequantized data of the eleventh data is reconstruction data of a 3D map point descriptor of one of the plurality of 3D map points; The third prediction module is configured to perform prediction on input residual data of twelfth data, and output the twelfth data; The third dequantization module is configured to perform dequantization on the input thirteenth data and output dequantized data of the thirteenth data; and the twelfth data is reconstructed data of a 3D map point spatial position of one of the plurality of 3D map points or the thirteenth data, and the dequantized data of the thirteenth data is reconstructed data of a 3D map point spatial position of one of the plurality of 3D map points; 16. The apparatus of claim 15.

19. 1. An apparatus for decoding a 3D map, comprising a transmission module and a decompression module, wherein: the transmission module is configured to receive a bitstream of a 3D map, where the 3D map includes a plurality of 3D map points; The decompression module is configured to decompress the bitstream of the 3D map to obtain reconstruction data of the 3D map, where the reconstruction data of the 3D map includes reconstruction data of the plurality of 3D map points: The decompression module comprises a decapsulation module, and a prediction module and / or a dequantization module; the de-encapsulation module is configured to process an input bitstream of the 3D map to output first data; The prediction module is configured to perform prediction on input residual data of second data and output the second data; The dequantization module is configured to perform dequantization on the input third data and output dequantized data of the third data; the first data is the residual data of the second data or the third data, the second data is the reconstructed data of the 3D map or the third data, and the dequantized data of the third data is the reconstructed data of the 3D map; The prediction module includes a first prediction module, a second prediction module, and a third prediction module; and / or the dequantization module includes a first dequantization module, a second dequantization module, and a third dequantization module; The first prediction module is configured to perform prediction on input residual data of eighth data, and output the eighth data; The first dequantization module is configured to perform dequantization on the input ninth data, and output dequantized data of the ninth data; the eighth data is reconstructed data of one of a plurality of area descriptors or the ninth data, and the dequantized data of the ninth data is reconstructed data of one of the plurality of area descriptors; The second prediction module is configured to perform prediction on input residual data of a tenth data item, and output the tenth data item; The second dequantization module is configured to perform dequantization on the input eleventh data, and output dequantized data of the eleventh data; the tenth data is reconstruction data of a 3D map point descriptor of one of the plurality of 3D map points or the eleventh data, and the dequantized data of the eleventh data is reconstruction data of a 3D map point descriptor of one of the plurality of 3D map points; The third prediction module is configured to perform prediction on input residual data of twelfth data, and output the twelfth data; The third dequantization module is configured to perform dequantization on the input thirteenth data, and output dequantized data of the thirteenth data; the twelfth data is reconstructed data of a 3D map point spatial location of one of the plurality of 3D map points or the thirteenth data, and the dequantized data of the thirteenth data is reconstructed data of a 3D map point spatial location of one of the plurality of 3D map points.

20. the decompression module having a first decompression sub-module and a second decompression sub-module; the first decompression sub-module is configured to decompress an input bitstream of fourteenth data and output the fourteenth data; the second decompression sub-module is configured to decompress an input bitstream of fifteenth data and output the fifteenth data; and The fourteenth data is reconstruction data of one of a plurality of area descriptors, and the fifteenth data is reconstruction data of one of the plurality of 3D map points.

15. Apparatus according to any one of claims 10 to 14.

21. 1. An apparatus for decoding a 3D map, comprising a transmission module and a decompression module, wherein: the transmission module is configured to receive a bitstream of a 3D map, where the 3D map includes a plurality of 3D map points; the decompression module is configured to decompress the bitstream of the 3D map to obtain reconstruction data of the 3D map, where the reconstruction data of the 3D map includes reconstruction data of the plurality of 3D map points; the decompression module having a first decompression sub-module and a second decompression sub-module; the first decompression sub-module is configured to decompress an input bitstream of fourteenth data and output the fourteenth data; the second decompression sub-module is configured to decompress an input bitstream of fifteenth data and output the fifteenth data; The apparatus, wherein the fourteenth data is reconstruction data of one of a plurality of area descriptors, and the fifteenth data is reconstruction data of one of the plurality of 3D map points.

22. The first decompression sub-module includes a first decapsulation module, a first prediction module, and / or a first dequantization module; the second decompression sub-module includes a second decapsulation module, and a second prediction module and / or a second dequantization module; the first de-encapsulation module is configured to process the input bitstream of the 3D map and output sixteenth data; The first prediction module is configured to perform prediction on input residual data of seventeenth data, and output the seventeenth data; The first dequantization module is configured to perform dequantization on the input eighteenth data, and output dequantized data of the eighteenth data; the sixteenth data is the residual data of the seventeenth data or the eighteenth data, the seventeenth data is reconstructed data of one of the plurality of area descriptors or the eighteenth data, and the dequantized data of the eighteenth data is reconstructed data of one of the plurality of area descriptors; the second de-encapsulation module is configured to process the input bitstream of the 3D map to output nineteenth data; The second prediction module is configured to perform prediction on input residual data of a twentieth data item, and output the twentieth data item; The second dequantization module is configured to perform dequantization on the input twenty-first data and output dequantized data of the twenty-first data; and The nineteenth data is the residual data of the twentieth data or the twenty-first data, the twenty-first data is reconstructed data of one of the plurality of 3D map points or the twenty-first data, and the dequantized data of the twenty-first data is reconstructed data of one of the plurality of 3D map points.

22. Apparatus according to claim 20 or 21.

23. the decompression module having a first decompression sub-module, a second decompression sub-module, and a third decompression sub-module; the first decompression sub-module is configured to decompress an input bitstream of twenty-second data and output the twenty-second data; the second decompression sub-module is configured to decompress an input bitstream of twenty-third data and output the twenty-third data; the third decompression sub-module is configured to decompress an input bitstream of twenty-fourth data and output the twenty-fourth data; and The twenty-second data is reconstruction data of one of a plurality of area descriptors, the twenty-third data is reconstruction data of a 3D map point descriptor of one of the plurality of 3D map points, and the twenty-fourth data is reconstruction data of a 3D map point spatial position of one of the plurality of 3D map points.

15. Apparatus according to any one of claims 10 to 14.

24. 1. An apparatus for decoding a 3D map, comprising a transmission module and a decompression module, wherein: the transmission module is configured to receive a bitstream of a 3D map, where the 3D map includes a plurality of 3D map points; the decompression module is configured to decompress the bitstream of the 3D map to obtain reconstruction data of the 3D map, where the reconstruction data of the 3D map includes reconstruction data of the plurality of 3D map points; the decompression module having a first decompression sub-module, a second decompression sub-module, and a third decompression sub-module; the first decompression sub-module is configured to decompress an input bitstream of twenty-second data and output the twenty-second data; the second decompression sub-module is configured to decompress an input bitstream of twenty-third data and output the twenty-third data; the third decompression sub-module is configured to decompress an input bitstream of twenty-fourth data and output the twenty-fourth data; the twenty-second data is reconstruction data of one of a plurality of area descriptors, the twenty-third data is reconstruction data of a 3D map point descriptor of one of the plurality of 3D map points, and the twenty-fourth data is reconstruction data of a 3D map point spatial position of one of the plurality of 3D map points.

25. The first decompression sub-module includes a first decapsulation module, and a first prediction module and / or a first quantization module; the second decompression sub-module includes a second decapsulation module, and a second prediction module and / or a second quantization module; and the third decompression sub-module includes a third decapsulation module, and a third prediction module and / or a third quantization module; the first de-encapsulation module is configured to process the input bitstream of the 3D map to output 25th data; The first prediction module is configured to perform prediction on input residual data of 26 data, and output the 26 data; The first dequantization module is configured to perform dequantization on the input 27th data, and output dequantized data of the 27th data; the 25th data is the residual data of the 26th data or the 27th data, the 26th data is reconstructed data of one of the plurality of area descriptors or the 27th data, and the dequantized data of the 27th data is reconstructed data of one of the plurality of area descriptors; the second de-encapsulation module is configured to process the input bitstream of the 3D map and output 28 data; The second prediction module is configured to perform prediction on input residual data of a 29th data item, and output the 29th data item; The second dequantization module is configured to perform dequantization on the input 30th data, and output dequantized data of the 30th data; the twenty-eighth data is the residual data of the twenty-ninth data or the thirty-ninth data, the twenty-ninth data is reconstructed data of a 3D map point descriptor of one of the plurality of 3D map points or the thirty-ninth data, and the dequantized data of the thirty-ninth data is reconstructed data of a 3D map point descriptor of one of the plurality of 3D map points; the third de-encapsulation module is configured to process the input bitstream of the 3D map and output thirty-first data; The third prediction module is configured to perform prediction on input residual data of 32 data, and output the 32 data; The third dequantization module is configured to perform dequantization on the input 33rd data, and output dequantized data of the 33rd data; The 31st data is the residual data of the 32nd data or the 33rd data, the 32nd data is reconstructed data of a 3D map point spatial position of one of the plurality of 3D map points or the 33rd data, and the dequantized data of the 33rd data is reconstructed data of a 3D map point spatial position of one of the plurality of 3D map points.

25. Apparatus according to claim 23 or 24.

26. the apparatus for encoding the 3D map is an electronic device and the apparatus for decoding the 3D map is a cloud server; the electronic device is configured to transmit the bitstream of the 3D map to the cloud server after the 3D map is generated; The cloud server performs decompression to obtain reconstructed data of the 3D map, and determines a current pose of the second electronic device by referring to visual information from a second electronic device and the 3D map from the second electronic device.

26. Apparatus according to any one of claims 10 to 25.

27. 27. The apparatus of claim 10, wherein the bitstream of any one of the plurality of 3D map points comprises a bitstream of a 3D map point descriptor and a bitstream of 3D map point space locations, the bitstream of the 3D map point descriptor comprising residual data of the 3D map point descriptor, and the bitstream of the 3D map point space locations comprising residual data of the 3D map point space locations.

28. 1. A method for decoding a 3D map, comprising: receiving a bitstream of a 3D map, wherein the 3D map includes a plurality of 3D map points, the bitstream of the 3D map includes a bitstream of the plurality of 3D map points, and the bitstream of any one of the plurality of 3D map points includes residual data of the 3D map point; and decompressing the bitstream of the 3D map to obtain reconstruction data of the 3D map, wherein the reconstruction data of the 3D map includes reconstruction data of the plurality of 3D map points, and the reconstruction data of any one of the plurality of 3D map points includes reconstruction data of a 3D map point descriptor, the 3D map point descriptor being a vector used to represent local features of the 3D map point, and the reconstruction data of the 3D map point descriptor being used for matching between 3D map points. A method for providing the above.

29. The reconstruction data of the 3D map further includes reconstruction data of a plurality of area descriptors, any one of the plurality of area descriptors describing characteristics of some or all of the plurality of 3D map points.

29. The method of claim 28.

30. 30. The method of claim 29, wherein the reconstruction data for any one of the plurality of 3D map points comprises reconstruction data of a 3D map point spatial position.

31. transmitting a 3D map download request, wherein the 3D map download request includes location indication information. and The step of receiving a bitstream of 3D maps comprises: receiving a bitstream of the 3D map corresponding to the position indication information; 31. The method of any one of claims 28 to 30, comprising:

32. The step of receiving a bitstream of 3D maps comprises: receiving the bitstream of the 3D map generated by an electronic device; 31. The method of any one of claims 28 to 30, comprising:

33. decompressing the bitstream of the 3D map to obtain reconstruction data of the 3D map, processing the bitstream of the 3D map to obtain first data; and performing prediction on residual data of second data to obtain the second data, and / or performing dequantization on third data to obtain dequantized data of the third data, wherein: The first data is the residual data of the second data or the third data, the second data is the reconstructed data of the 3D map or the third data, and the dequantized data of the third data is the reconstructed data of the 3D map.

33. The method of any one of claims 28 to 32, comprising:

34. the apparatus for encoding the 3D map is an electronic device and the apparatus for decoding the 3D map is a cloud server; the electronic device is configured to transmit the bitstream of the 3D map to the cloud server after the 3D map is generated; The cloud server performs decompression to obtain reconstructed data of the 3D map, and determines a current pose of the second electronic device by referring to visual information from a second electronic device and the 3D map from the second electronic device.

34. The method of any one of claims 28 to 33.

35. 35. The method of claim 28, wherein the bitstream of any one of the plurality of 3D map points comprises a bitstream of a 3D map point descriptor and a bitstream of 3D map point space locations, the bitstream of the 3D map point descriptor comprising residual data of the 3D map point descriptor, and the bitstream of the 3D map point space locations comprising residual data of the 3D map point space locations.

36. 1. An apparatus for decoding a 3D map, comprising: one or more processors; and a memory configured to store one or more programs; wherein: The one or more programs, when executed by the one or more processors, enable the one or more processors to perform the method of any one of claims 28 to 35. Device.

37. 1. An apparatus for decoding a 3D map, comprising: a memory configured to store a 3D map in a bitstream format obtained through compression, wherein the 3D map includes a plurality of 3D map points, the 3D map data includes data of the plurality of 3D map points, the 3D map bitstream includes a bitstream of the plurality of 3D map points, and the bitstream of any one of the plurality of 3D map points includes residual data of the 3D map point; and a decoder configured to decompress the 3D map in a bitstream format obtained through compression to obtain a reconstructed 3D map, wherein the reconstructed 3D map includes the plurality of 3D map points, the data of the reconstructed 3D map includes reconstruction data of the plurality of 3D map points, the reconstruction data of any one of the plurality of 3D map points includes reconstruction data of a 3D map point descriptor, the 3D map point descriptor being a vector used to represent local features of the 3D map point, and the reconstruction data of the 3D map point descriptor being used for matching between 3D map points; An apparatus comprising:

38. The data of the reconstructed 3D map further includes reconstruction data of a plurality of area descriptors, any one of the plurality of area descriptors describing characteristics of some or all of the plurality of 3D map points.

38. The apparatus of claim 37.

39. 39. The apparatus of claim 37 or 38, wherein the reconstruction data for any one of the plurality of 3D map points comprises reconstruction data of a 3D map point spatial position.

40. 40. The apparatus of any one of claims 37 to 39, wherein the apparatus for encoding the 3D map is a cloud server or an electronic device.

41. the apparatus for encoding the 3D map is an electronic device and the apparatus for decoding the 3D map is a cloud server; the electronic device is configured to transmit the bitstream of the 3D map to the cloud server after the 3D map is generated; The cloud server performs decompression to obtain reconstructed data of the 3D map, and determines a current pose of the second electronic device by referring to visual information from a second electronic device and the 3D map from the second electronic device.

41. Apparatus according to any one of claims 37 to 40.

42. 42. The apparatus of claim 37, wherein the bitstream of any one of the plurality of 3D map points comprises a bitstream of a 3D map point descriptor and a bitstream of 3D map point space locations, the bitstream of the 3D map point descriptor comprising residual data of the 3D map point descriptor, and the bitstream of the 3D map point space locations comprising residual data of the 3D map point space locations.

43. A computer readable storage medium comprising a computer program, which when run on a computer enables the computer to carry out the method of any one of claims 28 to 35.

44. 36. A computer program comprising computer program code, which, when run on a computer, enables the computer to carry out the method of any one of claims 28 to 35.

45. 1. A method for generating a bitstream of a 3D map by an encoding device, comprising:

10. A method for encoding a 3D map comprising: compressing data of the 3D map to obtain a bitstream of the 3D map; and transmitting the bitstream of the 3D map to a decoding device, wherein the 3D map comprises a plurality of 3D map points; the data of the 3D map comprises data of the plurality of 3D map points; the bitstream of the 3D map comprises a bitstream of the plurality of 3D map points; the bitstream of any one of the plurality of 3D map points comprises residual data of the 3D map point; and the data of any one of the plurality of 3D map points comprises a 3D map point descriptor, the 3D map point descriptor being a vector used to represent local features of the 3D map point, and the 3D map point descriptor being used for matching between 3D map points.

Images