A lightweight method and system for 3D asset models

Abstract

The present invention discloses a lightweight method and system for 3D asset models. The system includes a cloud server and a target vehicle. The cloud server determines the transmission delay of a data request, obtains 3D map data of a corresponding road according to location information, determines an image compression constant according to the transmission delay and display device information, performs video-based point cloud compression on the 3D map data using the image compression constant to obtain a map data packet, and sends the map data packet to the target vehicle. The target vehicle acquires two-dimensional images around the vehicle body, parses the map data packet to perform 3D map reconstruction to obtain an original 3D map, and aligns and fuses the two-dimensional images with the original 3D map to obtain a target 3D map. By dynamically determining the image compression constant and removing color attribute data, the size of the map data packet is reduced, the transmission efficiency is improved, and the two-dimensional images sensed by the vehicle in real time are combined with the three-dimensional map data provided by the cloud, improving the accuracy and practicality of the map.

Description

Technical Field

[0001] The present invention relates to the technical field of 3D image processing, and particularly to a lightweight method and system for 3D asset models. Background Art

[0002] With the continuous improvement of the intelligence level of automobiles, consumers' demand for automotive navigation systems is also increasing. As a new generation of navigation system, 3D navigation has become the first choice of consumers with its intuitive and accurate features. The 3D map of 3D navigation not only provides basic information such as roads, intersections, and buildings that traditional 2D maps possess, but also presents elements such as roads, vehicles, pedestrians, and obstacles in the real world in a three-dimensional manner to users through three-dimensional modeling technology.

[0003] 3D maps are usually stored as 3D asset models in a cloud server. When a vehicle needs 3D navigation, it sends a data request to the cloud server to obtain the 3D map of the area where the vehicle is located. However, traditional 3D map data transmission often has a long transmission time due to the large amount of data. Especially in mobile scenarios, such as during vehicle driving, the timeliness of obtaining and displaying 3D maps in real time is poor. Summary of the Invention

[0004] The object of the present invention is to solve the problems in the above background art, and to propose a lightweight method and system for 3D asset models.

[0005] The object of the present invention can be achieved by the following technical solutions:

[0006] In the first aspect of the embodiments of the present invention, a lightweight system for 3D asset models is provided. The system includes a cloud server and a target vehicle; wherein:

[0007] The target vehicle sends a data request to the cloud server; the data request includes the position information and display device information of the target vehicle.

[0008] The cloud server determines the transmission delay of the data request, obtains 3D map data of the corresponding road according to the position information, determines an image compression constant according to the transmission delay and the display device information, uses the image compression constant to perform video-based point cloud compression on the 3D map data to obtain a map data packet, and sends the map data packet to the target vehicle; the map data packet includes a geometry map, an occupancy map, and an attribute map, and the color attribute data is removed from the attribute map.

[0009] The target vehicle acquires two-dimensional images around the vehicle body, parses the map data packet to perform 3D map reconstruction to obtain an original 3D map, and aligns and fuses the two-dimensional images with the original 3D map to obtain a target 3D map.

[0010] In a second aspect of the embodiments of the present invention, a method for lightweighting a 3D asset model is provided. The method is applied to a cloud server and includes:

[0011] Receiving a data request sent by a target vehicle; the data request includes the location information and display device information of the target vehicle;

[0012] Determining the transmission delay of the data request, obtaining 3D map data of the corresponding road according to the location information, and determining an image compression constant according to the transmission delay and the display device information;

[0013] Using the image compression constant to perform video-based point cloud compression on the 3D map data to obtain a map data packet, and sending the map data packet to the target vehicle; the map data packet includes a geometry map, an occupancy map, and an attribute map, and the color attribute data is removed from the attribute map; so that the target vehicle can obtain a two-dimensional image around the vehicle body, parse the map data packet to perform 3D map reconstruction to obtain an original 3D map, and align and fuse the two-dimensional image with the original 3D map to obtain a target 3D map.

[0014] Optionally, the image compression constant includes a scaling constant and a quantization parameter; determining the image compression constant according to the transmission delay and the display device information includes:

[0015] Querying a preset quantization parameter table according to the transmission delay to obtain a quantization parameter; the quantization parameter table records the corresponding relationship between the transmission delay and the quantization parameter;

[0016] Querying a preset scaling constant table according to the display device information to obtain a scaling constant; the scaling constant table records the corresponding relationship between the display device information and the scaling constant.

[0017] Optionally, the using the image compression constant to perform video-based point cloud compression on the 3D map data to obtain a map data packet includes:

[0018] Step 1, calculating a target data preservation rate and a target frame utilization rate for compression according to the scaling constant and the quantization parameter, dividing the point cloud of the 3D map data into a preset number of cuboids, and removing the cuboids that do not contain point cloud points to obtain an initial cuboid set;

[0019] Step 2, if there is no cuboid in the initial cuboid set, execute Step 5; otherwise, for each cuboid in the initial cuboid set, project the original point cloud in the cuboid into 2D spaces with different angles to generate patches;

[0020] Step 3, if the size of the cuboid is not the preset minimum size and the patch of the cuboid does not meet the preset conditions, divide the point cloud of the cuboid into a preset number of cuboids, remove the cuboids that do not contain point cloud points, update the initial cuboid set, and return to execute Step 2; the preset conditions are whether the data preservation rate and frame utilization rate of the patch relative to the original point cloud are greater than the target data preservation rate and target frame utilization rate respectively;

[0021] Step 4, if the size of the cuboid is the preset minimum size, or the patch of the cuboid meets the preset conditions, remove the cuboid from the initial cuboid set and divide it into the target cuboid set, and return to execute Step 2;

[0022] Step 5, put the patches corresponding to the cuboids in the target cuboid set into the 2D video frame, encode the 2D video frame to obtain a geometry map, an occupancy map, and an attribute map, and pack the geometry map, occupancy map, and attribute map into a V-PCC bitstream as a map data packet.

[0023] Optionally, projecting the original point cloud in the cuboid into 2D spaces with different angles to generate a patch includes:

[0024] Project the original point cloud in the cuboid onto the 2D space according to the six faces of the cuboid to obtain six 2D images;

[0025] Determine the 2D image with the most captured point cloud points among the six 2D images as the first image, and the 2D image with the second most projected points as the second image;

[0026] Determine the point cloud points in the original point cloud in the cuboid that are not projected onto the first image to generate a third image;

[0027] Fuse the first image, the second image, and the third image to obtain the patch of the cuboid.

[0028] Optionally, fusing the first image, the second image, and the third image to obtain the patch of the cuboid includes:

[0029] Determine the blank points in the third image and fill them with the point cloud points at the corresponding positions in the first image to obtain a fourth image;

[0030] Take the union of the first image and the fourth image to obtain a fifth image, and add the fifth image and the second image to obtain a target image as the patch of the cuboid.

[0031] Optionally, calculating the target data preservation rate and target frame utilization rate for compression according to the scaling constant and quantization parameter is specifically: where B is the target data preservation rate, Z is the target frame utilization rate, a is the scaling constant, S is the quantization parameter, and ω is a preset constant.

[0032] A third aspect of the embodiments of the present invention provides a lightweight method for 3D asset models, which is applied to a target vehicle. The method includes:

[0033] Sending a data request to the cloud server; the data request includes the location information and display device information of the target vehicle; so that the cloud server determines the transmission delay of the data request, obtains 3D map data of the corresponding road according to the location information, determines an image compression constant according to the transmission delay and the display device information, and uses the image compression constant to perform video-based point cloud compression on the 3D map data to obtain a map data packet, and sends the map data packet to the target vehicle; the map data packet includes a geometry map, an occupancy map, and an attribute map, and the color attribute data is removed from the attribute map;

[0034] Obtaining two-dimensional images around the vehicle body, parsing the map data packet to perform 3D map reconstruction to obtain an original 3D map, and aligning and fusing the two-dimensional images with the original 3D map to obtain a target 3D map.

[0035] Optionally, the image compression constant includes a scaling constant and a quantization parameter; the quantization parameter is obtained by querying a preset quantization parameter table according to the transmission delay; the quantization parameter table records the corresponding relationship between the transmission delay and the quantization parameter; the scaling constant is obtained by querying a preset scaling constant table according to the display device information; the scaling constant table records the corresponding relationship between the display device information and the scaling constant.

[0036] Optionally, the format of the map data packet is a V-PCC bitstream; parsing the map data packet to perform 3D map reconstruction to obtain an original 3D map, and aligning and fusing the two-dimensional images with the original 3D map to obtain a target 3D map includes:

[0037] Extracting an occupancy bitstream, a geometry bitstream, and an attribute bitstream from the map data packet;

[0038] Decoding these three bitstreams to obtain an occupancy map, a geometry image, and an initial attribute image, and generating three video frame sequences;

[0039] Performing geometric reconstruction and smoothing on the occupancy map, the geometry image, and the attribute image to obtain a reconstructed point cloud as the original 3D map;

[0040] Aligning the two-dimensional image with the original 3D map, extracting the color features of the two-dimensional image, and recoloring the original 3D map according to the color features to obtain a target 3D map.

[0041] Advantages of the present invention:

[0042] An embodiment of the present invention provides a lightweight system for 3D asset models. The system includes a cloud server and a target vehicle. Specifically: The target vehicle sends a data request to the cloud server. The data request includes the location information and display device information of the target vehicle. The cloud server determines the transmission delay of the data request, obtains the 3D map data of the corresponding road according to the location information, determines an image compression constant according to the transmission delay and display device information, performs video-based point cloud compression on the 3D map data using the image compression constant to obtain a map data packet, and sends the map data packet to the target vehicle. The map data packet includes a geometry map, an occupancy map, and an attribute map, and the color attribute data is removed from the attribute map. The target vehicle acquires two-dimensional images around the vehicle body, parses the map data packet to perform 3D map reconstruction to obtain an original 3D map, and aligns and fuses the two-dimensional images with the original 3D map to obtain a target 3D map.

[0043] This solution determines the image compression constant according to the current network state (transmission delay) of the target vehicle and the display device information. The vehicle obtains a map data packet from the cloud to generate an original 3D map without color, and then colors the original 3D map with the two-dimensional images collected locally by the vehicle to obtain the target 3D map. By dynamically determining the image compression constant and removing the color attribute data, the size of the map data packet is reduced, the transmission efficiency is improved, and the two-dimensional images perceived by the vehicle in real time are combined with the three-dimensional map data provided by the cloud, improving the accuracy and practicality of the map. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The present invention will be further described below with reference to the accompanying drawings.

[0045] Figure 1 It is a flowchart of a lightweight method for 3D asset models applied to a cloud server;

[0046] Figure 2 It is a flowchart of performing video-based point cloud compression on the 3D map data provided by the embodiment of the present invention;

[0047] Figure 3 It is a flowchart of a lightweight method for 3D asset models applied to a target vehicle. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0049] An embodiment of the present invention provides a lightweight system for 3D asset models. The system includes a cloud server and a target vehicle; wherein:

[0050] The target vehicle sends a data request to the cloud server; the data request includes the location information and display device information of the target vehicle.

[0051] The cloud server determines the transmission delay of the data request, obtains the 3D map data of the corresponding road according to the location information, determines the image compression constant according to the transmission delay and display device information, performs video-based point cloud compression on the 3D map data using the image compression constant to obtain a map data packet, and sends the map data packet to the target vehicle; the map data packet includes a geometry map, an occupancy map, and an attribute map, and the color attribute data is removed from the attribute map.

[0052] The target vehicle obtains the two-dimensional images around the vehicle body, parses the map data packet to perform 3D map reconstruction to obtain the original 3D map, and aligns and fuses the two-dimensional images with the original 3D map to obtain the target 3D map.

[0053] A lightweight system for 3D asset models provided by an embodiment of the present invention determines the image compression constant according to the current network state (transmission delay) of the target vehicle and the display device information. The vehicle obtains a colorless original 3D map by generating a map data packet from the cloud, and then colors the original 3D map with the two-dimensional images collected locally by the vehicle to obtain the target 3D map. By dynamically determining the image compression constant and removing the color attribute data, the size of the map data packet is reduced, the transmission efficiency is improved, and the two-dimensional images sensed by the vehicle in real time are combined with the three-dimensional map data provided by the cloud, improving the accuracy and practicality of the map.

[0054] An embodiment of the present invention provides a lightweight method for 3D asset models applied to a cloud server. Refer to Figure 1 , Figure 1 which is a flowchart of a lightweight method for 3D asset models applied to a cloud server. The method includes:

[0055] S101, receiving a data request sent by the target vehicle;

[0056] S102, determining the transmission delay of the data request, obtaining the 3D map data of the corresponding road according to the location information, and determining the image compression constant according to the transmission delay and display device information;

[0057] S103, performing video-based point cloud compression on the 3D map data using the image compression constant to obtain a map data packet, and sending the map data packet to the target vehicle; so that the target vehicle obtains the two-dimensional images around the vehicle body, parses the map data packet to perform 3D map reconstruction to obtain the original 3D map, and aligns and fuses the two-dimensional images with the original 3D map to obtain the target 3D map.

[0058] The data request includes the location information of the target vehicle and the display device information; the map data packet includes a geometry map, an occupancy map, and an attribute map, and the color attribute data is removed from the attribute map.

[0059] A lightweight method for a 3D asset model applied to a cloud server provided by an embodiment of the present invention determines an image compression constant according to the current network state (transmission delay) of the target vehicle and the display device information. The vehicle obtains a map data packet from the cloud to generate an original colorless 3D map, and then colors the original 3D map with the two-dimensional images collected locally by the vehicle to obtain a target 3D map. By dynamically determining the image compression constant and removing the color attribute data, the size of the map data packet is reduced, the transmission efficiency is improved, the two-dimensional images sensed by the vehicle in real time are combined with the three-dimensional map data provided by the cloud, and the accuracy and practicality of the map are improved.

[0060] In one implementation, the transmission delay reflects the current network state of the target vehicle. When the transmission delay is small, indicating a good network state, the image compression constant can be adjusted to retain more point cloud points when compressing the 3D map data, making the reconstructed target 3D map clearer; otherwise, a reasonable number of point cloud points are retained when compressing the 3D map data, and the target 3D map can be displayed in real time to ensure the basic 3D navigation function.

[0061] In one implementation, in Video-Based Point Cloud Compression (V-PCC), the geometry map, the occupancy map, and the attribute map are three important components.

[0062] The geometry map stores the spatial position information of the point cloud data and is related to point cloud reconstruction geometry. It is obtained by projecting the geometric information of the point cloud video onto a two-dimensional plane. The geometry map plays an important role in the V-PCC coding standard because it not only directly reflects the spatial structure of the point cloud but also is used to guide the generation of the attribute map. During the coding process, the coding quality of the geometry map will directly affect the coding quality of the attribute map.

[0063] The occupancy map is used in V-PCC to represent the occupancy of the point cloud in three-dimensional space. It is usually a binary image, where white pixels represent the space occupied by the point cloud and black pixels represent the unoccupied space. The occupancy map plays a key role in the encoding process because it helps the encoder identify which regions are valid point cloud data and which regions are invalid or background data. This helps the encoder allocate encoding resources more efficiently, thereby improving the compression efficiency. In V-PCC, the occupancy map is usually generated from the geometry map. By performing thresholding or segmentation operations on the geometry map, the occupancy map can be obtained. The occupancy map is further used to generate the depth map and texture map, which are used in the subsequent encoding process to recover the geometric and attribute information of the point cloud.

[0064] The attribute map stores the attribute information of the point cloud data, such as color, reflectivity, normal, etc. This attribute information is crucial for the visual rendering and subsequent processing of the point cloud. In V-PCC, the attribute map is usually obtained by projecting the attribute information of the point cloud onto a two-dimensional plane corresponding to the geometry map. The encoding quality of the attribute map directly affects the visual effect of the point cloud video. During the encoding process, the attribute map is encoded based on the geometry map and the occupancy map. In the embodiments of the present invention, the attribute map only contains reflectivity and normal attribute information and does not contain color attribute information. Removing the color attribute information can greatly reduce the data volume of the map data packet and improve the transmission timeliness, without affecting the reconstruction of the original 3D map, and the original 3D map can also implement the 3D navigation function.

[0065] In one embodiment, the image compression constant includes a scaling constant and a quantization parameter; determining the image compression constant according to the transmission delay and display device information includes:

[0066] Querying a preset quantization parameter table according to the transmission delay to obtain the quantization parameter; the quantization parameter table records the correspondence between the transmission delay and the quantization parameter;

[0067] Querying a preset scaling constant table according to the display device information to obtain the scaling constant; the scaling constant table records the correspondence between the display device information and the scaling constant.

[0068] In one implementation, the scaling constant is usually used in V-PCC to adjust the scale of the point cloud data to better adapt to the processing range of the encoder. The quantization parameter (Quantization Parameter, QP) is a parameter used in V-PCC to control the quantization step size during the encoding process. Quantization is the process of mapping the continuous values of a signal into multiple discrete amplitudes. The size of the quantization parameter directly determines the size of the quantization step, which in turn affects the data volume and quality after encoding. Both the scaling constant and the quantization parameter affect the data volume of the compressed map data packet.

[0069] The display device information is the size of the target vehicle's display device. The scale of the point cloud data is determined by the size of the display device to optimize data transmission and display effects.

[0070] In one implementation, the preset quantization parameter table and scaling constant table are set by technicians according to actual needs. The quantization parameter is proportional to the transmission delay, and the value range of the quantization parameter is usually between 0 and 51. The scaling constant is inversely proportional to the size of the target vehicle display device, and the scaling constant is between 0.01 and 1.

[0071] In one embodiment, see Figure 2 , Figure 2 The flowchart of performing video-based point cloud compression on 3D map data provided by an embodiment of the present invention. Performing video-based point cloud compression on 3D map data using image compression constants to obtain a map data packet includes:

[0072] Step 1: Calculate the target data preservation rate and target frame utilization rate of compression according to the scaling constant and the quantization parameter, divide the point cloud of the 3D map data into a preset number of cuboids, and remove the cuboids that do not contain point cloud points to obtain an initial cuboid set;

[0073] Step 2: If there is no cuboid in the initial cuboid set, execute step 5; otherwise, for each cuboid in the initial cuboid set, project the original point cloud in the cuboid into a 2D space with different angles to generate a patch;

[0074] Step 3: If the size of the cuboid is not the preset minimum size and the patch of the cuboid does not meet the preset conditions, the point cloud of the cuboid is divided into a preset number of cuboids, and the cuboids that do not contain point cloud points are removed to update the initial cuboid set, and then return to execute step 2; the preset conditions are whether the data preservation rate and frame utilization rate of the patch relative to the original point cloud are greater than the target data preservation rate and target frame utilization rate respectively;

[0075] Step 4: If the size of the cuboid is the preset minimum size, or the patch of the cuboid meets the preset conditions, the cuboid is removed from the initial cuboid set and divided into the target cuboid set, and the process returns to step 2;

[0076] Step 5: Put the patches corresponding to each cuboid in the target cuboid set into a 2D video frame, encode the 2D video frame to obtain a geometry map, an occupancy map and an attribute map, and package the geometry map, the occupancy map and the attribute map into a V-PCC bit stream as a map data packet.

[0077] In one implementation, the target data preservation rate reflects the information loss during the encoding process, that is, to what extent the encoded data can restore the geometric shape and attribute information of the original point cloud data. The higher the target data preservation rate, the closer the encoded data is to the original data, and the less information loss. However, correspondingly, the amount of encoded data may also be larger. In practical applications, the target data preservation rate is usually not less than 87%.

[0078] In one implementation, the target frame utilization rate is used to measure the utilization efficiency of frame resources during the encoding process. It reflects how the encoder effectively uses each frame to transmit information when processing point cloud data. In V-PCC, the point cloud data is mapped into a series of video frames, and the encoding of the point cloud data is achieved by compressing these video frames. The higher the target frame utilization rate, the fewer frames required by the encoder to transmit the same amount of information, thereby being able to more effectively utilize frame resources and reduce the amount of encoded data. In practical applications, the target frame utilization rate is usually higher than 75%.

[0079] In one implementation, the preset quantity can be set to 8, and the preset minimum size is 1 / 64 of the point cloud volume of the 3D map data.

[0080] In one embodiment, projecting the original point cloud within the cuboid into 2D spaces with different angles to generate patches includes:

[0081] Projecting the original point cloud within the cuboid onto 2D spaces according to the six faces of the cuboid to obtain six 2D images;

[0082] Determine the 2D image that captures the most point cloud points among the six 2D images as the first image, and the 2D image with the second most projected points as the second image;

[0083] Determine the point cloud points in the original point cloud within the cuboid that are not projected onto the first image to generate a third image;

[0084] Fuse the first image, the second image, and the third image to obtain the patch of the cuboid.

[0085] In one implementation, the original point cloud within the cuboid is respectively saved in the first image and the third image, and the first image and the third image are enhanced by the second image, which can alleviate the discontinuity at the boundaries between patches and improve the image quality of 3D map reconstruction.

[0086] In one embodiment, fusing the first image, the second image, and the third image to obtain the patch of the cuboid includes:

[0087] Determine the blank points in the third image and fill them with the point cloud points at the corresponding positions in the first image to obtain a fourth image;

[0088] The union of the first image and the fourth image is obtained to get the fifth image, and the fifth image is added to the second image to obtain the target image as the patch of the cuboid.

[0089] In one embodiment, calculating the compressed target data saving rate and the target frame utilization rate according to the scaling constant and the quantization parameter is specifically as follows: Where B is the target data saving rate, Z is the target frame utilization rate, a is the scaling constant, S is the quantization parameter, and ω is a preset constant.

[0090] In one implementation, ω is taken as 100.

[0091] The embodiment of the present invention provides a lightweight method for a 3D asset model applied to a target vehicle. Refer to Figure 3 , Figure 3 which is a flowchart of a lightweight method for a 3D asset model applied to a target vehicle. The method includes:

[0092] S301, sending a data request to the cloud server; so that the cloud server determines the transmission delay of the data request, obtains the 3D map data of the corresponding road according to the location information, determines the image compression constant according to the transmission delay and the display device information, uses the image compression constant to perform video-based point cloud compression on the 3D map data to obtain a map data packet, and sends the map data packet to the target vehicle;

[0093] S302, obtaining the two-dimensional images around the vehicle body, parsing the map data packet to perform 3D map reconstruction to obtain the original 3D map, and aligning and fusing the two-dimensional images with the original 3D map to obtain the target 3D map.

[0094] The data request includes the location information and the display device information of the target vehicle; the map data packet includes a geometry map, an occupancy map, and an attribute map, and the color attribute data is removed from the attribute map.

[0095] The lightweight method for a 3D asset model applied to the cloud server provided by the embodiment of the present invention determines the image compression constant according to the current network state (transmission delay) of the target vehicle and the display device information. The vehicle obtains the colorless original 3D map by generating the map data packet from the cloud, and then colors the original 3D map with the two-dimensional images collected locally by the vehicle to obtain the target 3D map. By dynamically determining the image compression constant and removing the color attribute data, the size of the map data packet is reduced, the transmission efficiency is improved, and the two-dimensional images sensed by the vehicle in real time and the three-dimensional map data provided by the cloud are combined, improving the accuracy and practicality of the map.

[0096] In one embodiment, the image compression constant includes a scaling constant and a quantization parameter; the quantization parameter is obtained by querying a preset quantization parameter table according to the transmission delay; the quantization parameter table records the corresponding relationship between the transmission delay and the quantization parameter; the scaling constant is obtained by querying a preset scaling constant table according to the display device information; the scaling constant table records the corresponding relationship between the display device information and the scaling constant.

[0097] In one embodiment, the format of the map data packet is a V-PCC bitstream;

[0098] Parsing the map data packet to perform 3D map reconstruction to obtain the original 3D map, and aligning and fusing the two-dimensional image with the original 3D map to obtain the target 3D map includes:

[0099] Extracting the occupancy bitstream, geometric bitstream, and attribute bitstream from the map data packet;

[0100] Decoding these three bitstreams to obtain an occupancy map, a geometric image, and an initial attribute image, and generating three video frame sequences;

[0101] Performing geometric reconstruction, smoothing, etc. on the occupancy map, geometric image, and attribute image to obtain a reconstructed point cloud as the original 3D map;

[0102] Aligning the two-dimensional image with the original 3D map, extracting the color features of the two-dimensional image, and recoloring the original 3D map according to the color features to obtain the target 3D map.

[0103] In one implementation, aligning the two-dimensional image with the original 3D map can perform the following steps: identifying the common feature points (at least three) of the two-dimensional image and the 3D map; establishing a coordinate system matching the 3D map according to the scales of the two-dimensional image and the 3D map, and performing common feature point matching in the same coordinate system; using the spatial correction tool in ArcGIS to align the two-dimensional image and the 3D map. Using the georeferencing tool to specify the spatial positions of each point of the two-dimensional image.

[0104] The above has described a specific embodiment of the present invention in detail, but the described content is only a preferred embodiment of the present invention and cannot be considered as limiting the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of the present invention application should still fall within the scope covered by the patent of the present invention.

Claims

1. A lightweight system for 3D asset models, characterized in that: The system includes a cloud server and a target vehicle; wherein: The target vehicle sends a data request to the cloud server; the data request includes the location information and display device information of the target vehicle; The cloud server determines a transmission delay of the data request, obtains 3D map data of the corresponding road according to the location information, determines an image compression constant according to the transmission delay and the display device information, uses the image compression constant to perform video-based point cloud compression on the 3D map data to obtain a map data packet, and sends the map data packet to the target vehicle; the map data packet includes a geometric map, an occupancy map, and an attribute map, and the attribute map removes color attribute data; The target vehicle obtains a two-dimensional image around the vehicle body, parses the map data packet to perform 3D map reconstruction to obtain an original 3D map, and aligns and fuses the two-dimensional image with the original 3D map to obtain a target 3D map; The step of using an image compression constant to perform video-based point cloud compression on the 3D map data to obtain a map data packet includes: Step 1: Calculate the target data preservation rate and target frame utilization rate of compression according to the scaling constant and the quantization parameter, divide the point cloud of the 3D map data into a preset number of cuboids, and remove the cuboids that do not contain point cloud points to obtain an initial cuboid set; Step 2: If there is no cuboid in the initial cuboid set, execute step 5; otherwise, for each cuboid in the initial cuboid set, project the original point cloud in the cuboid into a 2D space with different angles to generate a patch; Step 3: If the size of the cuboid is not a preset minimum size and the patch of the cuboid does not meet the preset conditions, the point cloud of the cuboid is divided into a preset number of cuboids, and the cuboids that do not contain point cloud points are removed to update the initial cuboid set, and the process returns to step 2; the preset conditions are whether the data preservation rate and frame utilization rate of the patch relative to the original point cloud are greater than the target data preservation rate and target frame utilization rate, respectively; Step 4: If the size of the cuboid is the preset minimum size, or the patch of the cuboid meets the preset conditions, the cuboid is removed from the initial cuboid set and divided into the target cuboid set, and the process returns to step 2; Step 5: Put the patches corresponding to each cuboid in the target cuboid set into a 2D video frame, encode the 2D video frame to obtain a geometry map, an occupancy map, and an attribute map, and package the geometry map, the occupancy map, and the attribute map into a V-PCC bit stream as a map data packet; Projecting the original point cloud in the cuboid into 2D space with different angles to generate patches includes: According to the six faces of the cuboid, the original point cloud in the cuboid is projected into the 2D space to obtain six 2D images; Determine the 2D image that captures the most point cloud points among the six 2D images as the first image, and the 2D image that captures the second most projected points as the second image; Determine the point cloud points in the original point cloud within the cuboid that are not projected onto the first image to generate a third image; The first image, the second image and the third image are fused to obtain the patch of the cuboid.

2. A lightweight method for a 3D asset model, characterized in that: The method is applied to a cloud server, and the method includes: Receiving a data request sent by a target vehicle; the data request includes location information and display device information of the target vehicle; Determine a transmission delay of the data request, obtain 3D map data of a corresponding road according to the location information, and determine an image compression constant according to the transmission delay and the display device information; Using an image compression constant to perform video-based point cloud compression on the 3D map data to obtain a map data packet, and sending the map data packet to the target vehicle; the map data packet includes a geometric map, an occupancy map, and an attribute map, and the attribute map removes color attribute data; so that the target vehicle obtains a two-dimensional image around the vehicle body, parses the map data packet to perform 3D map reconstruction to obtain an original 3D map, and aligns and fuses the two-dimensional image with the original 3D map to obtain a target 3D map; The step of using an image compression constant to perform video-based point cloud compression on the 3D map data to obtain a map data packet includes: Step 1: Calculate the target data preservation rate and target frame utilization rate of compression according to the scaling constant and the quantization parameter, divide the point cloud of the 3D map data into a preset number of cuboids, and remove the cuboids that do not contain point cloud points to obtain an initial cuboid set; Step 2: If there is no cuboid in the initial cuboid set, execute step 5; otherwise, for each cuboid in the initial cuboid set, project the original point cloud in the cuboid into a 2D space with different angles to generate a patch; Step 3: If the size of the cuboid is not a preset minimum size and the patch of the cuboid does not meet the preset conditions, the point cloud of the cuboid is divided into a preset number of cuboids, and the cuboids that do not contain point cloud points are removed to update the initial cuboid set, and the process returns to step 2; the preset conditions are whether the data preservation rate and frame utilization rate of the patch relative to the original point cloud are greater than the target data preservation rate and target frame utilization rate, respectively; Step 4: If the size of the cuboid is the preset minimum size, or the patch of the cuboid meets the preset conditions, the cuboid is removed from the initial cuboid set and divided into the target cuboid set, and the process returns to step 2; Step 5: Put the patches corresponding to each cuboid in the target cuboid set into a 2D video frame, encode the 2D video frame to obtain a geometry map, an occupancy map, and an attribute map, and package the geometry map, the occupancy map, and the attribute map into a V-PCC bit stream as a map data packet; Projecting the original point cloud in the cuboid into 2D space with different angles to generate patches includes: According to the six faces of the cuboid, the original point cloud in the cuboid is projected into the 2D space to obtain six 2D images; Determine the 2D image that captures the most point cloud points among the six 2D images as the first image, and the 2D image that captures the second most projected points as the second image; Determine the point cloud points in the original point cloud within the cuboid that are not projected onto the first image to generate a third image; The first image, the second image and the third image are fused to obtain the patch of the cuboid.

3. The lightweight method of a 3D asset model according to claim 2, characterized in that: The image compression constant includes a scaling constant and a quantization parameter; and determining the image compression constant according to the transmission delay and the display device information includes: According to the transmission delay, a preset quantization parameter table is queried to obtain a quantization parameter; the quantization parameter table records the corresponding relationship between the transmission delay and the quantization parameter; A preset scaling constant table is queried according to the display device information to obtain the scaling constant; the scaling constant table records the corresponding relationship between the display device information and the scaling constant.

4. The lightweight method of a 3D asset model according to claim 3, characterized in that: The patch of the cuboid obtained by fusing the first image, the second image and the third image includes: Determine the blank points in the third image and fill them with the point cloud points at the corresponding positions in the first image to obtain a fourth image; The first image and the fourth image are unioned to obtain a fifth image, and the fifth image and the second image are added to obtain a target image as a patch of the cuboid.

5. The lightweight method of a 3D asset model according to claim 4, characterized in that: The target data preservation rate and target frame utilization rate of compression are calculated based on the scaling constant and quantization parameter: Where B is the target data preservation rate, Z is the target frame utilization rate, a is the scaling constant, S is the quantization parameter, and ω is a preset constant.

6. A lightweight method for a 3D asset model, characterized in that: The method is applied to a target vehicle, and the method comprises: Sending a data request to a cloud server; the data request includes the location information and display device information of the target vehicle; so that the cloud server determines the transmission delay of the data request, obtains the 3D map data of the corresponding road according to the location information, determines the image compression constant according to the transmission delay and the display device information, uses the image compression constant to perform video-based point cloud compression on the 3D map data to obtain a map data packet, and sends the map data packet to the target vehicle; the map data packet includes a geometric map, an occupancy map, and an attribute map, and the attribute map removes color attribute data; Acquire a two-dimensional image around the vehicle body, parse the map data packet to perform 3D map reconstruction to obtain an original 3D map, and align and fuse the two-dimensional image with the original 3D map to obtain a target 3D map; The step of using an image compression constant to perform video-based point cloud compression on the 3D map data to obtain a map data packet includes: Step 1: Calculate the target data preservation rate and target frame utilization rate of compression according to the scaling constant and the quantization parameter, divide the point cloud of the 3D map data into a preset number of cuboids, and remove the cuboids that do not contain point cloud points to obtain an initial cuboid set; Step 2: If there is no cuboid in the initial cuboid set, execute step 5; otherwise, for each cuboid in the initial cuboid set, project the original point cloud in the cuboid into a 2D space with different angles to generate a patch; Step 3: If the size of the cuboid is not a preset minimum size and the patch of the cuboid does not meet the preset conditions, the point cloud of the cuboid is divided into a preset number of cuboids, and the cuboids that do not contain point cloud points are removed to update the initial cuboid set, and the process returns to step 2; the preset conditions are whether the data preservation rate and frame utilization rate of the patch relative to the original point cloud are greater than the target data preservation rate and target frame utilization rate, respectively; Step 4: If the size of the cuboid is the preset minimum size, or the patch of the cuboid meets the preset conditions, the cuboid is removed from the initial cuboid set and divided into the target cuboid set, and the process returns to step 2; Step 5: Put the patches corresponding to each cuboid in the target cuboid set into a 2D video frame, encode the 2D video frame to obtain a geometry map, an occupancy map, and an attribute map, and package the geometry map, the occupancy map, and the attribute map into a V-PCC bit stream as a map data packet; Projecting the original point cloud in the cuboid into 2D space with different angles to generate patches includes: According to the six faces of the cuboid, the original point cloud in the cuboid is projected into the 2D space to obtain six 2D images; Determine the 2D image that captures the most point cloud points among the six 2D images as the first image, and the 2D image that captures the second most projected points as the second image; Determine the point cloud points in the original point cloud within the cuboid that are not projected onto the first image to generate a third image; The first image, the second image and the third image are fused to obtain the patch of the cuboid.

7. The lightweight method of a 3D asset model according to claim 6, characterized in that: The image compression constant includes a scaling constant and a quantization parameter; the quantization parameter is obtained by querying a preset quantization parameter table according to the transmission delay; the quantization parameter table records the corresponding relationship between the transmission delay and the quantization parameter; The scaling constant is obtained by querying a preset scaling constant table according to the display device information; the scaling constant table records the corresponding relationship between the display device information and the scaling constant.

8. The lightweight method of a 3D asset model according to claim 7, characterized in that: The format of the map data packet is a V-PCC bit stream; Parsing the map data packet to reconstruct the 3D map to obtain an original 3D map, and aligning and fusing the two-dimensional image with the original 3D map to obtain a target 3D map includes: extracting an occupancy bitstream, a geometry bitstream and an attribute bitstream from the map data packet; Decoding the three bit streams to obtain an occupancy map, a geometric image, and an initial attribute image, and generating three video frame sequences; Perform geometric reconstruction and smoothing on the occupancy map, geometric image and attribute image to obtain the reconstructed point cloud as the original 3D map; The two-dimensional image is aligned with the original 3D map, color features of the two-dimensional image are extracted, and the original 3D map is recolored according to the color features to obtain a target 3D map.

Images