A point cloud information processing method, device, equipment and storage medium

Abstract

The application discloses a point cloud information processing method, device and equipment and a storage medium, and comprises the following steps: acquiring current point cloud frame data, wherein the current point cloud frame data comprises three-dimensional coordinate information of a plurality of point clouds; performing two-dimensional space mapping processing on the three-dimensional coordinate information of the plurality of point clouds to obtain a current depth image corresponding to the current point cloud frame data; comparing the current depth image with a previous depth image to obtain a frame difference image, wherein the previous depth image is a depth image corresponding to previous point cloud frame data adjacent to the current point cloud frame data; determining a target sending image from the current depth image and the frame difference image with smaller data quantity; and sending the target sending image to an image rendering module. The application can effectively reduce the data quantity of point cloud data, facilitate the storage and sending of point cloud data, and ensure real-time transmission of point cloud data.

Description

Technical Field

[0001] This invention relates to the field of digital maps, specifically to a point cloud information processing method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid development of 3D data acquisition equipment, it has been widely used in many fields, especially the autonomous driving industry. Point cloud is the definition of a set of points in three-dimensional space. Point cloud involves a huge amount of data. Based on the fact that a point has at least 3 coordinates and an additional set of attributes, such as color information, the data size of a single point can reach more than 20 bytes. A single frame of point cloud data generated by a LiDAR contains hundreds of thousands of points. If the scanning frequency is 10 frames per second, the amount of data generated in 1 second will reach more than 300MB. The huge amount of point cloud data is inconvenient to store and send. Summary of the Invention

[0003] To overcome the shortcomings and deficiencies of existing technologies, this invention discloses a point cloud information processing method, apparatus, device, and storage medium, which can effectively reduce the amount of point cloud data, facilitate the storage and transmission of point cloud data, and thus ensure real-time transmission of point cloud data. The method includes:

[0004] Acquire the current point cloud frame data, which includes the three-dimensional coordinate information of multiple point clouds;

[0005] The three-dimensional coordinate information of the multiple point clouds is processed by two-dimensional spatial mapping to obtain the current depth image corresponding to the current point cloud frame data;

[0006] The current depth image is compared with the previous depth image to obtain a frame difference image; the previous depth image is the depth image corresponding to the previous point cloud frame data adjacent to the current point cloud frame data.

[0007] The image with smaller data volume in the current depth image and the frame difference image is used to determine the target image to be sent.

[0008] The target image is sent to the image rendering module, so that the image rendering module generates a three-dimensional real-world image corresponding to the target image.

[0009] Furthermore, before performing two-dimensional spatial mapping processing on the three-dimensional coordinate information of the plurality of point clouds to obtain the current depth image corresponding to the current point cloud frame data, the method further includes:

[0010] Based on the distances between the multiple point clouds in the current point cloud frame data and the detection origin, a target point cloud located in a preset detection area is determined; the preset detection area includes a near clipping plane that is close to the detection origin.

[0011] The depth value of the target point cloud is determined based on the distance between the target point cloud and the detection origin, and the distance between the near cut surface and the detection origin.

[0012] Further, the step of performing two-dimensional spatial mapping processing on the three-dimensional coordinate information of the plurality of point clouds to obtain the current depth image corresponding to the current point cloud frame data includes:

[0013] The three-dimensional coordinate information of the target point cloud is processed by two-dimensional spatial mapping to obtain the two-dimensional coordinate information of the target point cloud in the current depth image;

[0014] Based on the two-dimensional coordinate information of the target point cloud in the current depth image and the depth value of the target point cloud, the current depth image corresponding to the current point cloud frame data is obtained.

[0015] Further, before obtaining the current depth image corresponding to the current point cloud frame data based on the two-dimensional coordinate information of the target point cloud in the current depth image and the depth value of the target point cloud, the process includes:

[0016] Based on the two-dimensional coordinate information of the target point cloud in the current depth map, the depth value of the target point cloud is stored to establish a correspondence between the depth value of the target point cloud and the two-dimensional coordinate information of the target point cloud.

[0017] Furthermore, determining the target transmission image based on the current depth image and the frame difference image further includes:

[0018] Given the detection start time, the target image is determined to be the current depth image.

[0019] Further, the step of sending the target image to the image rendering module, so that the image rendering module generates a three-dimensional real-world image corresponding to the target image, includes:

[0020] The target image is sent to the image rendering module, so that the image rendering module determines the current depth image to be processed based on the image type of the target image, and performs image rendering on the current depth image to be processed to obtain the three-dimensional real scene image corresponding to the target image.

[0021] Further, the step of sending the target image to the image rendering module, so that the image rendering module generates a three-dimensional real-world image corresponding to the target image, includes:

[0022] The target image is sent to the image rendering module, so that when the image type is the frame difference image, the image rendering module obtains the current depth image to be processed based on the frame difference image and the associated depth image, and performs image rendering on the current depth image to be processed to obtain the three-dimensional real scene image corresponding to the target image.

[0023] On the other hand, this embodiment also provides a point cloud information processing device, including:

[0024] The acquisition module is used to acquire the current point cloud frame data, which includes the three-dimensional coordinate information of multiple point clouds;

[0025] The mapping processing module is used to perform two-dimensional spatial mapping processing on the three-dimensional coordinate information of the multiple point clouds to obtain the current depth image corresponding to the current point cloud frame data.

[0026] The information comparison module is used to compare the difference information between the current depth image and the previous depth image to obtain a frame difference image; the previous depth image is the depth image corresponding to the previous point cloud frame data adjacent to the current point cloud frame data;

[0027] The determination module identifies the target image to be sent by comparing the current depth image with the image with the smaller data volume in the frame difference image.

[0028] The sending module sends the target image to the image rendering module, so that the image rendering module generates a three-dimensional real-world image corresponding to the target image.

[0029] Thirdly, this application also provides an electronic device, the device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement a point cloud information processing method as described above.

[0030] Fourthly, this embodiment provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded by a processor and executed as described above in a point cloud information processing method.

[0031] Implementing this invention has the following beneficial effects:

[0032] This application maps the 3D coordinate information of multiple point clouds in the current point cloud frame data from 3D space to 2D space to obtain the current depth image. The 2D coordinate information of multiple point clouds in the depth image can represent the 3D coordinate information of multiple point clouds in 3D space. The current data frame only needs to store one depth image, which effectively reduces the amount of data in the point cloud frame data. The current depth image is compared with the previous depth image to obtain a frame difference map. The data size of the current depth image and the previous depth image are compared. The image with smaller data size is sent to the image rendering module, which saves network bandwidth, improves the efficiency of point cloud data transmission, ensures the real-time transmission of point clouds, and further improves rendering efficiency. Attached Figure Description

[0033] To more clearly illustrate the technical solution of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart of a point cloud information processing method provided in an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of the target point cloud in three-dimensional space provided in an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of the target point cloud in two-dimensional space provided in an embodiment of the present invention:

[0037] Figure 4 This is a schematic diagram of the preset detection area provided in an embodiment of the present invention;

[0038] Figure 5 This is a schematic diagram of an implementation scenario provided by an embodiment of the present invention;

[0039] Figure 6 This is a schematic diagram of the point cloud information processing device provided in an embodiment of the present invention. Detailed Implementation

[0040] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0041] Traditional point cloud compression methods involve downsampling to reduce the number of points, using techniques like random downsampling and voxel downsampling. Currently, standardization organizations have developed two standardized models: Video-based Point Cloud Compression (V-PCC), which projects 3D point clouds onto a 2D plane and then compresses them using HEVC video; and Geometry-based Point Cloud Compression (G-PCC), which uses an octree structure for spatial compression. G-PCC primarily provides efficient lossless and lossy compression for point clouds used in autonomous driving, 3D mapping, and other applications employing LiDAR. Existing downsampling schemes are highly dependent on the number of points; high compression can result in too few points, leading to poor rendering quality and significant data loss. While uniform point cloud downsampling algorithms ensure rendering quality, the data volume remains large, resulting in low compression ratios and hindering effective real-time transmission. V-PCC and G-PCC algorithms achieve high compression ratios for point clouds, but the computational demands of encoding and decoding make real-time performance difficult to guarantee, leading to choppy or stuttering rendering. Currently used point cloud compression algorithms do not specifically optimize for point cloud data generated by vehicle-mounted LiDAR, resulting in less than ideal final effects. Therefore, the technical problem this embodiment aims to solve is to effectively reduce the amount of point cloud data, facilitating its storage and transmission, thereby ensuring real-time transmission. The method includes:

[0042] S110: Obtain the current point cloud frame data, which includes the three-dimensional coordinate information of multiple point clouds;

[0043] The execution entity in this embodiment is the vehicle-mounted processor, i.e., the CPU. The vehicle-mounted LiDAR generates a point cloud frame every 100ms. Each point cloud frame includes the three-dimensional coordinate information of multiple point clouds, i.e., x-axis, y-axis, and z-axis coordinates. Each coordinate is of type float and occupies 4 bytes. Therefore, the three-dimensional coordinate information of a point cloud occupies at least 12 bytes. With the addition of a set of attributes, such as color information or depth information, the data volume of a point cloud can reach more than 20 bytes. A LiDAR generates a frame of point cloud data containing hundreds of thousands of points, and the amount of data generated by the vehicle-mounted LiDAR per second reaches more than 300MB.

[0044] S120: Perform two-dimensional spatial mapping processing on the three-dimensional coordinate information of multiple point clouds to obtain the current depth image corresponding to the current point cloud frame data;

[0045] The x, y, and z coordinates of multiple point clouds in the current point cloud frame are mapped to the uv coordinates of the current depth image. One point cloud frame corresponds to one depth image, such as... Figure 2 and Figure 3 As shown, r p For the target point cloud, in the current depth map, the u-coordinate of the target electric cloud is... The v coordinate is Where d is the distance between the target point cloud and the detection origin, i.e., the lidar. The coordinates of multiple point clouds in the current depth image can represent the three-dimensional coordinate information of multiple point clouds in three-dimensional space. Only one depth image needs to be saved for a cloud frame data, without saving the three-dimensional coordinate information of multiple point clouds in a cloud frame data, thus reducing the amount of data stored.

[0046] S130: Compare the difference information between the current depth image and the previous depth image to obtain the frame difference image; the previous depth image is the depth image corresponding to the previous point cloud frame data adjacent to the current point cloud frame data;

[0047] In this embodiment, the current depth image is stored in a first two-dimensional array, and a second two-dimensional array is created. The second two-dimensional array has the same data storage capacity as the first two-dimensional array. The two-dimensional coordinate information in the current depth image and the previous depth image is traversed, and the difference information is compared. Data with differences is written into the second two-dimensional array. If the detection range of the vehicle-mounted LiDAR remains unchanged, there is almost no difference information between the current depth image and the previous depth image, that is, the data in the two-dimensional array is 0. It can be seen that, except at the detection start time, a current depth image and a frame difference image are obtained after processing each frame of point cloud data.

[0048] S140: Determine the target image to be sent by comparing the current depth image with the frame difference image where the data volume is smaller;

[0049] To reduce the amount of data transmitted, the image with less data between the current depth image and the frame difference image is selected as the target image for transmission. When the detection range of the vehicle-mounted LiDAR is within its range or when the vehicle-mounted LiDAR moves slowly (the detection range changes slowly), the amount of data in the frame difference image will be much less than the amount of data in the current depth image. Selecting the image with less data as the target image for transmission can save network bandwidth and ensure the real-time transmission of point cloud data.

[0050] S150: Send the target image to the image rendering module so that the image rendering module generates a 3D real-world image corresponding to the target image.

[0051] After receiving the target image, the image rendering module performs point cloud reconstruction and rendering based on it. Normally, point cloud data is decoded and preprocessed on the CPU or GPU (Graphics Processing Unit). After processing, the processed data is copied to the GPU's available video memory (used to store rendering data processed or about to be extracted by the graphics chip). To reduce the amount of data copying, in this embodiment, both point cloud reconstruction and rendering are handled by the GPU, avoiding large amounts of data copying. The target image obtained after CPU processing is sent to the GPU in the image rendering module for image rendering, further avoiding large amounts of data copying and ensuring real-time rendering of the 3D real-world image. Only one depth image needs to be stored for the current data frame, effectively reducing the amount of point cloud frame data. The current depth image is compared with the previous depth image to obtain a frame difference map. The data size of the current depth image and the previous depth image is compared, and the image with smaller data size is sent to the image rendering module, saving network bandwidth, improving point cloud data transmission efficiency, ensuring real-time point cloud transmission, and further improving rendering efficiency.

[0052] Furthermore, before performing two-dimensional spatial mapping processing on the three-dimensional coordinate information of multiple point clouds to obtain the current depth image corresponding to the current point cloud frame data, the method also includes:

[0053] Based on the distances between multiple point clouds in the current point cloud frame data and the detection origin, the target point cloud located in the preset detection area is determined; the preset detection area includes near clipping surfaces that are close to the detection origin.

[0054] Determine the preset detection area, refer to Figure 4 The detection area includes the near and far cutting planes relative to the detection origin. The area between the near and far cutting planes is the preset detection area. Point clouds outside the preset detection area are removed to obtain the target point cloud. Only the data of the target point cloud needs to be processed, which reduces the amount of data to be processed and avoids interference from other noise points in the processing of the target point cloud.

[0055] The depth value of the target point cloud is determined based on the distance between the target point cloud and the detection origin, as well as the distance between the near cut surface and the detection origin.

[0056] The distance between the target point cloud and the detection origin, i.e., the distance between the target point cloud and the vehicle-mounted LiDAR, and the distance between the near clipping plane and the detection origin in 3D space, can be used to obtain the original depth value of the target point cloud. Normalizing this original depth value yields the depth value, which is the pixel value. The calculation formula is as follows:

[0057] depth=(near / d–near / far) / (1–near / far)

[0058] Where, near is the distance between the near clipping plane and the detection origin, far is the distance between the far clipping plane and the detection origin, d is the distance between the target point cloud and the vehicle-mounted LiDAR, and near / d is the original depth value. The original depth value is normalized so that the distance between the near clipping plane and the detection origin is 0, and the distance between the far clipping plane and the detection origin is 1. That is, the minimum distance between the target point cloud and the detection origin is when it is located on the near clipping plane, and the maximum distance is when it is located on the far clipping plane. In this embodiment, the distance value can be an actual value or a relative value.

[0059] In one implementation, the three-dimensional coordinate information of multiple point clouds is subjected to two-dimensional spatial mapping processing to obtain a current depth image corresponding to the current point cloud frame data, including:

[0060] The three-dimensional coordinate information of the target point cloud is processed by two-dimensional spatial mapping to obtain the two-dimensional coordinate information of the target point cloud in the current depth image;

[0061] The x, y, and z coordinates of multiple point clouds in the current point cloud frame are mapped to the uv coordinates of the current depth image. One point cloud frame corresponds to one depth image, such as... Figure 2 and Figure 3 As shown, r p For the target point cloud, in the current depth map, the u-coordinate of the target power source is: The v coordinate is The two-dimensional coordinate information of the target point cloud in the current depth image is: the u coordinate is The v coordinate is

[0062] Based on the two-dimensional coordinate information of the target point cloud in the current depth image and the depth value of the target point cloud, the current depth image corresponding to the current point cloud frame data is obtained.

[0063] Storing depth values ​​in the current depth image means that only one depth image and the depth value of the target electric cloud within that image are needed for a single point cloud frame. This eliminates the need to store coordinate information. The depth value can be selected based on accuracy requirements: 8-bit, 16-bit, or 32-bit. Testing showed that using 16-bit depth storage can represent a distance range of 0 to 65535, requiring only 2 bytes to meet most accuracy requirements. The storage format for the current depth image and the frame difference image mainly includes: a file header and depth data. The file header primarily describes the depth map's attribute information, such as the data frame number and frame type, as shown in the table.

[0064] Table 1: Byte Usage Lookup Table

[0065] name type Occupied bytes illustrate fileSignature string 6 Fixed as "DDCPCD" headerSize int 4 File header size id long 8 Point cloud frame number type char 1 0: Original image 1: Residual image nearPlane float 4 Near cut surface farPlane float 4 Remote cutting surface xRange int 4 Horizontal scanning accuracy yRange int 4 Vertical scanning accuracy xOffset float 4 Horizontal angle offset yOffset float 4 Vertical angle offset

[0066] In one implementation, before obtaining the current depth image corresponding to the current point cloud frame data based on the two-dimensional coordinate information of the target point cloud in the current depth image and the depth value of the target point cloud, the following steps are included:

[0067] Based on the two-dimensional coordinate information of the target point cloud in the current depth map, the depth value of the target point cloud is stored to establish the correspondence between the depth value of the target point cloud and the two-dimensional coordinate information of the target point cloud.

[0068] Accordingly, the depth value of the target point cloud is stored at its position in the current depth image to establish a correspondence between the depth value and the two-dimensional coordinate information of the target point cloud. Since the target point cloud is part of the image content in the current depth image, it is not necessary to store the two-dimensional coordinate information of the target point cloud; only the depth value needs to be stored. For one frame of point cloud data, only one depth image needs to be stored. The current depth image contains the depth value of the target point cloud. When the image rendering module performs 3D point cloud reconstruction, the three-dimensional coordinate information of the target point cloud is restored by inversely reconstructing the image. The pixels (depth values) are ordered in the depth image. The UV coordinates of the pixel can be calculated based on this order, which is the two-dimensional coordinate information of the target point cloud in the current depth image. Then, the three-dimensional coordinate information of the target point cloud is further calculated. Based on the two-dimensional coordinate information of the target point cloud in the current depth image, storing the depth value of the target point cloud can effectively reduce the amount of data stored in the point cloud data.

[0069] In one implementation, determining the target transmission image based on the current depth image and the frame difference image further includes:

[0070] Given the initial detection time, the image sent by the target is determined to be the image at the current depth.

[0071] In this embodiment, the image with smaller data volume in the frame difference image and the current depth image is selected as the target transmission object. At the start of detection, i.e. the first frame depth image, the current depth image is directly selected as the target transmission image and sent to the image rendering module. This provides an object for comparing the read data volume of the second frame depth image, so as to generate a frame difference image of the second frame depth image compared with the first frame depth image. The smaller data volume is sent to the image rendering module to reduce the amount of data sent.

[0072] In one implementation, a target image is sent to an image rendering module, which then generates a 3D real-world image corresponding to the target image, including:

[0073] The target image is sent to the image rendering module, which then determines the current depth image to be processed based on the image type of the target image, and performs image rendering on the current depth image to obtain the 3D real-world image corresponding to the target image.

[0074] After receiving the target image, the image rendering module determines the current depth image to be processed based on the image type. The current depth image to be processed is the image that needs to be rendered. The image type of the target image can be determined based on its attributes. If the target image is determined to be the current depth image based on its image type, then the current depth image to be processed is the current depth image. If the target image is determined to be a frame difference image based on its image type, then the frame difference image is fused with the previous depth image to be processed cached by the image rendering module to obtain the current depth image to be processed. By determining the image type of the target image, the current depth image to be processed is determined so that the data contained in the current depth image to be processed is consistent with the data contained in the current depth image. This reduces the amount of data sent and saves network bandwidth while ensuring that the generated 3D real-world image meets the visualization requirements of the terminal.

[0075] Before rendering the image, 3D point cloud reconstruction is required based on the target transmitted image. The target transmitted image and its attribute information are used as input parameters. Based on the current depth image to be processed, the distance *d* between the target point cloud and the detection origin is calculated. Then, based on the depth values ​​of the target point cloud, the 3D coordinates of the target point cloud in 3D space can be determined. For the depth values ​​of each point cloud in the current depth image to be processed, the distance *d* between the target point cloud and the detection origin in 3D space is determined. The calculation formula is as follows:

[0076] d=near / (depth*(1-near / far)+near / far)

[0077] Where near is the distance between the near cut surface and the detection origin, far is the distance between the far cut surface and the detection origin, and near / d is the original depth value.

[0078] Then, based on the two-dimensional coordinate information of each point cloud in the current depth image to be processed, and combined with the distance d between the target point cloud and the detection origin, the three-dimensional coordinate information of the target point cloud in three-dimensional space is obtained.

[0079] Based on the 3D coordinates of the target point cloud in 3D space and the screen pixel size of the display device, the radius of each target point cloud is adjusted so that the target point cloud can cover the target object and clearly display the outline of the target object. Furthermore, the 3D coordinates of the edge vertices of the target point cloud are calculated. Furthermore, the 3D coordinates of the edge vertices of the target point cloud are colored, which can be a fixed color or a rainbow color set according to the height. By back-digging through the current depth image to be processed, the current point cloud frame data can be obtained, that is, the 3D coordinates of multiple point clouds in the current point cloud frame in 3D space, thus completing the 3D reconstruction of the point cloud.

[0080] In one implementation, a target image is sent to an image rendering module, which then generates a 3D real-world image corresponding to the target image, including:

[0081] The target image is sent to the image rendering module, so that the image rendering module can obtain the current depth image to be processed based on the frame difference image and the associated depth image when the image type is a frame difference image. The current depth image to be processed is then rendered to obtain the three-dimensional real scene image corresponding to the target image.

[0082] After receiving the target image, the image rendering module determines the current depth image to be processed based on the image type. The image type includes the original image type and the frame difference image type. If the current depth image is the original image type, it is the current depth image and is directly determined as the current depth image to be processed. If the image type is the frame difference image type, the frame difference image is fused with the depth image to be processed cached in the previous frame by the image rendering module to obtain the current depth image to be processed.

[0083] The image rendering module sends the target image received at the start of detection as the current depth image. The current depth image of the first frame is the depth image to be processed. At the second frame, if the image rendering module sends the target image received at the start of detection as the frame difference image, the image rendering module fuses the frame difference image with the current depth image to be processed saved in the previous frame to obtain the depth image to be processed saved in the second frame. The data contained in the current depth image to be processed saved in the second frame is consistent with the current depth image of the second frame. That is, the current depth image obtained in each frame is consistent with the data contained in the depth image to be processed obtained by the image rendering module in each frame. By sending images with a small amount of data, the final generated 3D real scene image can meet the visualization requirements of the terminal, with a maximum error within 5 cm and a near error within 1 cm.

[0084] Reference Figure 5 The scene diagram shows that the vehicle-mounted laser acquires the point cloud data corresponding to the target object and uploads it to the vehicle-mounted host, i.e., the vehicle-mounted processor. The vehicle-mounted processor performs two-dimensional spatial mapping processing on each frame of point cloud data to obtain the current depth map for each frame of point cloud data. It compares the difference information between the current depth map and the previous frame depth map to obtain the frame image of the current depth map and the previous frame depth map. The display terminal can be the vehicle-mounted terminal. The image with the smaller data volume between the current depth map and the frame difference map is the target image to be sent and is sent to the vehicle-mounted terminal. The GPU of the vehicle-mounted terminal determines the image type of the target image through the attribute information of the target image and obtains the current depth image to be processed based on the image type. The CPU of the vehicle-mounted terminal reconstructs the 3D point cloud of the current depth image to be processed and renders the image to obtain a three-dimensional real scene image.

[0085] This embodiment also provides a point cloud information processing device, which can implement all the above-described method steps, see reference. Figure 6 The device includes:

[0086] The acquisition module 610 is used to acquire the current point cloud frame data, which includes the three-dimensional coordinate information of multiple point clouds.

[0087] The mapping processing module 620 is used to perform two-dimensional spatial mapping processing on the three-dimensional coordinate information of multiple point clouds to obtain the current depth image corresponding to the current point cloud frame data.

[0088] The information comparison module 630 is used to compare the difference information between the current depth image and the previous depth image to obtain the frame difference image; the previous depth image is the depth image corresponding to the previous point cloud frame data adjacent to the current point cloud frame data.

[0089] The determination module 640 determines the target image to be sent by comparing the current depth image with the frame difference image, which has a smaller amount of data.

[0090] The sending module 650 sends the target image to the image rendering module so that the image rendering module generates a three-dimensional real-world image corresponding to the target image.

[0091] In this embodiment, the point cloud information processing device further includes:

[0092] The first determining module is used to determine the target point cloud located in the preset detection area based on the distance between multiple point clouds in the current point cloud frame data and the detection origin; the preset detection area includes near clipping planes that are close to the detection origin;

[0093] The second determination module is used to determine the depth value of the target point cloud based on the distance between the target point cloud and the detection origin, and the distance between the near clipping surface and the detection origin.

[0094] The first generation module is used to perform two-dimensional spatial mapping processing on the three-dimensional coordinate information of the target point cloud to obtain the two-dimensional coordinate information of the target point cloud in the current depth image;

[0095] The second generation module is used to obtain the current depth image corresponding to the current point cloud frame data based on the two-dimensional coordinate information of the target point cloud in the current depth image and the depth value of the target point cloud.

[0096] The storage module is used to store the depth value of the target point cloud based on the two-dimensional coordinate information of the target point cloud in the current depth map, so as to establish the correspondence between the depth value of the target point cloud and the two-dimensional coordinate information of the target point cloud.

[0097] The third determination module is used to determine, at the start of detection, that the target sent image is the current depth image.

[0098] The fourth determining module is used to send the target image to the image rendering module, so that the image rendering module can determine the current depth image to be processed based on the image type of the target image, and perform image rendering on the current depth image to be processed to obtain the three-dimensional real scene image corresponding to the target image.

[0099] The fifth determining module is used to send the target image to the image rendering module, so that when the image type is a frame difference image, the image rendering module obtains the current depth image to be processed based on the frame difference image and the associated depth image, and performs image rendering on the current depth image to be processed to obtain the three-dimensional real scene image corresponding to the target image.

[0100] Implementing this embodiment has the following effects:

[0101] This embodiment maps the 3D coordinate information of multiple point clouds in the current point cloud frame data from 3D space to 2D space to obtain the current depth image. The 2D coordinate information of multiple point clouds in the depth image can represent the 3D coordinate information of multiple point clouds in 3D space. The current data frame only needs to store one depth image, which effectively reduces the amount of data in the point cloud frame data. The difference information of the current depth image and the previous depth image is compared to obtain the frame difference image. The data volume of the current depth image and the previous depth image are compared. The image with smaller data volume is sent to the image rendering module, which saves network bandwidth and ensures the real-time transmission of point clouds. The image rendering module receives the image to be sent. If the image to be sent is a frame difference image, the frame difference image is fused with the cached previous frame image, which can meet the visualization requirements of the terminal.

[0102] Embodiments of the present invention also provide an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement a point cloud information processing method as described in the method embodiment.

[0103] Embodiments of the present invention also provide a storage medium, which can be disposed in a server to store at least one instruction, at least one program, code set, or instruction set for implementing a point cloud information processing method in the method embodiments. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the point cloud information processing method provided in the above method embodiments.

[0104] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0105] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0106] The foregoing description has fully disclosed the specific embodiments of the present invention. It should be noted that any modifications made to the specific embodiments of the present invention by those skilled in the art do not depart from the scope of the claims. Accordingly, the scope of the claims is not limited to the foregoing specific embodiments.

Claims

1. A point cloud information processing method, characterized in that, Applications in automotive processors include: Acquire current point cloud frame data, which includes the three-dimensional coordinate information of multiple point clouds; the current point cloud frame data is generated by vehicle-mounted LiDAR. Based on the distances between the multiple point clouds in the current point cloud frame data and the detection origin, a target point cloud located in a preset detection area is determined; the preset detection area includes a near clipping surface close to the detection origin; based on the distance between the target point cloud and the detection origin, and the distance between the near clipping surface and the detection origin, the depth value of the target point cloud is determined; The three-dimensional coordinate information of the target point cloud is processed by two-dimensional spatial mapping to obtain the two-dimensional coordinate information of the target point cloud in the current depth image; based on the two-dimensional coordinate information of the target point cloud in the current depth image and the depth value of the target point cloud, the current depth image corresponding to the current point cloud frame data is obtained. The current depth image is compared with the previous depth image to obtain a frame difference image; the previous depth image is the depth image corresponding to the previous point cloud frame data adjacent to the current point cloud frame data. The image with smaller data volume in the current depth image and the frame difference image is used to determine the target image to be sent. The target image is sent to the image rendering module, so that the image rendering module generates a three-dimensional real-world image corresponding to the target image.

2. The point cloud information processing method according to claim 1, characterized in that, Before obtaining the current depth image corresponding to the current point cloud frame data based on the two-dimensional coordinate information of the target point cloud in the current depth image and the depth value of the target point cloud, the process includes: Based on the two-dimensional coordinate information of the target point cloud in the current depth map, the depth value of the target point cloud is stored to establish a correspondence between the depth value of the target point cloud and the two-dimensional coordinate information of the target point cloud.

3. The point cloud information processing method according to claim 1, characterized in that, The step of determining the target transmission image based on the current depth image and the frame difference image further includes: Given the detection start time, the target image is determined to be the current depth image.

4. The point cloud information processing method according to claim 1, characterized in that, The step of sending the target image to the image rendering module, so that the image rendering module generates a three-dimensional real-world image corresponding to the target image, includes: The target image is sent to the image rendering module, so that the image rendering module determines the current depth image to be processed based on the image type of the target image, and performs image rendering on the current depth image to be processed to obtain the three-dimensional real scene image corresponding to the target image.

5. The point cloud information processing method according to claim 1, characterized in that, The step of sending the target image to the image rendering module, so that the image rendering module generates a three-dimensional real-world image corresponding to the target image, includes: The target image is sent to the image rendering module, so that when the image type is the frame difference image, the image rendering module performs fusion processing based on the frame difference image and the previous frame of the depth image to be processed in the image rendering module to obtain the current depth image to be processed, and performs image rendering on the current depth image to be processed to obtain the three-dimensional real scene image corresponding to the target image.

6. A point cloud information processing device, characterized in that, include: The acquisition module is used to acquire the current point cloud frame data, which includes the three-dimensional coordinate information of multiple point clouds; The current point cloud frame data is generated by the vehicle-mounted lidar; The mapping processing module is used to determine the target point cloud located in a preset detection area based on the distance between the multiple point clouds in the current point cloud frame data and the detection origin; the preset detection area includes a near clipping plane that is close to the detection origin; The depth value of the target point cloud is determined based on the distance between the target point cloud and the detection origin, and the distance between the near clipping surface and the detection origin. The three-dimensional coordinate information of the target point cloud is processed by two-dimensional spatial mapping to obtain the two-dimensional coordinate information of the target point cloud in the current depth image; Based on the two-dimensional coordinate information of the target point cloud in the current depth image and the depth value of the target point cloud, the current depth image corresponding to the current point cloud frame data is obtained; The information comparison module is used to compare the difference information between the current depth image and the previous depth image to obtain a frame difference image; the previous depth image is the depth image corresponding to the previous point cloud frame data adjacent to the current point cloud frame data; The determination module identifies the target image to be sent by comparing the current depth image with the image with the smaller data volume in the frame difference image. The sending module sends the target image to the image rendering module, so that the image rendering module generates a three-dimensional real-world image corresponding to the target image.

7. An electronic device, characterized in that, The device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement a point cloud information processing method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor as described in any one of claims 1 to 5.

Images