Point cloud data processing method and device, electronic equipment and computer storage medium

Abstract

The embodiment of the invention discloses a point cloud data processing method and device, electronic equipment and a computer storage medium, and the method comprises the steps: obtaining point cloud data comprising a plurality of pieces of original point data, carrying out the compression conversion of the original point data based on a preset transmission protocol, determining the corresponding transmission point data, and transmitting the corresponding transmission point data to a server; and transmitting the transmission point data so as to generate a display picture based on the transmission point data. Therefore, according to the embodiment of the invention, the original point data is transmitted after being compressed and converted, so that the bandwidth can be saved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a method, apparatus, electronic device, and computer storage medium for processing point cloud data. Background Art

[0002] A point cloud is a data set, where each point in the data set represents a set of geometric coordinates and an intensity value, and the intensity value is determined according to the intensity of the returned signal recorded by the surface reflectivity of the object. When these points are combined together, a point cloud is formed, and the 3D shape or object in space is expressed through the point cloud.

[0003] However, due to the huge amount of data in the point cloud data, it will occupy a very high bandwidth during data storage and transmission, thereby affecting the efficiency of data processing and data transmission. Summary of the Invention

[0004] In view of this, an object of the embodiments of the present invention is to provide a method, apparatus, electronic device, and computer storage medium for processing point cloud data to save bandwidth.

[0005] In a first aspect, an embodiment of the present invention aims to provide a method for processing point cloud data, the method comprising:

[0006] Obtain point cloud data, where the point cloud data includes a plurality of original point data;

[0007] Perform compression conversion on the original point data based on a preset transmission protocol to determine corresponding transmission point data;

[0008] Transmit the transmission point data to generate a display screen based on each of the transmission point data.

[0009] Further, the data type of the original point data is float16.

[0010] Further, the preset transmission protocol is the GRPC protocol, and the data type of the transmission point data is uint64 or uint32.

[0011] Further, the original point data includes a plurality of coordinate data, and performing compression conversion on the original point data based on a preset transmission protocol to determine corresponding transmission point data includes:

[0012] Perform compression on each of the coordinate data to determine corresponding compressed coordinate data;

[0013] Perform bit splicing on each of the compressed coordinate data to determine corresponding transmission point data.

[0014] Further, the data type of the original point data is float16, and the compression of each coordinate data to determine the corresponding compressed coordinate data includes:

[0015] Compress each coordinate data in float16 form into the corresponding 16-bit integer data to determine the corresponding compressed coordinate data.

[0016] Further, the generation of the display screen based on each transmission point data includes:

[0017] Use at least one thread to generate a display screen based on each transmission point data.

[0018] Further, the generation of the display screen based on each transmission point data includes:

[0019] Decode and process each of the transmitted transmission point data to generate corresponding display data;

[0020] Screen and determine the rendering data corresponding to the display screen from each display data;

[0021] Generate a display screen according to the rendering data.

[0022] Further, the screening and determination of the rendering data corresponding to the display screen from each display data includes:

[0023] Screen each display data according to the point cloud generation time to determine the rendering data corresponding to the display screen.

[0024] Further, the acquisition of point cloud data includes:

[0025] Acquire point cloud data within a preset height and / or a preset range.

[0026] In a second aspect, an embodiment of the present invention aims to provide a point cloud data processing device, and the device includes:

[0027] An acquisition unit, configured to acquire point cloud data, where the point cloud data includes a plurality of original point data;

[0028] A compression unit, configured to perform compression conversion on the original point data based on a preset transmission protocol to determine corresponding transmission point data;

[0029] A transmission unit, configured to transmit the transmission point data to generate a display screen based on each transmission point data.

[0030] In a third aspect, embodiments of the present invention aim to provide an electronic device, including a memory and a processor, where the memory is used to store one or more computer program instructions, and the one or more computer program instructions are executed by the processor to implement the method described in any one of the above.

[0031] In a fourth aspect, embodiments of the present invention aim to provide a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method steps described in any one of the above are implemented.

[0032] The technical solution of the embodiments of the present invention obtains point cloud data including a plurality of original point data, performs compression conversion on the original point data based on a preset transmission protocol to determine corresponding transmitted point data; and transmits the transmitted point data to generate a display screen based on each of the transmitted point data. Thus, in this embodiment, by performing compression conversion on the original point data and then transmitting it, it is possible to save the bandwidth required for data processing and transmission, which is beneficial to improving the data processing and data transmission efficiency. Description of the Drawings

[0033] Through the following description of the embodiments of the present invention with reference to the drawings, the above and other objects, features, and advantages of the present invention will become clearer. In the drawings:

[0034] Figure 1 is a flowchart of the method for processing point cloud data according to an embodiment of the present invention;

[0035] Figure 2 is a flowchart of the method for determining transmitted point data according to an embodiment of the present invention;

[0036] Figure 3 is a flowchart of the method for generating a display screen according to an embodiment of the present invention;

[0037] Figure 4 is a flowchart of the process of processing point cloud data according to an embodiment of the present invention;

[0038] Figure 5 is a schematic diagram of the device for processing point cloud data according to an embodiment of the present invention;

[0039] Figure 6 is a schematic diagram of the electronic device according to an embodiment of the present invention. Detailed Embodiments

[0040] The present application will be described based on embodiments, but the present application is not limited to these embodiments. In the following detailed description of the present application, some specific details are described in detail. Those skilled in the art can fully understand the present application without the description of these details. In order to avoid obscuring the essence of the present application, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0041] In addition, those of ordinary skill in the art should understand that the drawings provided herein are for illustrative purposes only, and the drawings are not necessarily drawn to scale.

[0042] Unless the context clearly requires otherwise, words such as "including" and "comprising" in the entire application document should be interpreted as having an inclusive meaning rather than an exclusive or exhaustive meaning; that is, it means "including but not limited to".

[0043] In the description of the present application, it should be understood that terms such as "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0044] For the solutions described in this specification and the embodiments, if they involve personal information processing, they will all be processed on the premise of having a legal basis (such as obtaining the consent of the personal information subject, or being necessary for performing a contract, etc.), and will only be processed within the specified or agreed scope. If the user refuses to process personal information other than the necessary information required for the basic functions, it will not affect the user's use of the basic functions.

[0045] The application of point cloud data can express 3D shapes or objects in space. However, due to the large amount of data in point cloud data, it will occupy a high bandwidth when storing and transmitting data, which will affect the efficiency of data processing and data transmission. Taking the scenario of an autonomous driving vehicle as an example, point clouds are often used to express the surrounding environment of the vehicle (including road signs, obstacles, buildings, bus stops, etc.). At the same time, by displaying the point cloud distribution in real time on the front-row PAD (i.e., a tablet computer or touch screen controller located in the front row of the vehicle), it is convenient for the vehicle itself or the safety officer on the vehicle to observe the road conditions. However, due to the currently overly large amount of point cloud data, it occupies a high bandwidth when storing and transmitting data, and the data transmission to the front-row PAD is also limited by the bandwidth of the PAD network cable adapter, resulting in stuttering of the point cloud display, which is not conducive to observing the road surface conditions. In view of this, the embodiments of the present invention aim to provide a method for processing point cloud data to save bandwidth during the process of processing point cloud data, thereby improving the efficiency of data processing and transmission.

[0046] Further, in this embodiment, the processing of point cloud data in the autonomous driving scenario is taken as an example for illustration. At the same time, for the convenience of understanding the content in this embodiment, a brief introduction to the existing point cloud data processing methods will be given first. Specifically, in the existing point cloud data processing methods in autonomous driving, the collected point cloud data is usually stored in the data format of 32-bit float. Moreover, when the point cloud data is transmitted to the front-row PAD to generate a display screen showing the surrounding environment of the vehicle, the point cloud data still adopts the float data form. Among them, the point cloud data is in meters, which can ensure that the accuracy of each point in the point cloud is above the millimeter level. At the same time, since a float data fixedly occupies 4 bytes, a point in the point cloud data includes data under three coordinates of x, y, and z, corresponding to 3 data, and thus 12 bytes are required.

[0047] It should be understood that in this embodiment, the processing of point cloud data in the autonomous driving scenario is taken as an example for illustration. However, in other scenarios that require processing point cloud data (such as scenarios of target detection and semantic segmentation based on radar point cloud, etc.), the method in this embodiment is equally applicable. Therefore, the usage scenarios of the point cloud data processing method are not limited here.

[0048] Figure 1 is the flowchart of the point cloud data processing method according to the embodiment of the present invention. As Figure 1 shown, the point cloud data processing method in this embodiment includes the following steps.

[0049] In step S110, point cloud data is acquired.

[0050] In this embodiment, the point cloud data includes a plurality of original point data.

[0051] Further, for the convenience of representation, each original point data in this embodiment is identified by coordinate data in a coordinate system (such as a three-dimensional rectangular coordinate system, a spherical coordinate system, etc.). Each original point data includes a plurality of coordinate data. For example, when using three-dimensional rectangular coordinates to represent the original point data, the original point data includes three coordinate data, which are the coordinate data corresponding to the x, y, and z axis directions respectively, and the unit of the coordinate data can be meters.

[0052] Optionally, to further improve data transmission efficiency and save bandwidth, in this embodiment, when acquiring point cloud data, the point cloud data within a preset height and / or a preset range will be acquired. Specifically, when acquiring the point cloud data within the preset height, in this embodiment, the points with a height exceeding the preset height will be directly filtered out, and the data corresponding to the points with a height less than or equal to the preset height will be used as the point cloud data. When acquiring the point cloud data within the preset range, considering that only the environmental road conditions within a limited range (such as within 50 meters around the vehicle) will affect the driving of the autonomous vehicle, and the environmental factors too far from the vehicle (including roadside green belts, high-rise buildings, etc.) have little impact on the vehicle driving decision-making, in this embodiment, the point cloud data within the preset range will be acquired. Thus, in this embodiment, while ensuring that the point cloud data does not affect the overall decision-making effect, the above method can reduce the number of points for data transmission and processing, save the bandwidth used for data storage and transmission, and is conducive to improving the overall data processing and transmission efficiency.

[0053] Optionally, in this embodiment, to further save bandwidth and improve the overall data processing and transmission efficiency, for the points within the preset range in terms of height, in this embodiment, the point cloud data can be acquired by means of downsampling. For example, the downsampling is performed at an average ratio of 1 / 2. Among them, downsampling, also known as decimation, is a technique of multi-rate digital signal processing or a process of reducing the signal sampling rate, usually used to reduce the data transmission rate or the data size. Thus, through downsampling, the data volume can be further reduced, and further bandwidth can be saved.

[0054] Furthermore, the value of the preset height in this embodiment can be set according to experience or adjusted according to the actual usage scenario. For example, the value of the preset height can be adjusted according to the height of the autonomous vehicle model. At the same time, the size and shape of the preset range can be set according to experience. For example, the preset range is a range within 50 meters around the vehicle, and the preset range shape is a hemispherical shape, etc., or can be selected according to the actual usage scenario. For example, the size of the preset range can be enlarged or reduced according to the volume of the autonomous vehicle. Here, the specific setting methods of the preset height and the preset range are not limited, so as to provide a variety of flexible setting methods while improving the overall processing (including data processing and data transmission, etc.) efficiency of the point cloud data, and further facilitate the processing of the point cloud data.

[0055] In step S120, based on a preset transmission protocol, the original point data is compressed and converted to determine the corresponding transmitted point data.

[0056] In this embodiment, the data type of the original point data is float16. After obtaining the original data of each point, the original point data of each point will be stored in the data form of float16. Compared with the existing point cloud data in the form of float data, since float16 is a data compression method that reduces the video memory occupancy of floating-point numbers, the bandwidth occupied by the point cloud data can be reduced in this embodiment.

[0057] Optionally, the preset transmission protocol in this embodiment is the GRPC protocol. Among them, the full name of the GRPC protocol is Google Remote Procedure Call, which provides a data transmission method that defines services based on the Interface Definition Language (IDL) and automatically generates specific server-side and client-side codes during compilation. It enables the server and client to fully focus on the business without caring about the details of the communication protocol. It has the advantages of supporting multiple authentication and authorization mechanisms (such as TLS-based authentication, OAuth2 authorization, etc.); supporting the use of the HTTP / 2 protocol, including functions such as connection multiplexing, bidirectional streaming, server push, request prioritization, and header compression, which can save bandwidth, reduce the number of TCP connections, save CPU, and help mobile devices extend battery life, etc.; using the existing semantics of HTTP2 in the protocol design, sending the request and response data using the HTTP Body, and representing other control information with Headers; defining services using ProtoBuf. ProtoBuf is a data serialization protocol developed by Google (similar to XML, JSON, hessian), which can serialize data, has high compression and transmission efficiency, simple syntax, strong expressiveness; and supports multiple languages, etc. Therefore, in this embodiment, by adopting the GRPC protocol to transmit the point cloud data, it is beneficial to improve the transmission efficiency of the point cloud data and further improve the overall processing efficiency of the point cloud data.

[0058] Furthermore, in this embodiment, when adopting the original point data in the form of float16, since the x86 architecture on the vehicle does not have a native data conversion method and the data types supported by the GRPC protocol are not compatible with float16 either, therefore, in this embodiment, it is necessary to compress and convert each original point data to generate converted point data in a data form that the GRPC protocol can be compatible with and corresponding to each original point data.

[0059] Specifically, since the GRPC protocol can support data in the form of uint64 or uint32, where uint32 and uint64 are unsigned integer data types, representing 32-bit and 64-bit unsigned integers respectively. Therefore, the data type of the transmitted point data in this embodiment adopts uint64 or uint32 to achieve efficient transmission of the point cloud data while ensuring the integrity of the point cloud data.

[0060] It should be understood that the data types of the original point data and the transmission point data in this embodiment can also adopt other data types that can save bandwidth, and the selection of the transmission protocol can also be selected and adjusted according to the actual usage scenario. This is only an example here, but no strict restrictions are imposed on this.

[0061] Optionally, in this embodiment, it can be based on Figure 2 the method shown in to determine the transmission point data, which specifically includes the following steps.

[0062] In step S210, compress each coordinate data respectively to determine the corresponding compressed coordinate data.

[0063] In this embodiment, when the original point data adopts the float16 data type, the coordinate data corresponding to each original point data is also in the float16 format. Therefore, in this embodiment, each coordinate data in the float16 format will be compressed into the corresponding 16-bit integer data respectively, and the corresponding compressed coordinate data will be determined. At the same time, in this embodiment, the compressed 16-bit float integer data will be stored, which can ensure that the error does not exceed 1.5 centimeters and meet the subsequent display requirements. Thus, in this embodiment, by compressing the original point data to determine the corresponding compressed coordinate data, subsequent processing based on the compressed coordinate data can further save bandwidth and is beneficial to improving data processing and data transmission efficiency.

[0064] Optionally, in this embodiment, when compressing each coordinate data in the float16 format, first convert the float16 coordinate data into a binary format, and then compress the binary format float16 data according to a certain compression rule to generate the compressed coordinate data in the 16-bit integer form. Further, in this embodiment, the float16 data can be binary encoded according to the IEEE754 standard, expressed as an exponent and a mantissa; then convert the exponent part and the mantissa part into integers respectively, including converting the exponent part into an unsigned integer and the mantissa part into a signed integer; finally, merge the converted exponent and mantissa into a 16-bit integer.

[0065] It should be noted that in this embodiment, other compression rules can also be used to compress each coordinate data in the original point data to determine the corresponding compressed coordinate data. For example, the fixed-point number representation method can be used to convert floating-point numbers to integers, etc. The specific compression method can also be selected according to the actual situation. At the same time, it should be noted that compressing float16 data into 16-bit integer data may result in loss of data precision. Considering that float16 data uses a shorter number of binary digits to represent floating-point numbers, while 16-bit integers use a shorter number of digits to represent integers. Therefore, when performing compression, it is necessary to pay attention to the trade-off between the data precision requirements and the compression effect, so as to optimize the compression effect while ensuring data precision, which is beneficial to further saving bandwidth in data processing.

[0066] In step S220, bitwise concatenation is performed on each compressed coordinate data to determine the corresponding transmission point data.

[0067] In this embodiment, after determining the compressed coordinate data corresponding to each coordinate data in the original point data, the corresponding transmission point data is determined by performing bit-level concatenation (i.e., bitwise concatenation) on each compressed coordinate data. Among them, the bitwise concatenation of data is a general term, which refers to the operation of combining two or more bit sequences into one bit sequence according to certain rules. Thus, in this embodiment, the corresponding transmission point data in the form of a bit sequence is generated by performing bitwise concatenation on each compressed coordinate data, so that the transmission point data can accurately and completely represent the corresponding point cloud point, ensuring the accuracy in the process of point cloud data processing, and further improving the accuracy of point cloud data processing. And, by using the transmission point data in the form of a bit sequence, it is convenient for subsequent data transmission, which is beneficial to further improving the efficiency of data transmission.

[0068] Furthermore, after converting each coordinate data in the form of float16 data in the original point data into the corresponding compressed coordinate data in the form of 16-bit integer data, by performing bitwise concatenation on the compressed coordinate data, the corresponding transmission point data in the form of uint64 or uint32 data can be determined, so as to store the 16-bit data in the original point data in multiple 8-bit unsigned integers.

[0069] Optionally, when using the transmission point data in the form of uint64, the transmission point data corresponding to a point cloud point can be stored in a uint64. Since one bit in every 8 bits of the uint64 data in the GRPC protocol is occupied by a fixed control character or other identifier, only 7 bits of transmission point data can be stored in every 8 bits (that is, 7 bits of data can be accommodated in one byte). Thus, for each transmission point data including 3 coordinate data and corresponding to 48 bits of data, 7 bytes will be occupied. Compared with the 12 bytes required for each float data in the existing point cloud data processing method, the transmission point data in this embodiment can reduce the occupied bytes by 5, that is, compress the data to 7 / 12 of the original, which can reduce the bandwidth occupancy rate during the subsequent data transmission process and further save bandwidth.

[0070] Optionally, when using the transmission point data in the form of uint32, in an alternative implementation, the coordinate data on each coordinate axis in each point cloud point can be stored in a uint32 (which is also a variable-length encoding method). Then, one point requires 3 uint32s. And compared with the situation where each float data in the existing point cloud data processing method requires 12 bytes, considering possible data overflow, one transmission point data in this embodiment will occupy 6 - 9 bytes, that is, the data can be compressed to between 1 / 2 and 3 / 4 of the original. In another alternative implementation, two adjacent uint32s can be used to store two transmission point data. For example, (x 1 , x 2 ), (y 1 , y 2 ), (z 1 , z 2 ). At this time, compared with the situation where each float data in the existing point cloud data processing method requires 12 bytes and the situation where two float data require 24 bytes, two transmission point data in this embodiment will occupy 12 - 15 bytes, that is, the data can be compressed to between 1 / 2 and 5 / 8 of the original.

[0071] Thus, in this embodiment, by compressing each coordinate data in the form of float16 in the original point data respectively to determine the corresponding compressed coordinate data in 16-bit integer type, and then performing bit splicing on each 16-bit integer type of compressed coordinate data to determine the corresponding transmission point data in the form of uint64 or uint32, it is possible to realize the data compression conversion of the original point data, reduce the number of bytes occupied by the data, and save bandwidth for subsequent data transmission.

[0072] In step S130, the transmission point data is transmitted to generate a display screen based on each transmission point data.

[0073] In this embodiment, after compressing and converting each piece of original point data based on the foregoing method to determine the corresponding transmitted point data, each transmitted point is transmitted through a preset transmission protocol, so as to generate a display screen based on each piece of transmitted point data, and the environmental conditions around the autonomous vehicle are expressed through the display screen, facilitating the vehicle, the safety officer on the vehicle side, or the cloud to adjust the driving mode according to the environmental conditions and ensuring the safe and reliable operation of the autonomous vehicle.

[0074] Optionally, in this embodiment, the display screen is generated based on each piece of transmitted point data by the method as Figure 3 shown.

[0075] In step S310, each piece of transmitted point data after transmission is decoded and processed to generate corresponding display data.

[0076] In this embodiment, since the display cannot be directly performed through the transmitted point data, after receiving each piece of transmitted point data transmitted by the vehicle, the front-row PAD first decodes and processes each piece of data point data and generates corresponding visual display data. Among them, the encoding process is used to restore the transmitted point data to the corresponding original point data, that is, to deserialze the transmitted point data in the form of uint64 or uint32 data to generate the corresponding original point data. The processing process is used to convert the original point data into visual display data. Thus, by decoding and processing each piece of transmitted point data after transmission to generate corresponding display data, it is convenient for the visual expression of the point cloud data.

[0077] In step S320, the rendering data corresponding to the display screen is screened and determined from each piece of display data.

[0078] In this embodiment, to optimize the animation effect of the display screen and reduce screen jitter, the rendering data corresponding to the display screen is screened and determined from each piece of display data, so as to generate a continuously refreshed display screen based on the screened rendering data.

[0079] Optionally, in this embodiment, the display data corresponding to each point in the point cloud can be screened based on the point cloud generation time, the point cloud generation position, or other point cloud attributes, and the rendering data corresponding to the display screen is determined. Further, in this embodiment, each piece of display data is screened according to the point cloud generation time, and the display data corresponding to the points generated after a scan of the point cloud generator (such as, a lidar transmitter, etc.) is screened out. At this time, the refresh of the point cloud display occurs after the point cloud data of a complete circle of points is received, and a circle of points with the closest point cloud generation time is used as a frame of the display screen for rendering.

[0080] Specifically, when determining the display screen to be shown each time, the display time is filtered according to the time period corresponding to the start time and the end time when the point cloud generator scans one circle. For example, assume that the predetermined duration corresponding to one circle scanned by the point cloud generator is T, and the time when the first point cloud point (for example, the point cloud point corresponding to the position directly behind the vehicle) is generated among the scanned points in a certain circle is t 0 , then the points generated within the time period of t 0 +T are regarded as the points of the same circle, and the display data obtained by processing the original point data corresponding to the points of the same circle is used as the rendering data for the next display screen.

[0081] In step S330, a display screen is generated according to the rendering data.

[0082] In this embodiment, after determining the visual display data corresponding to the display screen, a display screen corresponding to the display data can be generated based on existing screen drawing methods (such as visualization tools like Matplotlib, Seaborn, Plotly, etc.). The method of generating a display screen according to the rendering data will not be elaborated here.

[0083] Furthermore, to improve the data transmission and screen display efficiency, at least one thread is used in this embodiment to generate a display screen based on each transmitted point data. For example, a decoding thread decodes and processes each transmitted point data after transmission to generate corresponding display data; and a rendering thread filters and determines the rendering data corresponding to the display screen from each display data, and generates a display screen according to the rendering data. Thus, in this embodiment, different threads can be used to process the transmitted point data respectively and render and generate a display screen according to the display data, which can further improve the data processing efficiency.

[0084] It should be noted that the data compression and conversion before data transmission in this embodiment can be completed based on the same process as data transmission, which is beneficial to ensuring data smoothness; or it can be completed using a process different from data transmission, which is beneficial to further improving data processing and data transmission efficiency.

[0085] The technical solution of the embodiment of the present invention obtains point cloud data including a plurality of original point data, performs compression and conversion on the original point data based on a preset transmission protocol to determine corresponding transmitted point data; and transmits the transmitted point data to generate a display screen based on each transmitted point data. Thus, in this embodiment, by performing compression and conversion on the original point data and then transmitting it, the bandwidth required for data transmission can be saved and point cloud display can be optimized.

[0086] Figure 4 is a flowchart of the point cloud data processing process of the embodiment of the present invention. As Figure 4As shown, in this embodiment, the point cloud data is processed through the following steps to generate a display screen.

[0087] In step S410, obtain the point cloud data.

[0088] In this embodiment, the point cloud data is the point cloud data within a preset height and / or a preset range, including a plurality of original point data. Each original point data is identified by coordinate data corresponding to the three directions of the x, y, and z axes in a three-dimensional rectangular coordinate system, and each coordinate data is represented and stored in the data form of float16. Thus, in this embodiment, unnecessary data can be reduced at the data source when obtaining the point cloud data, reducing the data occupancy bandwidth without affecting the data processing effect.

[0089] In step S420, compress each coordinate data in the float16 form in the original point data to determine the corresponding compressed coordinate data in the 16-bit integer form.

[0090] In step S430, perform bit splicing on each compressed coordinate data in the 16-bit integer form to determine the corresponding transmission point data in the uint64 data form.

[0091] In step S440, transmit each transmission point data based on the preset GRPC protocol.

[0092] In this embodiment, the transmission point data corresponding to each point in the point cloud data is transmitted to the front row PAD through the GRPC protocol.

[0093] In step S450, decode and process each transmitted point data to generate the corresponding display data.

[0094] In this embodiment, the front row PAD receives each transmitted point data transmitted by the vehicle and decodes and processes each transmitted point data to generate the corresponding display data.

[0095] In step S460, screen each display data according to the point cloud generation time to determine the rendering data corresponding to the display screen.

[0096] In this embodiment, according to the start time and end time when the point cloud generator scans a circle, the points corresponding to the point cloud generation time within the corresponding time period of the start time value and the end time are used as the points generated after the point cloud generator scans a circle, and the display data corresponding to this circle of points is determined as the rendering data corresponding to the display screen. This can enable the display screen of the point cloud to be refreshed each time the rendering data corresponding to a circle of points is completely received, making the display of the display screen more vivid.

[0097] In step S470, generate a display screen according to the rendering data.

[0098] The technical solution of this embodiment obtains a certain number of point cloud data, compresses and converts the coordinate data in the form of float16 in each original point data in the point cloud data to determine the corresponding transmission point data in the form of uint64 data, then transmits each transmission point data based on the GRPC protocol, and decodes and processes each transmitted transmission point data to generate the corresponding display data. The display data corresponding to the display screen is determined by screening according to the generation time of the point cloud, and finally the display screen is generated according to the rendering data. Thus, by obtaining point cloud data through the above method in this embodiment, unnecessary data can be reduced from the data source; the bandwidth occupancy rate in the data processing and transmission processes is greatly reduced by compressing and converting the data and using the preset transmission protocol, and the data transmission efficiency is improved; the display screen is also generated through the screened point cloud data, which can optimize the rendering of the display screen while realizing the expression of environmental information based on the point cloud data.

[0099] Figure 5 It is a schematic diagram of the point cloud data processing device according to an embodiment of the present invention. As Figure 5 shown, the point cloud data processing device in this embodiment includes an acquisition unit 1, a compression unit 2, and a transmission unit 3. Among them, the acquisition unit 1 is used to acquire point cloud data, and the point cloud data includes a plurality of original point data. The compression unit 2 is used to compress and convert the original point data based on a preset transmission protocol to determine the corresponding transmission point data. The transmission unit 3 is used to transmit the transmission point data to generate a display screen based on each transmission point data.

[0100] Optionally, the data type of the original point data in this embodiment is float16, the preset transmission protocol is the GRPC protocol, and the data type of the transmission point data is uint64 or uint32.

[0101] Optionally, when the acquisition unit 1 in this embodiment acquires point cloud data, it is further used to acquire point cloud data within a preset height and / or a preset range. Further, the original point data in this embodiment includes a plurality of coordinate data, and the compression unit 2 is further used to compress each coordinate data separately to determine the corresponding compressed coordinate data; and perform bit splicing on each compressed coordinate data to determine the corresponding transmission point data.

[0102] Further, when the data type of the original point data is float16, when the compression unit 2 determines the compressed coordinate data, it is specifically used to compress each coordinate data in the form of float16 into the corresponding 16-bit integer data to determine the corresponding compressed coordinate data; and splice the compressed coordinate data in the form of float16 to generate the corresponding transmission point data in the form of uint64 or uint32.

[0103] Optionally, the transmission unit 3 in this embodiment is further configured to generate a display screen based on the data of each transmission point using at least one thread. Further, when generating a display screen based on the data of each transmission point, the transmission unit 3 in this embodiment is specifically configured to decode and process the transmitted data of each transmission point to generate corresponding display data; screen and determine the rendering data corresponding to the display screen from the display data; and generate a display screen according to the rendering data. Specifically, the transmission unit 3 is further configured to screen the display data according to the point cloud generation time to determine the rendering data corresponding to the display screen.

[0104] Figure 6 is a schematic diagram of the electronic device according to an embodiment of the present invention. As Figure 6 shown, Figure 6 the electronic device shown is a general address query device, which includes a general computer hardware structure, and at least includes a processor 61 and a memory 62. The processor 61 and the memory 62 are connected through a bus 63. The memory 62 is adapted to store instructions or programs executable by the processor 61. The processor 61 may be an independent microprocessor or a set of one or more microprocessors. Thus, the processor 61 executes the instructions stored in the memory 62 to execute the method flow of the embodiment of the present invention as described above to implement the processing of data and the control of other devices. The bus 63 connects the above-mentioned multiple components together and at the same time connects the above-mentioned components to a display controller 64, a display device, and an input / output (I / O) device 65. The input / output (I / O) device 65 may be a mouse, a keyboard, a modem, a network interface, a touch input device, a body sensing input device, a printer, and other devices well known in the art. Typically, the input / output (I / O) device 65 is connected to the system through an input / output (I / O) controller 66.

[0105] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a device (equipment), or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt a computer program product implemented on one or more computer-readable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0106] The present application is described with reference to the flowcharts of the methods, devices (equipment), and computer program products according to the embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.

[0107] These computer program instructions can be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the function specified in the process Figure 1 or functions specified in one process or multiple processes.

[0108] These computer program instructions can also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the function specified in the process Figure 1 or functions specified in one process or multiple processes.

[0109] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, which is used for a computer to execute the above-mentioned partial or all method embodiments.

[0110] That is, those skilled in the art can understand that all or part of the steps in implementing the above-mentioned method embodiments can be completed by specifying relevant hardware through a program. The program is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0111] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A method for processing point cloud data, characterized in that, the method comprises: obtaining point cloud data, which includes a plurality of original point data; performing compression conversion on the original point data based on a preset transmission protocol to determine corresponding transmitted point data; transmitting the transmitted point data to generate a display screen based on each of the transmitted point data.

2. The method according to claim 1, characterized in that, the data type of the original point data is float16.

3. The method according to claim 2, characterized in that, the preset transmission protocol is the GRPC protocol, and the data type of the transmitted point data is uint64 or uint32.

4. The method according to claim 1, characterized in that, the original point data includes a plurality of coordinate data, and the performing compression conversion on the original point data based on a preset transmission protocol to determine corresponding transmitted point data includes: compressing each of the coordinate data respectively to determine corresponding compressed coordinate data; performing bit splicing on each of the compressed coordinate data to determine corresponding transmitted point data.

5. The method according to claim 4, characterized in that, the data type of the original point data is float16, and the compressing each of the coordinate data respectively to determine corresponding compressed coordinate data includes: compressing each of the coordinate data in float16 form into corresponding 16-bit integer data respectively to determine corresponding compressed coordinate data.

6. The method according to claim 1, characterized in that, the generating a display screen based on each of the transmitted point data includes: using at least one thread to generate a display screen based on each of the transmitted point data.

7. The method according to claim 1, characterized in that, the generating a display screen based on each of the transmitted point data includes: performing decoding and processing on each of the transmitted point data after transmission to generate corresponding display data; screening and determining rendering data corresponding to the display screen from each of the display data; generating a display screen according to the rendering data.

8. The method according to claim 7, characterized in that, the screening and determining rendering data corresponding to the display screen from each of the display data includes: screening each of the display data according to the generation time of the point cloud to determine the rendering data corresponding to the display screen.

9. The method according to claim 1, characterized in that, the obtaining point cloud data includes: obtaining point cloud data within a preset height and / or a preset range.

10. A point cloud data processing device, characterized in that, the device comprises: an obtaining unit for obtaining point cloud data, which includes a plurality of original point data; a compression unit for performing compression conversion on the original point data based on a preset transmission protocol to determine corresponding transmitted point data; a transmission unit for transmitting the transmitted point data to generate a display screen based on each of the transmitted point data.

11. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1-9 are implemented.

Images