Point cloud data thinning method and device, electronic equipment and storage medium

Abstract

The invention discloses a point cloud data thinning method and device and a storage medium, and the method comprises the steps: obtaining point cloud data; carrying out interval sampling on the point cloud data, and extracting a plurality of thinning materials from the point cloud data; generating a point cloud fingerprint according to the plurality of diluted materials; based on a received page loading request and the point cloud fingerprint, determining a page supply material from the plurality of diluted materials; and processing the page supply material to obtain a complete thinning point cloud material. According to the page loading method and device, the better page supply material can be obtained through interval sampling and thinning, then the corresponding loading page is generated according to the page supply time to be output and displayed, and the page loading efficiency and the page loading display effect can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a method, device, electronic device and storage medium for thinning point cloud data. Background Art

[0002] Due to some inherent mechanical motion characteristics, the mainstream scanning characteristics of autonomous driving perception components are reciprocating or cyclic laser scanning, which affects the display and rendering of the perceived content. When displaying, the point cloud is too heavy, resulting in serious page loading jams (slow loading or failure to load). For example, the display frequency is only about 1HZ (only one picture can be displayed per second). In addition, when the display page is loading, other core indicators such as CPU occupancy are also very high, requiring large-scale overclocking processing. Excessive CPU overclocking processing will affect CPU performance, or a CPU with better performance will be required to meet the overclocking processing requirements, which is costly. Summary of the invention

[0003] The present application aims to at least solve the technical problems existing in the prior art that the data point cloud used for page loading is heavy, resulting in serious page loading jams and poor page loading display effects.

[0004] In order to solve the above technical problems, the present application provides a method for thinning point cloud data, including:

[0005] Get point cloud data;

[0006] Performing interval sampling on the point cloud data to extract a plurality of thinned materials from the point cloud data;

[0007] Generating a point cloud fingerprint according to the plurality of the thinned materials;

[0008] Determine a page supply material from a plurality of the thinned materials based on the received page load request and the point cloud fingerprint;

[0009] The page supplied material is processed to obtain complete thinned point cloud material.

[0010] In some embodiments, performing interval sampling on the point cloud data and extracting a plurality of sparse materials from the point cloud data comprises:

[0011] Sampling the point cloud data a preset number of times at preset time intervals; or

[0012] The point cloud data is rotated and sampled at intervals according to a preset interval sampling angle.

[0013] In some embodiments, generating a point cloud fingerprint according to a plurality of the thinned materials includes:

[0014] Grouping the plurality of thinned materials to obtain a grouping result;

[0015] A point cloud fingerprint is generated according to the grouping result.

[0016] In some embodiments, based on the received page load request and the point cloud fingerprint, determining the page supply material from the plurality of the thinned materials comprises:

[0017] After receiving the page loading request, extracting corresponding thinned materials from the plurality of thinned materials as page supply materials according to the point cloud fingerprint;

[0018] The page supplied material is processed to obtain complete sparse point cloud material, including:

[0019] Combining the page supply materials into complete thinned point cloud materials according to the page supply time;

[0020] A corresponding loading page is generated according to the thinned point cloud material.

[0021] In some embodiments, after generating a corresponding loading page according to the thinned point cloud material, the method further includes:

[0022] Determine whether business waiting is required;

[0023] If so, the generated loading page is displayed through lossy loading.

[0024] In some embodiments, after receiving the page loading request, the method further includes:

[0025] Acquire the end page code package according to the page loading request;

[0026] Loading the end page according to the end page code package;

[0027] The end pages are layered and layer accumulated;

[0028] A corresponding loading page is generated according to the layer accumulation result and the page supply material.

[0029] In some embodiments, obtaining point cloud data includes:

[0030] The scanning device installed on the vehicle side scans the entire point cloud data reciprocally or cyclically.

[0031] The present application also provides a device for thinning point cloud data, including:

[0032] An acquisition module configured to acquire point cloud data;

[0033] A sampling module, configured to perform interval sampling on the point cloud data and extract a plurality of thinned materials from the point cloud data;

[0034] A generating module, configured to generate a point cloud fingerprint according to a plurality of the thinned materials;

[0035] A determination module, configured to determine a page supply material from a plurality of the thinned materials based on the received page loading request and the point cloud fingerprint;

[0036] The processing module is configured to process the page supplied material to obtain a complete thinned point cloud material.

[0037] An embodiment of the present application also provides a vehicle, comprising the above-mentioned point cloud data thinning device.

[0038] An embodiment of the present application also provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program on the memory.

[0039] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0040] The point cloud data thinning method, device, vehicle, electronic device and storage medium provided in the embodiments of the present application obtain point cloud data, perform interval sampling on the point cloud data, and extract multiple thinned materials from the point cloud data; generate a point cloud fingerprint based on the multiple thinned materials; determine a page supply material from the multiple thinned materials based on the received page loading request and the point cloud fingerprint; process the page supply material to obtain a complete thinned point cloud material, and obtain better page supply material through interval sampling thinning, and then generate a corresponding loading page output display, which can effectively improve the page loading efficiency and page loading display effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0042] Figure 1 A flowchart of a method for thinning point cloud data according to an embodiment of the present application;

[0043] Figure 2 Another flow chart of the method for thinning point cloud data according to an embodiment of the present application;

[0044] Figure 3 A schematic diagram of a thinned material after one thinning sampling according to an embodiment of the present application;

[0045] Figure 4 A schematic diagram of a complete thinned point cloud material according to an embodiment of the present application;

[0046] Figure 5 A schematic diagram of a complete thinned point cloud material and page display in an embodiment of the present application;

[0047] Figure 6 Schematic diagram of the structure of the point cloud data thinning device according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0049] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0050] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0051] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0052] It should also be understood that, although the present application has been described with reference to some specific examples, those skilled in the art will be able to realize many other equivalent forms of the present application that have the features described in the claims and are therefore within the scope of protection defined thereby.

[0053] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0054] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0055] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0056] Figure 1 and Figure 2 FIG. 1 is a flow chart showing a method for thinning point cloud data provided by an embodiment of the present application. Figure 1 and Figure 2 As shown, the first embodiment of the present application provides a method for thinning point cloud data, comprising:

[0057] S101: Acquire point cloud data.

[0058] Point cloud data is a collection of point data on the surface of environmental information such as roads, vehicles, pedestrians, obstacles, etc., sensed by the automatic sensing element installed on the vehicle side. The automatic sensing element includes sensors such as laser radar, vehicle-mounted camera, millimeter wave radar and ultrasonic sensor. After the automatic sensing element senses the point cloud data, the point cloud data can be sent to a cloud server connected to the vehicle side and / or to a vehicle-side server installed on the vehicle side, so that the cloud server and / or the vehicle-side server can process the acquired point cloud data.

[0059] With the development of the Internet of Vehicles, driving-related data generated during the driving process of the vehicle (including the environmental information data sensed by the above-mentioned automatic sensing components) will be transmitted to a cloud server that is connected to the vehicle. As part of the Internet of Vehicles cloud platform (referred to as the cloud platform), the cloud server can be an independent server, a server cluster composed of multiple servers, or a cloud computing service center. When the cloud server is a server cluster composed of multiple servers or a cloud computing service center, different servers can realize different functions.

[0060] For example, Figure 2As shown, in this embodiment, the thinning method of point cloud data can be applied to the Internet of Vehicles system composed of a vehicle-side server and a cloud server, the cloud server includes a thinning server, a management server and a user server, wherein the vehicle-side server is used to process relevant data of the vehicle-side, for example, it can serve as an information transfer to send the point cloud data collected by the vehicle-side acquisition device to the cloud server, and the vehicle-side server can initiate an access request to the cloud server to realize the interaction between the vehicle-side server and the cloud server; the thinning server is specifically used to perform thinning processing on the acquired point cloud data; the management server is used to manage the data of the Internet of Vehicles cloud platform, for example, receiving and storing the vehicle driving data sent by the vehicle-side, and configuring the corresponding page generation code for the corresponding vehicle-side according to the thinning result of the thinning server; the user server is used to generate the corresponding page according to the corresponding page generation code and the point cloud data after thinning processing, and output the page through the cloud display and / or the vehicle-side display.

[0061] Among them, the cloud display is the display of the Internet of Vehicles cloud platform, so that users such as managers of the Internet of Vehicles cloud platform can view the driving-related data of each vehicle connected to the Internet of Vehicles cloud platform and remotely control the vehicle, etc. The vehicle-side display can be a vehicle-mounted display such as a central control screen, a display screen of a vehicle-mounted multimedia device or a windshield, or a display of a mobile device such as a mobile phone, tablet computer, smart watch, etc. in the car, so that users in the car can view driving-related data and control the driving of the vehicle, etc. For example, it is convenient for users in the car to check the road conditions in time through the vehicle-side display, and automatically or manually control the vehicle to slow down when the vehicle is driving on a congested road section.

[0062] In some other embodiments, the method for thinning point cloud data can also be applied to a vehicle networking system including other servers, which is not specifically limited in this application.

[0063] The Internet of Vehicles cloud platform can be a cloud platform for automobile companies (for example, providing remote control data for vehicles connected to the Internet of Vehicles platform based on environmental information data to achieve remote automatic control of vehicles), or it can be a national cloud platform (for example, providing navigation information data for vehicles connected to the Internet of Vehicles platform based on environmental information data).

[0064] Optionally, in step S101, obtaining point cloud data includes:

[0065] S1011: Scan the entire point cloud data reciprocally or cyclically through a scanning device disposed on the vehicle end.

[0066] The scanning device may be at least one of a laser scanner, a millimeter wave scanner, an ultrasonic scanner and a camera scanner, so as to sense and acquire point cloud data on the road when the vehicle is traveling.

[0067] The scanning device can obtain full point cloud data based on regular mechanical motion scanning, and the full point cloud data can be sent to the management server of the cloud platform as point cloud material. The management server can also simultaneously receive information such as the driving trajectory of the vehicle when the scanning device is working, the scanning angle and scanning time of the scanning device. The point cloud results of the laser scanner are of high quality and low noise, and the detailed scanning is clearer, which can meet the needs of high-precision scanning projects. Millimeter waves have strong penetrating ability, millimeter wave scanners have high resolution, fast imaging speed, and more accurate point cloud recognition. Ultrasonic scanners have high scanning accuracy and low cost. The camera scanner has a very fast scanning speed, and more than one million points can be obtained in a few seconds. Each time a surface is scanned, the measurement points obtained are relatively uniform, the point cloud data obtained is accurate, and the single scanning range is wide. In specific implementation, the corresponding scanning device can be selected according to actual needs.

[0068] Exemplarily, in this embodiment, point cloud data is acquired through the reciprocating or cyclic mechanical motion of the laser scanner.

[0069] The full point cloud data is all the environmental point cloud data within the current field of view of the vehicle. In this embodiment, by collecting the full point cloud data of the vehicle, more comprehensive data can be obtained to avoid data omissions. The data can be any disorderly distributed data.

[0070] Point cloud materials are materials used to generate pages. The user server can generate corresponding pages based on the point cloud materials obtained based on the full point cloud data, and then load and display the pages on the cloud display and / or the vehicle-side display.

[0071] The point cloud data collected by the scanning device may also be local point cloud data including the target object (eg, an obstacle in front of the vehicle).

[0072] Point cloud data contains information such as the three-dimensional coordinates (XYZ), laser reflection intensity, color (RGB) and grayscale of the points. The denser the point cloud, the more image details and information it reflects. However, when the amount of point cloud data is too large and the point cloud density (data density) is too large, some of the data will have little effect on subsequent page generation and loading, affecting the efficiency and effect of page generation and loading. Therefore, it is necessary to thin out some of the point cloud data to minimize the number of data points to ensure smooth page loading, and try to retain feature points to ensure complete loading and display of the page.

[0073] In some embodiments, after obtaining the point cloud data in step S101, the method further includes:

[0074] S201: Determine the collection range of the point cloud data according to the driving trajectory of the vehicle;

[0075] S202: analyzing the point cloud data within the acquisition range according to the environmental information of the location of the vehicle to determine whether thinning processing is required;

[0076] S203: If thinning processing is required, determine thinning parameters of the point cloud data, wherein the thinning parameters include at least one of a thinning position, a thinning amount, and a thinning rate.

[0077] After the vehicle collects full or partial point cloud data through the scanning device during driving, the management server can further determine the processing range of the point cloud data according to the vehicle's driving trajectory, and then analyze and process the point cloud data collected at different locations according to the environmental information of the different locations of the vehicle to determine whether thinning processing is needed. For example, when it is determined according to the vehicle's driving trajectory that the vehicle is driving on a suburban road, the suburban road can be determined as the first collection range, and the point cloud data collected within the first collection range is relatively sparse, and thinning processing may not be required; and when it is determined according to the vehicle's driving trajectory that the vehicle is driving on an urban road, the suburban road can be determined as the second collection range, and the point cloud data collected within the second collection range is dense, and thinning processing may be required, and the location and amount of thinning can be further determined based on the location information of the point cloud data and the data volume or data density of the point cloud data. For example, when it is determined that the point cloud data is data within the second acquisition range (urban roads), it is determined whether the data volume or data density within the range exceeds the preset threshold. If it exceeds, the three-dimensional coordinate position or range corresponding to the data exceeding the preset threshold is determined as the thinning position, and the thinning amount or thinning rate is determined according to the specific size of the data volume or data density. For example, when the data density of a certain position range within the first acquisition range exceeds the normal data density of the position range by one times, the thinning amount can be determined to be half of the point cloud data volume within the position range.

[0078] From the above, it can be seen that in this implementation, the acquired point cloud data can be pre-processed according to the driving information such as the vehicle's driving trajectory, and the point cloud data that may need to be thinned can be determined in a targeted manner, and then the thinning parameters such as the thinning position and thinning amount of the point cloud data can be determined, so as to avoid thinning judgments on all the acquired point cloud data (for example, it is necessary to traverse all the point cloud data to calculate the data density of each position), which has problems such as low processing efficiency and subsequent page loading still has problems such as jamming. At the same time, it also avoids unified thinning processing of point cloud data (the thinning processing methods of point cloud data in different acquisition ranges are different), which leads to the loss of important point cloud data, and then leads to problems such as incomplete information or page distortion of the loaded page.

[0079] The collection range of the above-mentioned different point cloud data is determined according to the type of road the vehicle is on. In specific implementation, the collection range of point cloud data can also be determined according to information such as intersections and traffic lights in environmental information. For example, when it is determined based on the vehicle's driving trajectory that the vehicle is close to a certain intersection, it is initially determined that the point cloud data within the range between the intersection and the vehicle may need to be thinned out, and then the thinning position and thinning amount (or thinning rate) that need to be thinned out are determined based on the location information and data density (or data volume) of the point cloud data within the range.

[0080] In some embodiments, in step S1011, the method further includes:

[0081] S301: Determine scanning parameters of the scanning device;

[0082] S302: Determine thinning processing parameters of the point cloud data according to the scanning parameters.

[0083] In this step, scanning parameters such as the scanning angle, scanning distance, scanning time and scanning accuracy of the scanning device can be obtained, and then the point cloud data can be analyzed and processed according to the scanning parameters. For example, the point cloud data corresponding to different times (such as morning or evening) can be thinned according to different scanning times. When the point cloud data is needed to identify obstacles, the obstacle features required at night need to be more accurate. Therefore, the standard thinned data density of the point cloud data collected at night is greater than the standard thinned data density of the point cloud data collected during the day, that is, the point cloud data collected at night is thinned only when its data density is greater than a larger value.

[0084] The scanning angle (scanning field of view) of the scanning device can be in the range of 360° horizontally and 360° vertically. When the vehicle is driving on the road, the sampling range can include 360° horizontally and 270° vertically to obtain point cloud data in the horizontal direction around the vehicle, in the vertical direction in front of the vehicle, and below the horizontal direction of the vehicle; when the vehicle is driving in a garage, point cloud data within the full angle range of 360° horizontally and 360° vertically can be collected.

[0085] The point cloud data collected according to the scanning distance is usually dense when close and sparse when far. That is, the closer the object is to the scanning device, the more points are collected on its surface and the denser the corresponding point cloud. Conversely, the farther the object is from the scanning device, the sparser the corresponding point cloud.

[0086] S102: performing interval sampling on the point cloud data, and extracting a plurality of thinned materials from the point cloud data.

[0087] After the thinning server acquires the point cloud data, it can perform small-interval sampling on the acquired point cloud data through a reciprocating scanning / interval scanning scheme to extract multiple thinned materials from the point cloud data.

[0088] The point cloud data acquired by the thinning server is disordered and messy data with different sparsity levels. Therefore, it is necessary to sample the acquired point cloud data to obtain thinning materials for thinning.

[0089] In this step, interval sampling is used, and the number of samples taken is small, which can effectively reduce the processing volume of point cloud data; however, the sample range remains unchanged, and sampling can be performed in the entire range to obtain more comprehensive and accurate thinning materials.

[0090] In some embodiments, in step S102, the point cloud data is sampled at intervals to extract a plurality of thinned materials from the point cloud data, including:

[0091] Sampling the point cloud data a preset number of times at preset time intervals; or

[0092] The point cloud data is rotated and sampled at intervals according to a preset interval sampling angle.

[0093] Among them, sampling the point cloud data a preset number of times at preset intervals means sampling with a certain time interval as the sampling standard. For example, when sampling 1 / 10, sampling is performed once every 10 times, that is, 10 point cloud interval samples are generated at a certain interval for extraction, and the 10 point cloud interval samples are 10 thinned materials.

[0094] The acquisition time of the original point cloud data is not fixed. Therefore, in this embodiment, the sampling time is standardized to equal intervals, and the original high-frequency (for example, 1s) acquired point cloud data is adjusted to a low frequency to perform thinning sampling of the point cloud data. For example, the acquired point cloud data is sampled every 30s, and 10 point cloud interval samples within these 30s are extracted every 30s as thinning materials.

[0095] The thinned materials extracted in the same time period mentioned above should be equivalently spliced, where equivalent splicing means that the reference standards of each point in the point cloud data are the same or there is a certain transformation relationship. For example, in the first thinned material, the coordinates of the first point are the three-dimensional coordinates in the first three-dimensional coordinate system. In the second thinned material, when the first point appears, the coordinates of the first point are still the three-dimensional coordinates in the first three-dimensional coordinate system, rather than the three-dimensional coordinates obtained in other coordinate systems; or there is a certain transformation relationship between the coordinate systems of the two thinned materials, so that they can be effectively spliced ​​later. Rotational interval sampling is to sample based on the rotation angle as the sampling standard. In specific implementation, the three-dimensional point cloud can be first two-dimensionally mapped to different two-dimensional planes, and then rotation sampling can be performed in the two-dimensional plane. For example, the three-dimensional point cloud can be mapped to the XOY plane (horizontal plane), and then rotated horizontally at intervals of a preset angle (for example, 30°), and rotational interval sampling can be performed within the horizontal range of 360°.

[0096] In a specific implementation, the mapping plane can be determined according to the target object to be sampled. For example, when the target object is an obstacle on the road, rotational interval sampling can be performed in the horizontal XOY plane. When the target object is a traffic light, the traffic light is generally installed at a certain height, and rotational interval sampling can be performed in the vertical XOZ plane or YOZ plane.

[0097] Furthermore, when performing rotational interval sampling, the sampling interval angle can be adjusted according to the angle adjustment parameters to improve the accuracy of rotational interval sampling. For example, when the rotation angle of part of the sampling range cannot be equally spaced, the edge angle range can be eliminated and the other angle ranges can be equally spaced. For another example, the sampling interval angles of different planes can be adjusted according to the angle adjustment parameters corresponding to different planes. The sampling interval angle of the XOY plane can be fine-tuned within the range of ±5° based on the determined initial interval sampling angle, and the sampling interval angle of the YOZ plane can be fine-tuned within the range of ±3° based on the determined initial interval sampling angle.

[0098] S103: Generate a point cloud fingerprint according to the plurality of thinned materials.

[0099] After thinning the point cloud data through step S102 to obtain multiple thinned materials (for example, the 10 thinned materials obtained by sampling once every 10 times as mentioned above), relevant point cloud fingerprints can be generated based on the multiple thinned materials. The point cloud fingerprints can be point cloud fingerprints of target objects such as pedestrians, other vehicles, lane lines, traffic signs, etc. on the road where the vehicle is traveling.

[0100] Point cloud fingerprints are used to describe the texture of the road on which the vehicle is traveling and the characteristics of the target objects on the road. For example, point cloud fingerprints can describe the slope, curvature, degree of winding (straight road or turning road), lane lines (lane lines are darker than other parts of the road surface), whether there are obstacles (existence of heterogeneous shapes), etc. of the road surface through the shape and color of the texture.

[0101] In some embodiments, in step S103, generating a point cloud fingerprint according to the plurality of thinned materials includes:

[0102] S1031: Grouping the plurality of thinned materials to obtain a grouping result;

[0103] S1032: Generate a point cloud fingerprint according to the grouping result.

[0104] Taking the above-mentioned 10 times of sampling and interval sampling to obtain thinned materials as an example, in this step, the 10 thinned materials extracted in the same time period can be grouped, and then the 10 thinned materials can be processed according to the grouping results to generate point cloud fingerprints.

[0105] Specifically, the thinning server can group the thinned materials according to preset classification categories. For example, the multiple thinned materials obtained by sampling can be grouped according to the categories of the detected target objects. Exemplarily, the thinned materials containing lane lines are divided into one group, and the thinned materials containing obstacles (which can be further subdivided into fixed obstacles and mobile obstacles such as pedestrians and vehicles) are divided into another group. Then, according to the features in each group of thinned materials, point cloud fingerprints of lane lines within the acquisition environment are generated (the width of the lane in which the vehicle is traveling, whether there is a line-pressing driving situation, etc. can be determined based on the point cloud fingerprints of the lane lines), as well as point cloud fingerprints of obstacles (the point cloud fingerprints of the obstacles can be used to determine whether the size of the obstacles will affect the driving of the vehicle, etc.).

[0106] Specifically, the thinning server can generate the point cloud fingerprint of the target object according to the geometric center coordinates, length, width, height and other geometric dimensions, color, reflection intensity, relative speed and other features of the point cloud. It is understandable that the same thinning material can include both lane lines and obstacles, and the thinning material is divided into a group including lane lines and a group including obstacles.

[0107] The method of extracting the thinned material in the next time period and generating the point cloud fingerprint is similar to the above method and will not be repeated here.

[0108] S104: Based on the received page loading request and the point cloud fingerprint, determine the page supply material from the plurality of the thinned materials.

[0109] Step S104 specifically includes:

[0110] After receiving the page loading request, corresponding thinned materials are extracted from the plurality of thinned materials according to the point cloud fingerprint as page supply materials.

[0111] The vehicle-side server can initiate access to the cloud platform. The user server can determine the page supply material from multiple thin materials sampled in the same time period based on the page loading request sent by the vehicle-side access and the generated point cloud fingerprint, thereby extracting the best single-sampling thin material. The single-sampling thin material is as follows: Figure 3 shown.

[0112] Since the above-mentioned point cloud fingerprint is a fingerprint containing the most comprehensive and accurate features of the target object generated by using multiple thinned materials extracted in the same time period, in order to improve the efficiency of page loading, in this step, the thinned material that best matches the above-mentioned point cloud fingerprint can be determined from multiple thinned materials extracted in the same time period based on the point cloud fingerprint, so as to avoid the point cloud density in the thinned material being too large, which still affects the page loading, or the point cloud density being too small, containing fewer features of the target object, and subsequently being unable to generate an image of the target object. In this embodiment, the thinned material whose point cloud density best matches the point cloud density in the point cloud fingerprint is extracted from multiple thinned materials and determined as the page supply material.

[0113] For example, the point cloud density of the vehicle in front of the vehicle in the point cloud fingerprint generated by the 6 grouped thinned materials is 100 points / m 2 (1m 2 In this case, the thinned material that matches the point cloud density of the point cloud fingerprint is determined from the 6 thinned materials that have been grouped and contain the front vehicle. For example, the point cloud density of the first thinned material is 50 / m 2 The point cloud density of the second thinned material is 90 points / m 2 The point cloud density of the third thinning material is 130 / m 2 At this time, it can be determined that the second thinned material is the page supply material. The point cloud density of the third thinned material is relatively large, and it may contain many data points that are irrelevant to the characteristics of the preceding vehicle, so it is removed.

[0114] The page supply material can be used directly to generate the page, or the page can be generated after thinning out the part of the page supply material where the point cloud is still heavy (for example, the point cloud density in complex road areas is still large).

[0115] It is understandable that the above page loading request can be initiated by the vehicle-side server to the cloud platform after starting the vehicle-side display, or it can be initiated to the cloud platform at any time during the driving process of the vehicle. When the vehicle-side server initiates a page loading request to the cloud platform after starting the vehicle-side display, the thinning server can execute steps S102 and S103 according to the point cloud data acquired in real time, and then execute step S104 to extract the corresponding thinned material from the multiple thinned materials sampled according to the point cloud fingerprint as the page supply material. The above page loading request can also be initiated by the vehicle-side server after the communication connection with the cloud platform is established, based on the access information of the received cloud platform.

[0116] S105: Process the page supplied material to obtain complete thinned point cloud material.

[0117] The above steps obtain the page supply materials of a certain time or a certain period of time. Therefore, in this step, the page supply materials of different times can be integrated after interval sampling and thinning, so as to obtain the complete thinned point cloud materials, so as to quickly generate the page. When sampling according to the rotation angle, since the time corresponding to each sampling rotation angle is different, the page supply materials are materials of different times.

[0118] In some embodiments, step S105 specifically includes:

[0119] S1051: combining the page supply materials into complete thinned point cloud materials according to the page supply time;

[0120] S1052: Generate a corresponding loading page according to the thinned point cloud material.

[0121] Among them, page supply time refers to the loading time of different pages. The user server of the cloud platform can extract the thinned material with the point cloud density that best matches the point cloud fingerprint from multiple thinned materials sampled in the same time period based on the point cloud fingerprint, taking into account road features and traffic signs, as the page supply material. The time point corresponding to the page supply material is a page supply time. Then, the page supply materials at different times are spliced ​​or merged in sequence according to the time sequence (the previous thinned material and the next thinned material can have a certain overlap to ensure the continuity of the spliced ​​data), and the complete thinned point cloud material determined according to the time flow is obtained (such as Figure 4 and Figure 5 A in the figure), and then generates the corresponding loading page output display (as shown in Figure 5 As shown in B), the loading page is a page flow, which can realize the sequential and smooth loading of different pages on the display. For example, the time corresponding to the first page supply material obtained from the first interval sampling is the first time, the time corresponding to the second page supply material obtained from the second interval sampling is the second time, and the time corresponding to the third page supply material obtained from the third interval sampling is the third time. The first time is earlier than the second time, and the second time is earlier than the third time. At this time, the first page supply material, the second page supply material, and the third page supply material can be sequentially spliced ​​together in time order to generate the first page. Each page supply material can be used as a frame of image.

[0122] It is understandable that the above-mentioned page supply material can also be a more complete thinned material regenerated based on multiple thinned materials, and this application does not specifically limit this.

[0123] In a specific implementation, the page supply materials corresponding to different grouping objects can also be merged according to the page supply time. For example, when the page supply materials at a certain time include the supply materials of the road surface and the supply materials of the obstacles, the two can be merged. In this embodiment, when thinning the point cloud data, thinning can be performed according to different target objects to effectively avoid the lack of features of some target objects caused by concentrated thinning. For example, when thinning the point cloud data of obstacles and the roads on which they are located, it can avoid excessive thinning of the point cloud data of obstacles or roads, and the thinning is more balanced.

[0124] The method for thinning point cloud data provided in the embodiment of the present application obtains point cloud data, performs interval sampling on the point cloud data, and extracts multiple thinned materials from the point cloud data; generates a point cloud fingerprint based on the multiple thinned materials; determines a page supply material from the multiple thinned materials based on the received page loading request and the point cloud fingerprint; processes the page supply material to obtain a complete thinned point cloud material, and can obtain better page supply material through interval sampling thinning, and then generate a corresponding loading page output display, which can effectively improve the page loading efficiency and the page loading display effect.

[0125] Specifically, by thinning out excessive point cloud data, the amount of data is reduced while retaining more information (such as road feature information), and redundant information is processed (reducing interference in point cloud data). Massive data is processed into reliable lightweight data, thereby improving the processing efficiency of point cloud data, and then improving page loading efficiency. For example, the efficiency of the loading process can be improved by more than ten times (from 3Hz to 50+Hz, refreshing 50 frames of images per second). At the same time, it can also reduce CPU usage and improve the operating performance of the cloud platform.

[0126] In this embodiment, after performing interval sampling on the point cloud data and using the sampled thinned materials to generate point cloud fingerprints, the preferred thinned materials determined according to the point cloud fingerprints are used as page supply materials, which can improve the page loading display effect. In addition, the above method can better support the system display of the cloud platform and help solve the defects of algorithm training and manual positioning algorithms (manual marking of road signs, etc.).

[0127] It should be noted that, in this embodiment, by setting up a dedicated sparse server to perform the above steps S102 to S104, setting up a user server for page generation and loading, the occupancy efficiency of each cloud server can be reduced, and the processing performance of the cloud platform can be improved. The CPU occupancy efficiency of the cloud server can be Figure 2 shown.

[0128] In some embodiments, after step S1043, generating a corresponding loading page according to the thinned point cloud material, the method further includes:

[0129] S401: Determine whether service waiting is required;

[0130] S402: If yes, the generated loading page is displayed through lossy loading.

[0131] Among them, business waiting refers to whether it is necessary to load and display the page immediately. In this embodiment, the generated loading page can be loaded and displayed according to the business needs of the cloud platform. If the business cannot wait, the cloud platform can make some trade-offs without damaging the main process and core functions. For example, while ensuring the smooth loading of the page, the operating performance of other components can be improved.

[0132] When the business can wait long enough or has sufficient bandwidth and performance, for example, the loaded page does not need to be displayed in real time, and can be delayed by 0.5-1s compared to other real-time displayed pages. The cloud platform can have stronger computing power under the condition of a large number of point cloud files, and the rendering and loading parts can also be improved, making it unnecessary to optimize loading waits and freezes.

[0133] In some embodiments, Figure 2 As shown, after receiving the page loading request, the method further includes:

[0134] S501: Acquire a terminal page code package according to the page loading request;

[0135] S502: Loading the end page according to the end page code package;

[0136] S503: Layering the end pages and performing layer accumulation;

[0137] S504: Generate a corresponding loading page according to the layer accumulation result and the page supply material.

[0138] The vehicle-side server initiates an access request to the cloud platform, which may include a page loading request. The management server of the cloud platform can search and obtain the corresponding end page code package according to the received page loading request, and configure different page loading and display modes for different vehicle models. For example, the corresponding end page code package can be searched and obtained according to the display size of the vehicle-side display. After the management server finds the corresponding end page code package, it sends the end page code package to the user server of the cloud platform. The user server loads and displays the corresponding end page according to the end page code package. The end page is the page that matches the size of the vehicle-side display, and the end page is layered, and then the layer accumulation is performed. After that, the layer accumulation result is sent to the thinning server. The thinning server configures the corresponding page supply material for each layered page according to the layer accumulation result and the page supply material determined according to the generated point cloud fingerprint, and then sends the page supply material of the configured page to the user server. The user server combines the different page supply materials containing the layered page according to the page supply time to generate a complete thinned point cloud material, and then generates the corresponding loading page. In this step, the corresponding page background can be configured for the page supply material through page layering to improve the page rendering performance.

[0139] For example, in this embodiment, each end page can be divided into a background layer, a content layer, a global control layer, and a temporary layer, and then the layers are accumulated to determine the layer accumulation result. For example, the bottom layer (the fourth layer) is the background layer, the second to last layer (the third layer) is the temporary layer, the second layer is the content layer, and the top layer (the first layer) is the global control layer. At this time, the material containing the vehicle in the page supply material can be configured to the content layer, and the material containing the environmental information can be configured to the background layer. For example, the vehicle of the first page supply material and the vehicle of the second page supply material are spliced ​​in the content layer, and the environmental information of the first page supply material and the environmental information of the second page supply material are spliced ​​in the background layer, thereby generating a loading page.

[0140] Exemplarily, after adopting the above-mentioned page rendering and display scheme (including steps S401 to S402 and / or steps S501 to S503), the rendering of the page is extremely smooth in the key cloud loading scenario, and lossy services can be achieved to greatly improve the user experience. For example, in the point cloud scenario, the point cloud stream can be played at 60Hz, and the loading and playback speeds are extremely smooth.

[0141] Figure 6 A schematic diagram of a device for thinning point cloud data in an embodiment of the present application is shown. Figure 6 As shown, based on the above-mentioned method for thinning point cloud data, the embodiment of the present application further provides a device for thinning point cloud data, including:

[0142] An acquisition module 10, configured to acquire point cloud data;

[0143] A sampling module 20 is configured to perform interval sampling on the point cloud data and extract a plurality of thinned materials from the point cloud data;

[0144] A generating module 30, configured to generate a point cloud fingerprint according to a plurality of the thinned materials;

[0145] A determination module 40 is configured to determine a page supply material from a plurality of the thinned materials based on the received page loading request and the point cloud fingerprint;

[0146] The processing module 50 is configured to process the page supplied material to obtain a complete thinned point cloud material.

[0147] In some embodiments, the sampling module 20 is further configured to:

[0148] Sampling the point cloud data a preset number of times at preset time intervals; or

[0149] The point cloud data is rotated and sampled at intervals according to a preset interval sampling angle.

[0150] In some embodiments, the generation module 30 is further configured to:

[0151] Grouping the plurality of thinned materials to obtain a grouping result;

[0152] A point cloud fingerprint is generated according to the grouping result.

[0153] In some embodiments, the determination module 40 is further configured to:

[0154] After receiving the page loading request, extracting corresponding thinned materials from the plurality of thinned materials as page supply materials according to the point cloud fingerprint;

[0155] The processing module 50 is further configured to:

[0156] Combining the page supply materials into complete thinned point cloud materials according to the page supply time;

[0157] A corresponding loading page is generated according to the thinned point cloud material.

[0158] In some embodiments, the point cloud data thinning device further includes a judgment module configured to:

[0159] After generating a corresponding loading page according to the thinned point cloud material, determining whether business waiting is required;

[0160] If so, the generated loading page is displayed through lossy loading.

[0161] In some embodiments, the determination module 40 is further configured to:

[0162] After receiving the page loading request, obtaining the end page code package according to the page loading request;

[0163] Loading the end page according to the end page code package;

[0164] The end pages are layered and layer accumulated;

[0165] A corresponding loading page is generated according to the layer accumulation result and the page supply material.

[0166] In some embodiments, the acquisition module 10 is further configured to:

[0167] The scanning device installed on the vehicle side scans the entire point cloud data reciprocally or cyclically.

[0168] Those skilled in the art will appreciate that the point cloud data thinning device may include more or fewer components, for example, may also include an adjustment interface for a page display interface, a communication interface, etc., or a combination of certain components, or different component arrangements.

[0169] It should be noted that the thinning device for point cloud data provided in the embodiment of the present application corresponds to the thinning method for point cloud data in the above-mentioned embodiment. Based on the above-mentioned thinning method for point cloud data, technicians in this field can understand the specific implementation methods and various variations of the thinning device for point cloud data in the embodiment of the present application. Any optional options in the embodiment of the thinning method for point cloud data are also applicable to the thinning device for point cloud data, which will not be repeated here.

[0170] An embodiment of the present application also provides a vehicle, comprising the point cloud data thinning device described in the above technical solution.

[0171] An embodiment of the present application also provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program on the memory.

[0172] In some embodiments, the processor executing the computer program may be a processing device including one or more general-purpose processing devices, such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), etc. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc.

[0173] The memory may be a read-only memory (ROM), a random access memory (RAM), a phase-change random access memory (PRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), an electrically erasable programmable read-only memory (EEPROM), other types of random access memory (RAM), a flash disk or other form of flash memory, a cache, a register, a static memory, a compact disk read-only memory (CD-ROM), a digital versatile disk (DVD) or other optical storage, a cassette or other magnetic storage device, or any other possible non-temporary medium used to store information or instructions that can be accessed by a computer device.

[0174] The electronic devices of the embodiments of the present application may include but are not limited to fixed terminal devices such as servers, desktop computers, digital TVs, etc., and mobile terminal devices such as vehicle-mounted devices (such as head-up display devices HUD), handheld devices (such as mobile phones, tablet computers, etc.), wearable devices (such as smart watches, smart bracelets, etc.), etc.

[0175] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0176] The computer-readable storage medium of the embodiment of the present application may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. In the embodiment of the present application, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device, for example, the above-mentioned memory.

[0177] The computer program of the embodiment of the present application can be organized into one or more computer executable components or modules. Any number and combination of such components or modules can be used to implement the various aspects of the present application. For example, the various aspects of the present application are not limited to the specific computer executable instructions or specific components or modules shown in the drawings and described herein. Other embodiments may include different computer executable instructions or components with more or less functions than those shown and described herein.

[0178] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A method for thinning point cloud data, characterized in that: include: Get point cloud data; Performing interval sampling on the point cloud data to extract a plurality of thinned materials from the point cloud data; Generating a point cloud fingerprint according to the plurality of the thinned materials; Determine a page supply material from a plurality of the thinned materials based on the received page load request and the point cloud fingerprint; The page supplied material is processed to obtain complete thinned point cloud material.

2. The method according to claim 1, characterized in that: The point cloud data is sampled at intervals to extract a plurality of thinned materials from the point cloud data, including: Sampling the point cloud data a preset number of times at preset time intervals; or The point cloud data is rotated and sampled at intervals according to a preset interval sampling angle.

3. The method according to claim 1, characterized in that Generating a point cloud fingerprint according to the plurality of the thinned materials includes: Grouping the plurality of thinned materials to obtain a grouping result; A point cloud fingerprint is generated according to the grouping result.

4. The method according to claim 1, characterized in that: Based on the received page loading request and the point cloud fingerprint, determining a page supply material from a plurality of the thinned materials comprises: After receiving the page loading request, extracting corresponding thinned materials from the plurality of thinned materials as page supply materials according to the point cloud fingerprint; The page supplied material is processed to obtain complete sparse point cloud material, including: Combining the page supply materials into complete thinned point cloud materials according to the page supply time; A corresponding loading page is generated according to the thinned point cloud material.

5. The method according to claim 4, characterized in that After generating a corresponding loading page according to the thinned point cloud material, the method further includes: Determine whether business waiting is required; If so, the generated loading page is displayed through lossy loading.

6. The method according to claim 1, characterized in that After receiving the page loading request, the method further includes: Acquire the end page code package according to the page loading request; Loading the end page according to the end page code package; The end pages are layered and layer accumulated; A corresponding loading page is generated according to the layer accumulation result and the page supply material.

7. The method according to claim 1, characterized in that Get point cloud data, including: The scanning device installed on the vehicle side scans the entire point cloud data reciprocally or cyclically.

8. A device for thinning point cloud data, characterized in that: include: An acquisition module configured to acquire point cloud data; A sampling module, configured to perform interval sampling on the point cloud data and extract a plurality of thinned materials from the point cloud data; A generating module, configured to generate a point cloud fingerprint according to a plurality of the thinned materials; A determination module, configured to determine a page supply material from a plurality of the thinned materials based on the received page loading request and the point cloud fingerprint; The processing module is configured to process the page supplied material to obtain a complete thinned point cloud material.

9. An electronic device, characterized in that: The method comprises at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program on the memory.

10. A computer storage medium, characterized in that: The computer-readable medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images