Display method and device of vehicle driving environment, medium and equipment

Abstract

The embodiment of the invention discloses a vehicle driving environment display method and device, a medium and equipment, and the method comprises the steps: obtaining first point cloud data at a first moment; performing classification processing on the first point cloud data to obtain second point cloud data including the type attribute of each point; based on the second point cloud data and the point cloud data of at least one second moment, determining the point cloud data of the driving environment of the vehicle at the first moment; the second moment is a moment before the first moment; and displaying the driving environment at the first moment based on the point cloud data of the driving environment. According to the embodiment of the invention, the user can effectively perceive the object form in the surrounding environment of the vehicle, and the visual expression effect of the point cloud is improved.

Description

Technical Field

[0001] The present disclosure relates to human-computer interaction technology, and in particular, to a display method, device, medium, and equipment for a vehicle driving environment. Background Art

[0002] In intelligent driving scenarios such as autonomous driving and assisted driving, visual expression can be based on point clouds. For example, based on the point cloud data collected by a lidar, the driving environment around the vehicle can be displayed on the vehicle's interaction interface. In the related art, due to factors such as the angle limitation of the lidar, cost, and the huge amount of point cloud data, the visual expression effect based on point clouds is poor. Summary of the Invention

[0003] Embodiments of the present disclosure provide a display method, device, medium, and equipment for a vehicle driving environment, so that users can effectively perceive the object forms in the vehicle's surrounding environment and improve the visual expression effect of point clouds.

[0004] In the first aspect of the embodiments of the present disclosure, a display method for a vehicle driving environment is provided, including: obtaining first point cloud data at a first moment; performing classification processing on the first point cloud data to obtain second point cloud data including the type attributes of each point; determining the point cloud data of the vehicle's driving environment at the first moment based on the second point cloud data and the point cloud data at at least one second moment; the second moment is a moment before the first moment; and displaying the driving environment at the first moment based on the point cloud data of the driving environment.

[0005] In the second aspect of the embodiments of the present disclosure, a display device for a vehicle driving environment is provided, including: an obtaining module for obtaining first point cloud data at a first moment; a first processing module for performing classification processing on the first point cloud data to obtain second point cloud data including the type attributes of each point; a second processing module for determining the point cloud data of the vehicle's driving environment at the first moment based on the second point cloud data and the point cloud data at at least one second moment; the second moment is a moment before the first moment; and a third processing module for displaying the driving environment at the first moment based on the point cloud data of the driving environment.

[0006] In the third aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, where the storage medium stores a computer program, and the computer program is used to execute the display method for a vehicle driving environment described in any one of the above embodiments of the present disclosure.

[0007] In a fourth aspect of the embodiments of the present disclosure, an electronic device is provided, which includes: a processor; a memory for storing executable instructions executable by the processor; and the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method for displaying a vehicle driving environment according to any one of the above embodiments of the present disclosure.

[0008] In a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, and when the instructions in the computer program product are executed by a processor, the method for displaying a vehicle driving environment provided by any one of the above embodiments of the present disclosure is executed.

[0009] Based on the method, device, medium, and equipment for displaying a vehicle driving environment provided by the above embodiments of the present disclosure, by classifying point cloud data, the semantic types of points in the point cloud data can be obtained. When displaying the driving environment based on the point cloud data, the shapes of objects in the environment can be effectively displayed based on the semantic types, improving the visual expression effect of the point cloud. Furthermore, by combining the point cloud data at the first moment and at least one second moment, the environmental range that the point cloud can express can be extended, facilitating the expression of the state of the global environment around the vehicle and further improving the visual expression effect of the point cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is an exemplary application scenario of the method for displaying a vehicle driving environment provided by the present disclosure;

[0011] Figure 2 is a schematic flowchart of the method for displaying a vehicle driving environment provided by an exemplary embodiment of the present disclosure;

[0012] Figure 3 is a schematic diagram of a display interface of a driving environment provided by an exemplary embodiment of the present disclosure;

[0013] Figure 4 is a schematic flowchart of the method for displaying a vehicle driving environment provided by another exemplary embodiment of the present disclosure;

[0014] Figure 5 is a schematic flowchart of the method for displaying a vehicle driving environment provided by still another exemplary embodiment of the present disclosure;

[0015] Figure 6 is a schematic flowchart of the method for displaying a vehicle driving environment provided by yet another exemplary embodiment of the present disclosure;

[0016] Figure 7 is a schematic flowchart of point cloud sparsification provided by an exemplary embodiment of the present disclosure;

[0017] Figure 8 is a schematic flowchart of the method for displaying a vehicle driving environment provided by still another exemplary embodiment of the present disclosure;

[0018] Figure 9 It is a schematic diagram of a driving environment display interface provided by another exemplary embodiment of the present disclosure;

[0019] Figure 10 It is a schematic structural diagram of a display device for a vehicle driving environment provided by an exemplary embodiment of the present disclosure;

[0020] Figure 11 It is a schematic structural diagram of a display device for a vehicle driving environment provided by another exemplary embodiment of the present disclosure;

[0021] Figure 12 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0022] To explain the present disclosure, exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all embodiments. It should be understood that the present disclosure is not limited by the exemplary embodiments.

[0023] It should be noted that: Unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure.

[0024] Overview of the present disclosure

[0025] In the process of implementing the present disclosure, the inventors found that in intelligent driving scenarios such as autonomous driving and assisted driving, visual expression can be based on point clouds. For example, based on the point cloud data collected by a lidar, the driving environment around the vehicle can be displayed on the vehicle's interaction interface. In the related art, due to factors such as the angle limitation of the lidar, cost, and the huge amount of point cloud data, the visual expression effect based on point clouds is poor. For example, the scanning angle of the lidar is limited, and it can only generate point clouds in a partial area in front of the vehicle, so it can only express the environment in a partial area in front of the vehicle.

[0026] Exemplary overview

[0027] Figure 1 It is an exemplary application scenario of the display method for a vehicle driving environment provided by the present disclosure. As Figure 1As shown, during the driving process of the vehicle (i.e., the host vehicle) 11 on the road, it is possible to collect point cloud data based on the sensors 12 on the host vehicle 11. For example, it is possible to collect the point cloud data at the current moment in real time. Furthermore, the display method of the driving environment of the vehicle according to the embodiments of the present disclosure can be used to display the driving environment of the host vehicle 11, so that the in-vehicle user can effectively perceive the surrounding environment of the host vehicle 11 through the displayed driving environment. For example, other vehicles around the host vehicle 11 (such as the other vehicle 13 in the figure), buildings 14 beside the road, trees 15, etc. can be viewed. Specifically, it is possible to obtain the first point cloud data at the first moment; the first moment can be any moment. For example, the first moment can be the current moment or a historical moment. Furthermore, it is possible to perform classification processing on the first point cloud data to obtain the second point cloud data containing the type attributes of each point; based on the second point cloud data and the point cloud data at at least one second moment, determine the point cloud data of the driving environment of the vehicle at the first moment; the second moment is a moment before the first moment. For example, when the first moment is the current moment, the second moment is a historical moment. Then, based on the point cloud data of the driving environment, the driving environment at the first moment can be displayed. Since the point cloud data of the driving environment contains the type attributes of the points (i.e., semantic types), that is, each point belongs to the other vehicle 13, or belongs to the building 14, or belongs to the tree 15, or belongs to the road surface, etc., when displaying the driving environment, it is possible to perform rendering and display according to the type attributes of the points, so that the displayed driving environment can effectively express the shapes of the objects, such as the shape of the other vehicle 13, the shape of the building 14, the shape of the tree 15, etc., effectively improving the visual expression effect. And since the point cloud data of the driving environment at the first moment is determined by combining the point cloud data at multiple moments, relative to the host vehicle 11 at the first moment, the point cloud data at different moments represents the environments in different ranges around the host vehicle 11 at the first moment. Therefore, it is possible to effectively express the state of the overall environment around the host vehicle 11 at the first moment, further improving the visual expression effect.

[0028] Exemplary method

[0029] Figure 2 is a schematic flowchart of a method for displaying a driving environment of a vehicle provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to an electronic device, specifically, for example, on an in-vehicle computing platform (or in-vehicle terminal), such as Figure 2 As shown, the method according to the embodiments of the present disclosure can include the following steps:

[0030] Step 210, obtain the first point cloud data at the first moment.

[0031] Among them, the first moment can be any moment during the driving process of the vehicle. For example, the first moment can be the current moment or any historical moment. The first point cloud data can be data collected based on the sensors on the vehicle.

[0032] In some alternative embodiments, the first point cloud data can be collected by a lidar. Alternatively, the data collected by other sensors can be converted into the first point cloud data. Alternatively, the first point cloud data can be obtained by combining a lidar and other sensors. The specific method for obtaining the point cloud data is not limited.

[0033] Step 220: Classify the first point cloud data to obtain a second point cloud data containing the type attributes of each point.

[0034] Wherein, the type attribute of a point refers to the type of the object to which the point belongs. The type attribute can include the types of various objects that may exist in the real driving environment. The objects can include dynamic objects and / or static objects. Dynamic objects are, for example, other vehicles, pedestrians, cyclists, animals, etc. that may move around the host vehicle, and static objects are, for example, static objects such as buildings, trees, roads, etc.

[0035] In some alternative embodiments, the first point cloud data can be classified based on a pre-configured point cloud classification model to obtain a second point cloud data containing the type attributes of each point. The point cloud classification model can adopt any implementable model, which is not limited in the embodiments of the present disclosure.

[0036] In some alternative embodiments, the step 220 of classifying the first point cloud data to obtain a second point cloud data containing the type attributes of each point includes:

[0037] Obtain image data at a first moment; classify the first point cloud data based on the image data and the first point cloud data to obtain the type attributes respectively corresponding to each point in the first point cloud data; and obtain the second point cloud data based on the first point cloud data and the type attributes respectively corresponding to each point.

[0038] Wherein, the image data can be collected by a camera on the vehicle. Combining the observation content of the environment in the image data for classifying the first point cloud data helps to improve the accuracy of the classification result of the first point cloud data.

[0039] In some alternative embodiments, a pre-trained multi-modal classification model can be used to process the image data and the first point cloud data to obtain the type attributes respectively corresponding to each point in the first point cloud data. Multi-modal means that the input includes data of multiple modalities. One sensor can correspond to one modality, and data from different sensors belong to different modalities of data. For example, the point cloud data is data of one modality, and the image data is data of another modality.

[0040] Step 230: Determine the point cloud data of the driving environment of the vehicle at the first moment based on the second point cloud data and the point cloud data at at least one second moment.

[0041] Among them, the second moment is a moment before the first moment. For example, if the first moment is represented as the t moment, the second moment can be the t - 1 moment, the t - 2 moment, the t - 3 moment, and so on. The number of second moments is not limited. The point cloud data of the driving environment of the vehicle at the first moment refers to the point cloud data of the environment around the vehicle at the first moment. For example, the environmental point cloud data in all directions such as in front of, behind, to the left, and to the right of the vehicle.

[0042] In some alternative embodiments, the point cloud data of the driving environment of the vehicle at the first moment can be the point cloud data within a preset range around the vehicle at the first moment. The preset range can be determined according to the sensing range of the sensors on the vehicle. For example, the preset range can be the range from d1 meters behind the vehicle to d2 meters in front of the vehicle, and from d3 meters on the left side to d4 meters on the right side of the vehicle.

[0043] In some alternative embodiments, the number of second moments can be determined based on the preset range, reducing the fusion of unnecessary point cloud data. For example, as the vehicle travels, the number of historical moments relative to the current moment (the first moment) continuously increases. The point cloud data of historical moments with a relatively large time interval from the current moment is no longer within the driving environment range of the current moment. Therefore, one or more second moments closer to the first moment can be determined for determining the point cloud data of the driving environment at the first moment.

[0044] In some alternative embodiments, the point cloud data of the second moment can be point cloud data containing type attributes. For example, the point cloud data of the second moment can be the point cloud data obtained by classifying the collected point cloud data at the second moment.

[0045] In some alternative embodiments, the point cloud data of any second moment can be the single - frame point cloud data of that second moment. The single - frame point cloud data can be point cloud data containing type attributes. For example, at that second moment, the collected point cloud data is classified to obtain single - frame point cloud data containing type attributes, and the single - frame point cloud data containing type attributes is stored. Then, at the first moment, the second point cloud data at the first moment is fused with the point cloud data of at least one second moment to obtain a larger - range point cloud data at the first moment, which is used to determine the point cloud data of the driving environment of the vehicle.

[0046] In some alternative embodiments, the point cloud data at any second moment may be the fused point cloud data at that second moment. The fused point cloud data refers to fusing the single-frame point cloud data at the second moment with the point cloud data at at least one moment before the second moment, and taking the fused point cloud data as the point cloud data at the second moment. That is to say, the point cloud data at any second moment may be the point cloud data of the overall range around the vehicle itself at that second moment. For example, at the initial moment T0, the 0th frame of point cloud data is obtained; at the moment T1, the 1st frame of point cloud data is obtained, and the 1st frame of point cloud data is fused with the 0th frame of point cloud data, and the fused point cloud data is taken as the point cloud data at the moment T1; at the moment T2, the 2nd frame of point cloud data is obtained, and the 2nd frame of point cloud data is fused with the point cloud data at the moment T1, and the fused point cloud data is taken as the point cloud data at the moment T2, and so on. Since the point cloud data at the moment T1 already incorporates the 0th frame of point cloud data, at the moment T2, it can be fused only with the point cloud data at the moment T1. Of course, the 2nd frame of point cloud data can also be fused with the point cloud data at the moment T1 and the point cloud data at the moment T0, and there is no specific limitation. Optionally, the single-frame point cloud data at each moment may be point cloud data including type attributes.

[0047] In some alternative embodiments, when the point cloud data at the second moment is fused point cloud data, the point cloud data of the driving environment of the vehicle at the first moment may be determined based on the second point cloud data and the point cloud data at the second moment closest to the first moment.

[0048] In some alternative embodiments, when fusing the second point cloud data with the point cloud data at the second moment, static point cloud data can be extracted from the point cloud data at the second moment, and the static point cloud data at the second moment can be fused with the second point cloud data to avoid the adverse impact of the movement of the dynamic point cloud data on the fusion result, making the dynamic point cloud data perceived at the first moment more accurate. Optionally, for the dynamic point cloud data in the point cloud data at the second moment, the positions of the points in the dynamic point cloud data at the first moment can be predicted according to the motion state of the dynamic object corresponding to the perceived dynamic point cloud data, and the dynamic point cloud data can be aligned to the first moment and fused with the second point cloud data. For the static point cloud data at the second moment, through the conversion between coordinate systems, the static point cloud data and the second point cloud data can be unified into the same coordinate system and fused in the same coordinate system. Optionally, the conversion relationship between the coordinate system at the second moment and the coordinate system at the first moment can be determined based on the motion parameters of the vehicle itself and the time difference. For example, according to the motion parameters of the vehicle itself and the time difference, the relative pose of the vehicle itself at the first moment relative to the second moment can be determined, and this relative pose represents the translation and rotation of the coordinate system at the first moment relative to the second moment, so that the conversion relationship between the coordinate system at the second moment and the coordinate system at the first moment can be determined, and then the static point cloud data can be converted to the coordinate system at the first moment based on the conversion relationship. Or, the conversion relationship between the coordinate system at the second moment and the coordinate system at the first moment can be determined based on the pose of the vehicle itself at the second moment and the pose of the vehicle itself at the first moment. The pose of the vehicle itself at any moment is the pose of the vehicle itself in the global coordinate system (such as the world coordinate system or the vehicle coordinate system at the starting position of the vehicle itself), representing the external parameters of the vehicle itself relative to the global coordinate system. Thus, with the global coordinate system as a reference, the conversion relationship between the coordinate system at the second moment and the coordinate system at the first moment can be determined. The pose of the vehicle itself at any moment can be determined by a positioning algorithm.

[0049] In some alternative embodiments, on the basis of including the type attributes of each point, the point cloud data of the driving environment can further include the state attributes of each point. The state attributes can include, for example, at least one of the color attribute, size attribute, transparency attribute, etc. of each point. Among them, the color attribute of each point can be used to display the color of the object, the transparency attribute of each point can be used to display the occlusion situation between objects at different positions, and the size attribute of each point can be used to represent the distance of the point. Through the state attributes of each point in the point cloud data, the visual effect of expressing the object by the point cloud can be further improved.

[0050] In some alternative embodiments, the points can be inflated in three-dimensional space into spheres or other shapes of a certain size through the size attribute of the points, so that the points in the point cloud data of the driving environment are three-dimensional shapes of a certain size (such as spheres). When displayed, the points at a short distance will occupy a larger area (or a larger number of pixels) in the display screen, and the points at a long distance will occupy a smaller area (or a smaller number of pixels), thereby distinguishing the distance of the points through the displayed size and giving a sense of distance to the displayed points.

[0051] In some alternative embodiments, the points may not be inflated in three-dimensional space, and the size attribute of the points is retained. Instead, during the 3D rendering process, the area size (or the number of pixels) occupied by each point in the display screen is determined according to the size attribute of the points. For example, the point is projected onto the display screen to obtain the projection point of the point on the display screen, and based on the area size occupied by the point in the display screen, the area occupied by the point in the display screen is determined with the projection point as the center. For example, the point is displayed as a circular area of a certain size in the display screen.

[0052] Step 240: Based on the point cloud data of the driving environment, display the driving environment at the first moment.

[0053] Among them, the driving environment at the first moment can be displayed on the display screen of the vehicle. The display screen of the vehicle can be any display screen in the vehicle, such as a central control screen, a HUD (Head-Up Display), a driver HMI, a passenger HMI, etc.

[0054] In some alternative embodiments, the point cloud data located within the display range can be determined from the point cloud data of the driving environment based on the specified display range; based on the camera parameters of the virtual camera relative to the vehicle's own camera pre-configured, the conversion relationship from the coordinate system of the point cloud data (such as the coordinate system of the vehicle's own at the first moment) to the coordinate system of the virtual camera is determined. According to this conversion relationship, the point cloud data within the display range is converted to the coordinate system of the virtual camera, and then projected onto the image coordinate system of the virtual camera to generate a display screen, and this display screen is displayed to achieve the display of the driving environment. The virtual camera is an observation camera for 3D rendering of the driving environment, that is, from the perspective of the virtual camera, it simulates the user's eyes observing the environment within a certain range around the vehicle at the position of the virtual camera, so that the user can observe the overall driving environment through the display screen. The position of the virtual camera can be a specified position above the rear of the vehicle. The parameters of the virtual camera include external parameters and internal parameters, which can be pre-calibrated according to actual needs. The specific display method is not limited to the above method and will not be elaborated here.

[0055] In some alternative examples, Figure 3 is a schematic diagram of the display interface of the driving environment provided by an exemplary embodiment of the present disclosure. As Figure 3As shown, the surrounding environment of the host vehicle 11 includes trees, other vehicles, lane lines, road surfaces, curbs, etc. By performing visual expression of the point cloud based on the method of the embodiments of the present disclosure, the state of the overall environment around the host vehicle 11 can be displayed, and the shape of the objects in the environment can be effectively displayed.

[0056] Optionally, in practical applications, the perception results of the map and / or other sensors can also be combined for displaying the driving environment, such as the navigation map in the lower right corner and the lane indicator in the upper left corner in the figure. Other sensors such as cameras, and the specific display content is not limited. For example, the lane lines, lanes, etc. can be perceived by combining the image data collected by the camera and rendered into the display screen. The display range of the display screen can be referenced by the host vehicle, that is, the host vehicle is always displayed at a specified position on the display screen, and the environment around the host vehicle changes dynamically according to the actual point cloud data.

[0057] The method for displaying the vehicle driving environment provided in this embodiment can obtain the semantic types of the points in the point cloud data by classifying the point cloud data at the first moment. When displaying the driving environment based on the point cloud data, the shape of the objects in the environment can be effectively displayed based on the semantic types, improving the visual expression effect of the point cloud. Furthermore, by combining the point cloud data at the first moment and at least one second moment, the point cloud data of the driving environment at the first moment is determined. Relative to the vehicle at the first moment, the point cloud data at different moments can represent the environments in different ranges around the vehicle at the first moment. Therefore, the state of the overall or larger range of the environment around the vehicle at the first moment can be effectively expressed, further improving the visual expression effect.

[0058] Figure 4 It is a schematic flowchart of the method for displaying the vehicle driving environment provided by another exemplary embodiment of the present disclosure.

[0059] In some optional embodiments, on the basis of the above Figure 2 shown embodiment, as Figure 4 shown, step 230 of determining the point cloud data of the driving environment of the vehicle at the first moment based on the second point cloud data and the point cloud data at at least one second moment may include:

[0060] Step 2310, determining the sparsified target fusion point cloud data based on the second point cloud data and the point cloud data at at least one second moment.

[0061] Among them, sparsification means reducing the number of points in the point cloud data. On the basis of ensuring that the point cloud data can effectively represent the shape of the object, redundant points are removed.

[0062] In some alternative embodiments, the second point cloud data can be first fused with the point cloud data at the second moment, and then the fused point cloud data can be sparsified to obtain the target fused point cloud data. Alternatively, the second point cloud data can be first sparsified, and then the sparsified point cloud data can be fused with the point cloud data at the second moment to obtain the target fused point cloud data. For example, in the case where the point cloud data at the second moment is already a sparsified point cloud, the sparsification of the point cloud data at the second moment can be avoided at the first moment. Then, the second point cloud data at the first moment can be first sparsified, and then the sparsified point cloud data can be fused with the point cloud data at the second moment. That is, the order of sparsification and fusion is not limited.

[0063] In some alternative embodiments, the second point cloud data or the fused point cloud data can be sparsified according to a preconfigured sparsification rule. The sparsification rule can be set according to factors such as the density requirements of points for different types of objects and the different degrees of attention to different regions of the driving environment. For example, for regions that need to be focused on (such as the region in front of the vehicle itself or other regions), a higher point density can be ensured, and for regions with a lower degree of attention, a lower point density can be set. Corresponding point densities can be set respectively for the point clouds of objects with different types of attributes. The specific sparsification rule is not limited.

[0064] Step 2320: Based on the target fused point cloud data, determine the point cloud data of the driving environment of the vehicle at the first moment.

[0065] In some alternative embodiments, after obtaining the target fused point cloud data, the target fused point cloud data can be determined as the point cloud data of the driving environment of the vehicle at the first moment.

[0066] In some alternative embodiments, the target fused point cloud data can be further processed to obtain the point cloud data of the driving environment of the vehicle at the first moment. For example, the target fused point cloud data can be further 3D processed to obtain the point cloud data of the driving environment, so that the displayed driving environment has a stronger sense of space and further improves the visual expression effect. The 3D processing can include, for example, color gradient of the point cloud data and size gradient of the points in the point cloud data.

[0067] In the embodiments of the present disclosure, by sparsifying the point cloud data, the number of points in the point cloud data can be effectively reduced, avoiding the large load on the in-vehicle display system caused by the full display of the point cloud, reducing the occupation of storage space, and improving the display processing efficiency.

[0068] Figure 5 It is a schematic flowchart of a method for displaying a vehicle driving environment provided by another exemplary embodiment of the present disclosure.

[0069] In some alternative embodiments, such as Figure 5As shown in the figure, determining the thinned target fused point cloud data based on the second point cloud data and the point cloud data at at least one second moment in step 2310 may include:

[0070] Step 23110: Thin the second point cloud data based on the type attributes of the points in the second point cloud data to obtain a third point cloud data.

[0071] Among them, the type attribute of each point represents the object type to which each point belongs. The minimum density of the required points can be set separately for different types of objects. On the basis of ensuring that the density requirements of each type of object are met, the second point cloud data is thinned to obtain the third point cloud data. For example, for the points in the point cloud with the same type attribute, sampling is performed at a certain distance interval. For multiple points that are relatively close, only one or fewer points are sampled as the retained points, and other points are filtered out, thereby reducing the number of points. The specific thinning method is not limited.

[0072] Step 23120: Determine the target fused point cloud data based on the third point cloud data and the point cloud data at at least one second moment.

[0073] Among them, the point cloud data at the second moment can be the data thinned at the second moment. The third point cloud data and the point cloud data at the second moment can be fused to obtain the target fused point cloud data.

[0074] In some optional embodiments, the static point cloud data in the point cloud data at the second moment can be fused with the third point cloud data to obtain the target fused point cloud data. Optionally, for the dynamic point cloud data in the point cloud data at the second moment, the position of the dynamic point cloud data at the first moment can be predicted through the motion state of the dynamic point cloud data. Furthermore, the dynamic point cloud data can be aligned to the first moment and in the same coordinate system as the third point cloud data to obtain the aligned dynamic point cloud data. The static point cloud data in the point cloud data at the second moment is converted to the same coordinate system as the third point cloud data, and the aligned dynamic point cloud data and the static point cloud data are fused with the third point cloud data to obtain the target fused point cloud data.

[0075] In the embodiments of the present disclosure, by thinning the point cloud data, the number of points in the point cloud data can be effectively reduced, avoiding the large load on the vehicle-mounted display system caused by the full display of the point cloud, reducing the occupation of storage space, and improving the display processing efficiency. And when the point cloud data at the second moment is the thinned point cloud data, at the first moment, only the second point cloud data at the first moment needs to be thinned, avoiding unnecessary thinning processing of the point cloud data at the second moment, and improving the thinning processing efficiency.

[0076] Figure 6It is a schematic flowchart of a method for displaying a vehicle driving environment provided by another exemplary embodiment of the present disclosure.

[0077] In some alternative embodiments, such as Figure 6 shown, determining the thinned target fused point cloud data based on the second point cloud data and the point cloud data at at least one second moment in step 2310 may include:

[0078] Step 23101, determining the fused point cloud data based on the second point cloud data and the point cloud data at at least one second moment.

[0079] Among them, the specific fusion operation of the second point cloud data and the point cloud data at the second moment can be referred to the foregoing embodiments and will not be elaborated here.

[0080] Step 23102, thinning the fused point cloud data based on the type attributes of the points in the fused point cloud data to obtain the target fused point cloud data.

[0081] Among them, the specific operation of thinning the fused point cloud data can be referred to the thinning operation of the second point cloud data and will not be elaborated here.

[0082] In some alternative embodiments, for the case where the point cloud data at the second moment is dense point cloud data, the second point cloud data and the point cloud data at the second moment can be first fused to obtain the fused point cloud data, and then the fused point cloud data can be thinned to obtain the target fused point cloud data.

[0083] In some alternative embodiments, for the case where the point cloud data at the second moment is thinned point cloud data, by fusing the second point cloud data and the point cloud data at the second moment to obtain the fused point cloud data, and then thinning the fused point cloud data, the point cloud data in the overlapping region of the second point cloud data and the point cloud data at the second moment can be thinned again, further reducing the number of points in the overlapping region.

[0084] In the embodiments of the present disclosure, the target fused point cloud data is obtained by the method of first fusing and then thinning, which can ensure the overall thinning effect of the target fused point cloud data, avoid too high point density in the overlapping region of the point cloud data at the first moment and the point cloud data at the second moment, further reduce the number of points in the overlapping region, and improve the display performance.

[0085] In some alternative embodiments, on the basis of the above embodiments, thinning the second point cloud data based on the type attributes of the points in the second point cloud data in step 23110 to obtain the third point cloud data includes:

[0086] Based on the type attributes of the points in the second point cloud data, determine the first sampling density corresponding to each type attribute; and / or, based on the current pose of the vehicle, determine the viewing range of a virtual camera for observing the driving environment; based on the viewing range, determine the second sampling density of the second point cloud data; sample the second point cloud data based on the first sampling density and / or the second sampling density to obtain the third point cloud data.

[0087] Among them, the first sampling density corresponding to any type attribute is the sampling density threshold that meets the point density requirements of the objects of that type attribute. The first sampling density serves as a constraint condition for sparsifying the point cloud data of the corresponding type attribute, such that the density of the sparsified point cloud is close to the first sampling density. The current pose of the vehicle is the pose of the vehicle at the first moment. The virtual camera is a pre-set camera with a fixed viewing range relative to the host vehicle, for example, a camera that observes the environment including the host vehicle and a preset range around the host vehicle from above the rear of the host vehicle. Refer to Figure 3 the viewing angle of the display screen shown. There may be no such camera on the actual vehicle. This virtual camera is used for rendering the display screen. Refer to the rendering-related content of the foregoing embodiments.

[0088] In some alternative embodiments, the range around the host vehicle can be divided into different levels of partitions according to the viewing range of the virtual camera and the degree of attention to the environment in different ranges. For example, the key attention area is used as a first-level partition and a relatively high sampling density is set. One or more other levels of partitions can be set after the first-level partition, and the specific number of levels is not limited. Different levels of partitions can be set with different second sampling densities.

[0089] In some alternative embodiments, the second point cloud data can be downsampled based on at least one of the first sampling density and the second sampling density to obtain the third point cloud data.

[0090] In the embodiments of the present disclosure, by using the first sampling density corresponding to the type attribute to constrain the sparsification of the second point cloud data, it can be ensured that the sparsified point cloud meets the density requirements of various types of objects. By determining the second sampling density based on the viewing range of the virtual camera to constrain the sparsification of the second point cloud data, it can be ensured that the sparsified point cloud meets the density requirements of the attention areas with different degrees of importance, thereby ensuring the visual expression effect while greatly reducing the number of points.

[0091] In some alternative embodiments, the step of sparsifying the second point cloud data based on the type attributes of the points in the second point cloud data in step 23110 to obtain the third point cloud data includes:

[0092] Based on the type attributes of the points in the second point cloud data, determine the first sampling density corresponding to each type attribute; sample the second point cloud data based on the first sampling density to obtain the third point cloud data.

[0093] Among them, the specific sampling operation can be referred to the foregoing embodiments.

[0094] In some alternative embodiments, the thinning of the second point cloud data based on the type attributes of the points in the second point cloud data in step 23110 to obtain the third point cloud data includes:

[0095] Based on the current pose of the vehicle, determine the viewing angle range of the virtual camera for observing the driving environment; based on the viewing angle range, determine the second sampling density of the second point cloud data; based on the second sampling density, sample the second point cloud data to obtain the third point cloud data.

[0096] Among them, the specific sampling operation can be referred to the foregoing embodiments.

[0097] In some alternative embodiments, the thinning of the second point cloud data based on the type attributes of the points in the second point cloud data in step 23110 to obtain the third point cloud data includes:

[0098] Based on the type attributes of the points in the second point cloud data, determine the first sampling density corresponding to each type attribute; based on the current pose of the vehicle, determine the viewing angle range of the virtual camera for observing the driving environment; based on the viewing angle range, determine the second sampling density of the second point cloud data; based on the first sampling density and the second sampling density, sample the second point cloud data to obtain the third point cloud data.

[0099] Among the points in the second point cloud data, for some point clouds that have type attributes and belong to the partition within the viewing angle range, the sampling density that can simultaneously meet the density requirements of the type attributes and the partition can be determined by combining the first sampling density and the second sampling density as the sampling density of the point cloud. For example, if the first sampling density is less than the second sampling density, the second sampling density is used as the sampling density of the point cloud to ensure that the densities of the thinned point clouds of different types and different partitions are within the required range and avoid affecting the display effect. The larger the sampling density, the more points are retained after thinning, and the better the display effect. Therefore, according to the balance experience of the display effect and the load, the first sampling density of different type attributes and the second sampling density of different visual partitions can be set to ensure a good display effect on the basis of significantly reducing the number of points.

[0100] In some alternative embodiments, based on the above-mentioned any one of the embodiments, determining the second sampling density of the second point cloud data based on the viewing angle range may include:

[0101] Based on the viewing angle range, determine at least one level of sub-point cloud data from the second point cloud data; according to the levels of the sub-point cloud data, determine the second sampling density corresponding to each sub-point cloud data.

[0102] Among them, different levels of partitions can be set according to the importance of different regions within the visual range. Each partition corresponds to a set of sub-point cloud data. Different levels of partitions correspond to different second sampling densities. The number of levels can be 2, 3, 4, etc., and specific values are not limited.

[0103] In the embodiments of the present disclosure, by dividing the viewing range of the virtual camera into different levels, a relatively large density can be retained for the key attention areas, and a relatively small density can be adopted for the secondary key areas, so as to reduce the number of points as much as possible on the basis of ensuring the display effect.

[0104] Figure 7 It is a schematic flowchart of point cloud sparsification provided by an exemplary embodiment of the present disclosure.

[0105] In some optional embodiments, based on the type attributes of the points in the second point cloud data in any of the above embodiments, sparsifying the second point cloud data to obtain the third point cloud data includes:

[0106] Step 231101, based on the type attributes of the points in the second point cloud data, determining the first object point cloud corresponding to each type attribute.

[0107] Among them, the first object point cloud corresponding to any type attribute refers to the set of points of the objects in the second point cloud data that belong to the same type attribute.

[0108] In some optional embodiments, for each type attribute, one or more first object point clouds corresponding to each object can be provided.

[0109] In some optional embodiments, for the set of points in the second point cloud data that belong to the same type attribute, the first object point cloud belonging to each object in the point set can be determined. For example, the second point cloud data includes one or more objects of the same type, such as multiple trees, multiple buildings, etc. After determining the point cloud corresponding to each type attribute according to the type attribute, the first object point cloud corresponding to each object can be determined by clustering or other means.

[0110] In some optional embodiments, the type attribute may include the object type and object identifier (such as object ID) to which the point belongs, that is, the IDs of multiple objects belonging to the same class are different. The first object point cloud corresponding to each object in each type attribute can be determined based on the object type and object identifier. Optionally, for the object type, one type can correspond to one object, or one object can correspond to one type. For example, multiple buildings correspond to different types respectively.

[0111] In some alternative embodiments, the object type of the same object can be represented as at least a first-level type. For example, the object corresponds to a first-level type, and different parts of the object can correspond to different second-level types. For instance, a tree is a first-level type, and a tree includes parts such as leaves, trunks, and flowers. The leaves, trunks, and flowers can respectively correspond to different second-level types, so that when displayed, different parts can display corresponding states. For example, the leaves are displayed as green, the trunks are displayed as gray, and the flowers are displayed as the color of the flowers, and so on. The specific type hierarchy can be set according to the actual object requirements, and the embodiments of the present disclosure do not make limitations in this regard.

[0112] Step 231102: Determine a first object contour point cloud and a first object internal point cloud according to the first object point cloud.

[0113] In some alternative embodiments, the first object contour point cloud can be determined by a pre-configured point cloud contour extraction algorithm or edge detection algorithm, etc., and then the point set in the first object point cloud except the first object contour point cloud is determined as the first object internal point cloud. The specific manner of determining the first object contour point cloud is not limited.

[0114] Step 231103: Sample the first object contour point cloud according to the third sampling density corresponding to the first object contour point cloud to obtain a second object contour point cloud.

[0115] Among them, since the point cloud can better express the contour of the object, a higher density can be retained for the contour of the object to make the object contour clearer when displayed. Based on this, the third sampling density can be determined according to the display effects of different object point clouds at different densities. Based on the third sampling density, downsample the first object contour point cloud to obtain a sparsified second object contour point cloud, reduce the number of points in the object contour point cloud, and ensure the effective expression of the object contour.

[0116] Step 231104: Sample the first object internal point cloud according to the distances between the points in the first object internal point cloud and the first object contour point cloud according to the density gradient rule to obtain a second object internal point cloud.

[0117] Among them, on the basis of clearly expressing the object contour, for the point cloud near the center of the object, if a smaller density is used, the impact on the user's perception is smaller, and the closer to the center, the smaller the impact. Therefore, for the first object internal point cloud, the density can be relatively reduced compared to the contour point cloud to further reduce the data volume of the point cloud. The density gradient rule is that the density gradually decreases from the object contour to the object center. Sample the first object internal point cloud according to the density gradient rule to obtain a second object internal point cloud, so that the density of the second object internal point cloud gradually decreases from the contour to the center.

[0118] In some alternative embodiments, the internal point cloud of the first object may be sampled according to the density gradient rule based on the distances between the points in the internal point cloud of the first object and the contour point cloud of the second object, to obtain the internal point cloud of the second object.

[0119] It should be noted that steps 231103 and 231104 are not in a sequential order.

[0120] Step 231105: Based on the contour point cloud of the second object and the internal point cloud of the second object, determine the thinned second object point cloud corresponding to the first object point cloud.

[0121] Among them, the contour point cloud of the second object and the internal point cloud of the second object may be combined to obtain the thinned second object point cloud.

[0122] Step 231106: Based on the second object point cloud, determine the third point cloud data.

[0123] Among them, for each first object point cloud, downsampling is performed according to the above process to obtain the corresponding thinned second object point cloud, and then the third point cloud data can be obtained based on each second object point cloud. For example, the thinned second object point cloud replaces the corresponding first object point cloud in the second point cloud data to obtain the third point cloud data.

[0124] In the embodiments of the present disclosure, by maintaining a relatively high density for the object contour point cloud and gradually decreasing the density of the internal point cloud of the object from the contour to the center of the object, the data volume of the point cloud can be further reduced while ensuring the clarity of the object contour, and the display performance can be further improved.

[0125] In some alternative embodiments, step 23110 of thinning the second point cloud data based on the type attributes of the points in the second point cloud data to obtain the third point cloud data includes:

[0126] Based on the current pose of the vehicle, determine the viewing range of the virtual camera for observing the driving environment; based on the viewing range, determine the second sampling density of the second point cloud data; based on the second sampling density, sample the second point cloud data to obtain intermediate point cloud data; furthermore, based on the type attributes of the points in the intermediate point cloud data, determine the first object point clouds corresponding to the respective type attributes; according to the first object point clouds, determine the first object contour point cloud and the first object internal point cloud; according to the third sampling density corresponding to the first object contour point cloud, sample the first object contour point cloud to obtain the second object contour point cloud; according to the distances between the points in the first object internal point cloud and the first object contour point cloud, sample the first object internal point cloud according to the density gradient rule to obtain the second object internal point cloud; based on the second object contour point cloud and the second object internal point cloud, determine the thinned second object point cloud corresponding to the first object point cloud; based on the second object point cloud, determine the third point cloud data.

[0127] Among them, the specific operation of obtaining the intermediate point cloud data can refer to the specific operation of obtaining the third point cloud data based on the second sampling density in the foregoing embodiment. The specific operation of obtaining the third point cloud data based on the intermediate point cloud data refers to steps 231101 to 231106, which will not be elaborated here.

[0128] In some optional embodiments, the specific operation of thinning the fused point cloud data based on the type attributes of the points in the fused point cloud data in step 23102 to obtain the target fused point cloud data can refer to the thinning operation in step 23110 and its associated specific operations in the foregoing embodiments. For example, simply replace the second point cloud data in the foregoing thinning-related embodiments with the fused point cloud data, and the third point cloud data with the target fused point cloud data. Details will not be repeated here.

[0129] Figure 8 It is a schematic flowchart of a method for displaying a vehicle driving environment provided by another exemplary embodiment of the present disclosure.

[0130] In some optional embodiments, step 2320 of determining the point cloud data of the vehicle's driving environment at the first moment based on the target fused point cloud data includes:

[0131] Step 23210, determining the state attributes of the points based on the type attributes of the points in the target fused point cloud data.

[0132] Among them, the state attributes include at least one of a size attribute, a color attribute, and a transparency attribute. The size attribute is used to represent the distance of the point. That is, according to the rule that objects appear larger when closer and smaller when farther away visually, the size attribute of the point is determined to add a sense of distance to the display of the point cloud data. The color attribute is used to represent the color of the object to facilitate the user's quick identification of different objects. The transparency attribute is used to increase the sense of perspective.

[0133] In some alternative embodiments, the color attributes of each point can be determined according to the color of the object in the real environment.

[0134] In some alternative embodiments, the transparency attributes of each point can be determined according to the height of each point relative to the ground.

[0135] In some alternative embodiments, the size attributes of each point can be determined according to the distance between each point and the observation point. The observation point is, for example, the position of the virtual camera described above, so as to provide a sense of visual distance for the user.

[0136] In some alternative embodiments, since the point cloud data is sparsified, there is a certain spatial interval between points. The principle of determining the size attributes of each point is that after expanding the points according to the size attributes, the points will not affect each other and will not affect the overall shape of the object. Based on this, the maximum threshold of the size attributes of the points can be set so that the size of the nearest point does not exceed the maximum threshold.

[0137] Step 23220: Determine the point cloud data of the driving environment at the first moment based on the state attributes of each point in the target fusion point cloud data.

[0138] Among them, the state attributes of each point in the target fusion point cloud data can be fused with the target fusion point cloud data to obtain the point cloud data including type attributes and state attributes as the point cloud data of the driving environment.

[0139] In the embodiments of the present disclosure, by adding state attributes to the points in the target fusion point cloud data, the visual expression effect of the driving environment can be further improved, and the user's visual experience can be enhanced.

[0140] In some alternative embodiments, step 23210 of determining the state attributes of each point based on the type attributes of each point in the target fusion point cloud data includes:

[0141] Based on the type attributes of each point in the target fused point cloud data, determine the point sets corresponding to each type; for each point set corresponding to a type, based on the distance between each point in the point set and the vehicle and the color system corresponding to the type, perform color gradient processing on the point set to determine the color attributes corresponding to each point in the point set; based on the color attributes corresponding to each point in each point set, obtain the color attributes of each point in the target fused point cloud data; and / or, determine the height of each point in the target fused point cloud data relative to a preset plane; based on the height of each point relative to the preset plane, perform transparency gradient processing on the target fused point cloud data to obtain the transparency attributes corresponding to each point; and / or, determine the first position of a virtual camera for observing the driving environment; based on the distance between each point in the target fused point cloud data and the first position, perform size gradient processing on each point to obtain the size attributes corresponding to each point; based on at least one of the color attributes, transparency attributes, and size attributes of each point in the target fused point cloud data, determine the state attributes of each point.

[0142] Among them, according to the type attributes of each point, each point set corresponding to each type can be extracted from the target fused point cloud data. The preset plane can be the ground or other planes parallel to the ground.

[0143] In some alternative embodiments, each type can set the color system corresponding to the type according to the color of the object of the type in the actual environment, and then perform color gradient processing on the point set within the color system corresponding to the type according to the distance between each point and the host vehicle, to obtain the color values corresponding to points at different distances as the color attributes of each point. For example, for the tree type, a green color system can be used for the crown part to gradually change from deep to light, and for the trunk part, a color system consistent with the trunk color (such as a gray color system) can be used for gradient processing. There is no limitation on the color system and color gradient direction (from deep to light or from light to deep) for each type.

[0144] In some alternative embodiments, the height of each point in the target fused point cloud data relative to the preset plane can be determined based on the distance between the coordinates of each point in a specified coordinate system (such as the host vehicle coordinate system or the world coordinate system) and the preset plane. For example, in the host vehicle coordinate system, the preset plane is the plane where z = 0, and the z coordinate of the point is the height of the point relative to the preset plane. The gradient rule for performing transparency gradient processing according to height can be that the higher the height, the more transparent, that is, the higher the height, the larger the transparency value, and the transparency value of each point is obtained as the transparency attribute.

[0145] In some alternative embodiments, the first position of the virtual camera can be set according to actual visual expression requirements. The relative pose of the virtual camera with respect to the host vehicle can be pre-configured. The relative pose may include the first position of the camera and the viewing direction (or called the perspective direction). In the host vehicle coordinate system, this relative pose is the pose of the virtual camera. The first position of the virtual camera is the observation point for generating the display screen, and the direction of the virtual camera is the direction for observing the driving environment. That is to say, the displayed driving environment is the driving environment of the vehicle viewed from the observation point along the observation direction. According to the principle of objects appearing larger when closer and smaller when farther away visually, size gradient processing is performed on each point in the target fused point cloud data to obtain the size attributes corresponding to each point.

[0146] In the embodiments of the present disclosure, by determining the color attributes of each point, the color of the object can be further displayed, facilitating the user to quickly identify different objects in the environment. By determining the transparency attributes of each point, the higher the object is, the more transparent it becomes, enabling the user to see the situation of the lower objects through the higher objects. By performing gradient processing on the size of each point, a sense of distance with objects appearing larger when closer and smaller when farther away can be generated, thereby producing a perspective effect in the rendering of the 3D environment for human-computer interaction and further enhancing the visual expression effect of the point cloud.

[0147] In some alternative embodiments, determining the state attributes of each point based on the type attributes of each point in step 23210 includes:

[0148] Based on the type attributes of each point in the target fused point cloud data, determining the point sets corresponding to each type respectively; for each point set corresponding to a type, based on the distance between each point in the point set and the vehicle and the color system corresponding to this type, performing color gradient processing on the point set to determine the color attributes corresponding to each point in the point set; based on the color attributes corresponding to each point in each point set, obtaining the color attributes of each point in the target fused point cloud data; based on the color attributes of each point in the target fused point cloud data, determining the state attributes of each point.

[0149] Among them, the specific operations of each step can refer to the foregoing embodiments.

[0150] In some alternative embodiments, determining the state attributes of each point based on the type attributes of each point in step 23210 includes:

[0151] Determining the height of each point in the target fused point cloud data relative to a preset plane; based on the height of each point relative to the preset plane, performing transparency gradient processing on the target fused point cloud data to obtain the transparency attributes corresponding to each point; based on the transparency attributes of each point in the target fused point cloud data, determining the state attributes of each point.

[0152] Among them, the specific operations of each step can refer to the foregoing embodiments.

[0153] In some alternative embodiments, determining the state attributes of each point based on the type attributes of each point in the target fused point cloud data in step 23210 includes:

[0154] Determine the first position of a virtual camera for observing the driving environment; perform a size fade - in and fade - out process on each point based on the distance between each point in the target fused point cloud data and the first position to obtain the size attribute corresponding to each point; determine the state attribute of each point based on the size attribute of each point in the target fused point cloud data.

[0155] Among them, the specific operations of each step can be referred to the foregoing embodiments.

[0156] In some alternative embodiments, determining the state attributes of each point based on the type attributes of each point in the target fused point cloud data in step 23210 includes:

[0157] Determine the state attributes of each point based on the color attribute and size attribute of each point in the target fused point cloud; or determine the state attributes of each point based on the color attribute and transparency attribute of each point; or, determine the state attributes of each point based on the size attribute and transparency attribute of each point; or, determine the state attributes of each point based on the color attribute, size attribute, and transparency attribute of each point. The specific operations for determining the color attribute, size attribute, and transparency attribute of each point can be referred to the foregoing embodiments and will not be elaborated here.

[0158] Among them, the specific operations of each step can be referred to the foregoing embodiments.

[0159] In some alternative embodiments, based on any of the foregoing embodiments, the method of the embodiments of the present disclosure may further include:

[0160] Determine the viewing angle range of the virtual camera; based on the viewing angle range, determine a first sub - point cloud belonging to the covered road surface from the target fused point cloud data; determine the transparency attribute of each point in the first sub - point cloud according to a preset transparency rule.

[0161] Among them, the covered road surface means that when looking at the road surface from the virtual camera, the road surface is covered by the point cloud of other objects. In order to enable the user to see the road surface, a certain transparency can be set for the first sub - point cloud of the covered road surface. The preset transparency rule may include a preset transparency value or a rule for determining the transparency value. The preset transparency value can be set according to actual needs and is not limited in the embodiments of the present disclosure. For example, semi - transparency is performed on the first sub - point cloud, that is, the preset transparency value is 50%.

[0162] In some alternative embodiments, the road surface point cloud can be determined according to the type attributes of the points. By virtue of the planarity and continuity of the road surface, the complete road surface point cloud can be determined. Based on the distances and directions of the road surface point cloud and other point clouds from the first position respectively, it can be determined whether the other object point cloud covers the road surface, and then the first sub-point cloud covering the road surface can be determined. Alternatively, the road surface point cloud and other point clouds can be projected onto the image coordinate system of a virtual camera. For the road surface points and other points in the overlapping area, if the first distance of an other point from the virtual camera is less than that of a road surface point, it is determined that this other point is a point covering the road surface, and based on this, the first sub-point cloud covering the road surface is obtained. The specific method for determining the first sub-point cloud covering the road surface is not limited.

[0163] In an embodiment of the present disclosure, by performing a transparency processing on the sub-point cloud covering the road surface, the covering degree of the point cloud on the road surface can be reduced, and the visibility of the road surface can be improved.

[0164] In some alternative embodiments, according to a preset transparency rule, determining the transparency attributes of the points in the first sub-point cloud may include:

[0165] Based on the type attributes of the points in the target fusion point cloud data, the third object point cloud of each object is determined; based on the third object point cloud of each object and the first sub-point cloud, the target object point cloud distributed in the first sub-point cloud and the corresponding complete state of the object of the target object point cloud are determined; for each target object point cloud, in response to the corresponding complete state of the object of the target object point cloud being complete, the transparency attribute of each point in the target object point cloud is determined as a preset transparency value; or, in response to the corresponding complete state of the object of the target object point cloud being incomplete, the object sub-point cloud that belongs to the same object as the target object point cloud and is not distributed in the first sub-point cloud is obtained; a transparency gradient transition process is performed on the target object point cloud and the object sub-point cloud to obtain the transparency attributes of the points in the target object point cloud and the object sub-point cloud.

[0166] Among them, the specific operation for determining the third object point cloud is similar to the operation for determining the first object point cloud in the foregoing embodiments, and will not be elaborated here. The target object point cloud may be the third object point cloud or a part of the points in the third object point cloud. The corresponding complete state of the object of the target object point cloud is a state indicating whether the third object point cloud corresponding to the target object point cloud is completely in the first sub-point cloud, that is, whether the target object point cloud is a complete third object point cloud. The complete state of the object includes two states: complete and incomplete (or non-complete).

[0167] In some alternative embodiments, the number of target object point clouds may be one or more, that is, the first sub-point cloud may include the target object point clouds of one or more objects. For example Figure 3Among them, for the trees between the current road and the oncoming road, there are partial point clouds of multiple trees covering the oncoming road surface. The first sub-point cloud covering the road surface includes the target object point clouds of multiple trees. The target object point clouds belong to the partial point clouds of the third object point clouds of the trees, that is, the object integrity state of the target object point clouds is incomplete. If the object integrity state corresponding to the target object point clouds is incomplete, another part of the object sub-point clouds that are not distributed in the first sub-point clouds corresponding to the target object point clouds can be obtained from the corresponding third object point clouds, and transparency fade transition processing is performed on the target object point clouds and the other part of the object sub-point clouds. The transition principle is that the transparency value gradually decreases to opaque from the junction of the target object point clouds and the other part of the object sub-point clouds to the side away from the target object point clouds, so as to obtain the transparency attributes of each point in the target object point clouds and the transparency attributes of each point in the other part of the object sub-point clouds. For the case where the object integrity state corresponding to the target object point clouds is complete, the transparency attribute of the target object point clouds can be determined as the preset transparency value.

[0168] In some alternative embodiments, each point in the first sub-point clouds can be matched with each point in the target object point clouds, and the object integrity state corresponding to the target object point clouds can be determined according to the matching results.

[0169] In the embodiments of the present disclosure, for the point clouds of the objects partially covering the road surface, by making the point clouds covering the road surface transparent and performing transparency fade transition processing on the point clouds of the uncovered road surface part and the point clouds of the covered road surface part, the smoothness of the transparency of the same object is ensured, and the cliff-like transparency of the same object is avoided, such as part being semi-transparent and the other part being opaque, so that the sense of fragmentation of the object can be avoided and the user's visual experience can be ensured.

[0170] In some alternative embodiments, for the transparency attributes of the first sub-point clouds covering the road surface, transparency fade processing can also be performed in combination with the height of each point in the first sub-point clouds relative to the preset plane to determine the transparency attributes of each point.

[0171] In some alternative embodiments, on the basis of determining the state attributes of each point in the target fusion point cloud data, the first sub-point clouds covering the road surface can be determined, and then according to the preset transparency rules, the transparency attributes of each point in the first sub-point clouds can be determined. Based on the transparency attributes of each point in the first sub-point clouds, the state attributes of each point in the target fusion point cloud data are updated, that is, the transparency attributes of the first sub-point clouds in the target fusion point clouds are updated. Based on the updated transparency attributes, the state attributes of each point in the target fusion point cloud data are determined.

[0172] In some alternative embodiments, step 23120 of determining the target fusion point cloud data based on the third point cloud data and the point cloud data at at least one second moment may include:

[0173] For the point cloud data at each second moment, based on the type attributes of the points in the point cloud data at the second moment, determine the static point cloud data at the second moment; based on the static point cloud data at each second moment and the third point cloud data, determine the target fused point cloud data.

[0174] Among them, the static point cloud data at the second moment refers to the point cloud data of static objects in the point cloud data at the second moment. Static objects may include, for example, buildings, trees, road surfaces, curbs, cones, traffic lights, etc.

[0175] In some alternative embodiments, the type attributes of static objects can be preset in advance, and then based on the type attributes of the points in the point cloud data at the second moment, extract the points with the type attributes of static objects to obtain the static point cloud data at the second moment.

[0176] In some alternative embodiments, the type attributes of dynamic objects can be preset in advance, and then based on the type attributes of the points in the point cloud data at the second moment, filter out the points with the type attributes of dynamic objects to obtain the static point cloud data at the second moment.

[0177] In some alternative embodiments, the static point cloud data at the second moment can be transformed through coordinate transformation to the same coordinate system as the point cloud data at the first moment. In the same coordinate system, through point cloud merging, duplicate removal, etc., the target fused point cloud data is obtained.

[0178] In some alternative embodiments, determining the fused point cloud data based on the second point cloud data and the point cloud data at at least one second moment in step 23101 may include:

[0179] For the point cloud data at each second moment, based on the type attributes of the points in the point cloud data at the second moment, determine the static point cloud data at the second moment; based on the static point cloud data at each second moment and the second point cloud data, determine the fused point cloud data.

[0180] Among them, the specific operations of each step can refer to the foregoing embodiments and will not be elaborated here.

[0181] In the embodiments of the present disclosure, when fusing the point cloud data at multiple moments, by filtering out the dynamic point cloud data in the point cloud data at the second moment, the problem that the position of the dynamic point cloud data is inaccurate at the first moment due to the movement of dynamic objects can be avoided, and the accuracy and effectiveness of the fused point cloud data are improved.

[0182] In some alternative embodiments, based on any of the above embodiments, the method of the present disclosure may further include: caching at least one of the following point cloud data: the first point cloud data at the first moment, the second point cloud data, the third point cloud data, the target fused point cloud data, and the driving environment point cloud data of the vehicle at the first moment. By caching the relevant point cloud data, historical point cloud data can be provided for the display of the driving environment at subsequent moments of the first moment. For example, at the fourth moment after the first moment (equivalent to the current first moment), the point cloud data cached at the first moment becomes historical point cloud data (equivalent to the point cloud data at the current second moment), and the driving environment at the fourth moment can be displayed by combining the point cloud data at the fourth moment, the point cloud data at the first moment, and the point cloud data at the second moment. And so on, as the vehicle travels, the overall driving environment around the vehicle can be continuously displayed.

[0183] In some alternative examples, Figure 9 is a schematic diagram of a driving environment display interface provided by another exemplary embodiment of the present disclosure. As Figure 9 shown, based on the method of the embodiments of the present disclosure, the integrity of the point cloud data within the entire field of view can be ensured, better creating a sense of the world for the HMI (Human Machine Interface) for users, enhancing the user's immersion, and enhancing the user's trust in the intelligent driving function. In practical applications, the color of each object in the environment can be displayed on the display interface. For example, based on the color attributes of each point in the point cloud data, the object color is displayed, making the form of the object closer to the object in the real world for efficient human-machine interaction expression.

[0184] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs. Moreover, in the technical solution of the present disclosure, the collection and use of the user's personal information involved are carried out with the user's knowledge and authorization, and do not involve the illegal collection and unauthorized use of the user's personal information.

[0185] Each of the above embodiments of the present disclosure can be implemented independently or combined in any combination without conflict, which can be specifically set according to actual needs, and the present disclosure does not make any limitations.

[0186] Any of the vehicle driving environment display methods provided by the embodiments of the present disclosure can be executed by any suitable electronic device with data processing capabilities, including but not limited to: electronic devices such as terminal devices and servers. Alternatively, any of the vehicle driving environment display methods provided by the embodiments of the present disclosure can be executed by a processor. For example, the processor executes any of the vehicle driving environment display methods mentioned in the embodiments of the present disclosure by calling the corresponding instructions stored in the memory. This will not be elaborated further below.

[0187] Exemplary device

[0188] Figure 10 is a schematic structural diagram of a display device for a vehicle driving environment provided by an exemplary embodiment of the present disclosure. The device in this embodiment can be used to implement the corresponding method embodiment of the present disclosure, such as Figure 10 The device shown may include: an acquisition module 51, a first processing module 52, a second processing module 53, and a third processing module 54.

[0189] The acquisition module 51 is configured to acquire first point cloud data at a first moment.

[0190] The first processing module 52 is configured to perform classification processing on the first point cloud data to obtain second point cloud data including the type attributes of each point.

[0191] The second processing module 53 is configured to determine point cloud data of the vehicle driving environment at the first moment based on the second point cloud data and point cloud data at at least one second moment; the second moment is a moment before the first moment.

[0192] The third processing module 54 is configured to display the driving environment at the first moment based on the point cloud data of the driving environment.

[0193] Figure 11 is a schematic structural diagram of a display device for a vehicle driving environment provided by another exemplary embodiment of the present disclosure.

[0194] In some optional embodiments, based on the above Figure 10 shown embodiment, as Figure 11 shown, the second processing module 53 may include: a first processing unit 531 and a second processing unit 532.

[0195] The first processing unit 531 is configured to determine thinned target fusion point cloud data based on the second point cloud data and point cloud data at at least one second moment.

[0196] The second processing unit 532 is configured to determine point cloud data of the vehicle driving environment at the first moment based on the target fusion point cloud data.

[0197] In some optional embodiments, the first processing unit 531 is specifically configured to:

[0198] Based on the type attributes of each point in the second point cloud data, thin the second point cloud data to obtain third point cloud data. Based on the third point cloud data and point cloud data at at least one second moment, determine the target fusion point cloud data.

[0199] In some optional embodiments, the first processing unit 531 is specifically configured to:

[0200] Determine the fused point cloud data based on the second point cloud data and the point cloud data at at least one second moment. Sparsify the fused point cloud data based on the type attributes of the points in the fused point cloud data to obtain the target fused point cloud data.

[0201] In some alternative embodiments, based on the above embodiments, the first processing unit 531 is specifically configured to:

[0202] Determine the first sampling density corresponding to each type attribute based on the type attributes of the points in the second point cloud data; and / or determine the viewing angle range of the virtual camera for observing the driving environment based on the current pose of the vehicle; determine the second sampling density of the second point cloud data based on the viewing angle range; sample the second point cloud data based on the first sampling density and / or the second sampling density to obtain the third point cloud data.

[0203] In some alternative embodiments, the first processing unit 531 is specifically configured to:

[0204] Determine the first sampling density corresponding to each type attribute based on the type attributes of the points in the second point cloud data; sample the second point cloud data based on the first sampling density to obtain the third point cloud data.

[0205] In some alternative embodiments, the first processing unit 531 is specifically configured to:

[0206] Determine the viewing angle range of the virtual camera for observing the driving environment based on the current pose of the vehicle; determine the second sampling density of the second point cloud data based on the viewing angle range; sample the second point cloud data based on the second sampling density to obtain the third point cloud data.

[0207] In some alternative embodiments, the first processing unit 531 is specifically configured to:

[0208] Determine the first sampling density corresponding to each type attribute based on the type attributes of the points in the second point cloud data; determine the viewing angle range of the virtual camera for observing the driving environment based on the current pose of the vehicle; determine the second sampling density of the second point cloud data based on the viewing angle range; sample the second point cloud data based on the first sampling density and the second sampling density to obtain the third point cloud data.

[0209] In some alternative embodiments, based on any of the above embodiments, the first processing unit 531 is specifically configured to:

[0210] Determine at least one level of sub-point cloud data from the second point cloud data based on the viewing angle range; determine the second sampling density corresponding to each sub-point cloud data according to the level of each sub-point cloud data.

[0211] In some alternative embodiments, based on any of the above embodiments, the first processing unit 531 is specifically configured to:

[0212] Based on the type attributes of the points in the second point cloud data, determine the first object point clouds corresponding to the respective type attributes. According to the first object point clouds, determine the first object contour point cloud and the first object interior point cloud. Sample the first object contour point cloud according to the third sampling density corresponding to the first object contour point cloud to obtain a second object contour point cloud. Sample the first object interior point cloud according to the density gradient rule based on the distances between the points in the first object interior point cloud and the first object contour point cloud to obtain a second object interior point cloud. Based on the second object contour point cloud and the second object interior point cloud, determine the thinned second object point cloud corresponding to the first object point cloud. Based on the second object point cloud, determine the third point cloud data.

[0213] In some alternative embodiments, the first processing unit 531 is specifically configured to:

[0214] Based on the current pose of the vehicle, determine the viewing angle range of a virtual camera for observing the driving environment; based on the viewing angle range, determine the second sampling density of the second point cloud data; sample the second point cloud data according to the second sampling density to obtain intermediate point cloud data; then, based on the type attributes of the points in the intermediate point cloud data, determine the first object point clouds corresponding to the respective type attributes; according to the first object point clouds, determine the first object contour point cloud and the first object interior point cloud; sample the first object contour point cloud according to the third sampling density corresponding to the first object contour point cloud to obtain a second object contour point cloud; sample the first object interior point cloud according to the density gradient rule based on the distances between the points in the first object interior point cloud and the first object contour point cloud to obtain a second object interior point cloud; based on the second object contour point cloud and the second object interior point cloud, determine the thinned second object point cloud corresponding to the first object point cloud; based on the second object point cloud, determine the third point cloud data.

[0215] In some alternative embodiments, the second processing unit 532 is specifically configured to:

[0216] Based on the type attributes of the points in the target fusion point cloud data, determine the state attributes of the points. Among them, the state attributes include at least one of a size attribute, a color attribute, and a transparency attribute. Based on the state attributes of the points in the target fusion point cloud data, determine the point cloud data of the driving environment at the first moment.

[0217] In some alternative embodiments, the second processing unit 532 is specifically configured to:

[0218] Based on the type attributes of the points in the target fused point cloud data, determine the point sets corresponding to each type; for each point set corresponding to a type, based on the distances between the points in the point set and the vehicle and the color system corresponding to the type, perform color gradient processing on the point set to determine the color attributes corresponding to the points in the point set; based on the color attributes corresponding to the points in each point set, obtain the color attributes of the points in the target fused point cloud data; and / or, determine the height of each point in the target fused point cloud data relative to a preset plane; based on the heights of the points relative to the preset plane, perform transparency gradient processing on the target fused point cloud data to obtain the transparency attributes corresponding to the points; and / or, determine the first position of a virtual camera for observing the driving environment; based on the distances between the points in the target fused point cloud data and the first position, perform size gradient processing on the points to obtain the size attributes corresponding to the points; based on at least one of the color attributes, transparency attributes, and size attributes of the points in the target fused point cloud data, determine the state attributes of the points.

[0219] In some alternative embodiments, the second processing unit 532 is specifically configured to:

[0220] Based on the type attributes of the points in the target fused point cloud data, determine the point sets corresponding to each type; for each point set corresponding to a type, based on the distances between the points in the point set and the vehicle and the color system corresponding to the type, perform color gradient processing on the point set to determine the color attributes corresponding to the points in the point set; based on the color attributes corresponding to the points in each point set, obtain the color attributes of the points in the target fused point cloud data; based on the color attributes of the points in the target fused point cloud data, determine the state attributes of the points.

[0221] In some alternative embodiments, the second processing unit 532 is specifically configured to:

[0222] Determine the height of each point in the target fused point cloud data relative to a preset plane; based on the heights of the points relative to the preset plane, perform transparency gradient processing on the target fused point cloud data to obtain the transparency attributes corresponding to the points; based on the transparency attributes of the points in the target fused point cloud data, determine the state attributes of the points.

[0223] In some alternative embodiments, the second processing unit 532 is specifically configured to:

[0224] Determine the first position of a virtual camera for observing the driving environment; based on the distances between the points in the target fused point cloud data and the first position, perform size gradient processing on the points to obtain the size attributes corresponding to the points; based on the size attributes of the points in the target fused point cloud data, determine the state attributes of the points.

[0225] In some alternative embodiments, the second processing unit 532 is specifically configured to:

[0226] Based on the color attribute and size attribute of each point in the target fusion point cloud, determine the state attribute of each point; or based on the color attribute and transparency attribute of each point, determine the state attribute of each point; or, based on the size attribute and transparency attribute of each point, determine the state attribute of each point; or, based on the color attribute, size attribute and transparency attribute of each point, determine the state attribute of each point. For the specific operations of determining the color attribute, size attribute and transparency attribute of each point, reference can be made to the foregoing embodiments and will not be elaborated herein.

[0227] In some alternative embodiments, based on any of the foregoing embodiments, the second processing unit 532 is further configured to:

[0228] Determine the viewing range of the virtual camera; based on the viewing range, determine the first sub-point cloud covering the road surface from the target fusion point cloud data; according to a preset transparency rule, determine the transparency attribute of each point in the first sub-point cloud.

[0229] In some alternative embodiments, the second processing unit 532 is specifically configured to:

[0230] Based on the type attribute of each point in the target fusion point cloud data, determine the third object point cloud of each object; based on the third object point cloud of each object and the first sub-point cloud, determine the target object point cloud distributed in the first sub-point cloud and the complete state of the object corresponding to the target object point cloud; for each target object point cloud, in response to the complete state of the object corresponding to the target object point cloud being complete, determine the transparency attribute of each point in the target object point cloud as a preset transparency value; or, in response to the complete state of the object corresponding to the target object point cloud being incomplete, obtain the object sub-point cloud that belongs to the same object as the target object point cloud and is not distributed in the first sub-point cloud;

[0231] Perform a transparency gradient transition process on the target object point cloud and the object sub-point cloud to obtain the transparency attribute of each point in the target object point cloud and the object sub-point cloud.

[0232] In some alternative embodiments, the first processing unit 531 is specifically configured to:

[0233] For the point cloud data at each second moment, based on the type attribute of each point in the point cloud data at the second moment, determine the static point cloud data at the second moment; based on the static point cloud data at each second moment and the third point cloud data, determine the target fusion point cloud data.

[0234] In some alternative embodiments, the first processing unit 531 is specifically configured to:

[0235] For the point cloud data at each second moment, based on the type attributes of each point in the point cloud data at the second moment, determine the static point cloud data at the second moment; based on the static point cloud data at each second moment and the second point cloud data, determine the fused point cloud data.

[0236] In some optional embodiments, based on any of the above embodiments, the second processing module 53 may further be configured to: cache at least one of the following point cloud data: the first point cloud data at the first moment, the second point cloud data, the third point cloud data, the target fused point cloud data, and the point cloud data of the driving environment of the vehicle at the first moment.

[0237] Each of the above embodiments of the present disclosure may be implemented alone or in any combination without conflict, and may be specifically set according to actual needs. The present disclosure does not make any limitations.

[0238] For the beneficial technical effects corresponding to the exemplary embodiments of the present device, reference may be made to the corresponding beneficial technical effects in the above exemplary method section, and details will not be elaborated herein.

[0239] Exemplary electronic device

[0240] Figure 12 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure, including at least one processor 91 and a memory 92.

[0241] The processor 91 may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions.

[0242] The memory 92 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage media, and the processor 91 may run one or more computer program instructions to implement the methods of the various embodiments of the present disclosure above and / or other desired functions.

[0243] In one example, the electronic device 90 may further include: an input device 93 and an output device 94, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0244] The input device 93 may further include, for example, a touch screen, a microphone, various sensors, and so on. The sensors may include, for example, an image sensor (such as a camera), lidar, millimeter-wave radar, ultrasonic radar, a positioning sensor, a pressure sensor, an air quality sensor, a temperature sensor, etc. The image sensor, lidar, millimeter-wave radar, ultrasonic radar, etc. can be used for perceiving the surrounding environment, that is, detecting dynamic and static objects in the surrounding environment. The dynamic and static objects may include, for example, static objects such as lane lines, curbs, arrows, road signs, trees, buildings, etc., and dynamic objects such as surrounding vehicles, pedestrians, cyclists, etc. The positioning sensor is used to implement the positioning of the movable device where the electronic device is located (such as a vehicle, a robot, etc.). The positioning sensor may include, for example, an Inertial Measurement Unit (IMU), a Global Positioning System (GPS), etc. The pressure sensor can be used to detect the seat pressure. The temperature sensor can be used to detect the temperature inside the vehicle cockpit. The air quality sensor can be used to detect the air quality inside the vehicle cockpit.

[0245] The output device 94 can output various information to the outside, which may include, for example, a display, a speaker, a communication network, and remote output devices connected thereto, and so on.

[0246] Of course, for simplicity, Figure 12 only some of the components related to the present disclosure in the electronic device 90 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application scenarios, the electronic device 90 may further include any other appropriate components.

[0247] Exemplary computer program product and computer-readable storage medium

[0248] In addition to the above methods and devices, embodiments of the present disclosure may further provide a computer program product, including computer program instructions, which when run by a processor cause the processor to execute the steps in the methods of various embodiments of the present disclosure described in the above "Exemplary Method" section.

[0249] The computer program product can be written in any combination of one or more programming languages for programming code to perform the operations of the embodiments of the present disclosure. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0250] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods of various embodiments of the present disclosure described in the above "Exemplary Method" section.

[0251] The computer-readable storage medium may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium includes, for example but not limited to, a system, device, or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0252] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that they are essential for each embodiment of the present disclosure. In addition, the above-described specific details are only for the purposes of illustration and facilitating understanding, rather than limitations. The above details do not limit the present disclosure to necessarily adopt the above specific details for implementation.

[0253] Those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present disclosure and their equivalent technologies, the present disclosure also intends to include these modifications and variations.

Claims

1. A method for displaying a vehicle driving environment, comprising: Obtain the first point cloud data at the first moment; Classify the first point cloud data to obtain second point cloud data containing type attributes of each point; Determining point cloud data of a driving environment of the vehicle at a first moment based on the second point cloud data and at least one point cloud data at a second moment; The second moment is a moment before the first moment; The driving environment at the first moment is displayed based on the point cloud data of the driving environment.

2. The method according to claim 1, wherein: The step of determining the point cloud data of the driving environment of the vehicle at the first moment based on the second point cloud data and at least one point cloud data at the second moment includes: Determine the sparse target fused point cloud data based on the second point cloud data and the at least one point cloud data at the second moment; Based on the target fused point cloud data, point cloud data of the driving environment of the vehicle at the first moment is determined.

3. The method according to claim 2, wherein: The step of determining the sparse target fused point cloud data based on the second point cloud data and the at least one point cloud data at the second moment includes: Based on the type attribute of each point in the second point cloud data, thinning the second point cloud data to obtain third point cloud data; Determine the target fused point cloud data based on the third point cloud data and the at least one point cloud data at the second moment; or, Determine fused point cloud data based on the second point cloud data and the at least one point cloud data at the second moment; Based on the type attribute of each point in the fused point cloud data, the fused point cloud data is thinned to obtain the target fused point cloud data.

4. The method according to claim 3, wherein: The step of thinning the second point cloud data based on the type attribute of each point in the second point cloud data to obtain the third point cloud data includes: Based on the type attribute of each point in the second point cloud data, determining the first sampling density corresponding to each type attribute; and / or, Based on the current position of the vehicle, determining a viewing angle range of a virtual camera for observing a driving environment; Based on the viewing angle range, determining a second sampling density of the second point cloud data; Based on the first sampling density and / or the second sampling density, the second point cloud data is sampled to obtain the third point cloud data.

5. The method according to claim 4, wherein: The determining, based on the viewing angle range, a second sampling density of the second point cloud data includes: Based on the viewing angle range, determining at least one level of sub-point cloud data from the second point cloud data; The second sampling density corresponding to each sub-point cloud data is determined according to the level of each sub-point cloud data.

6. The method according to claim 3, wherein: The step of thinning the second point cloud data based on the type attribute of each point in the second point cloud data to obtain the third point cloud data includes: Based on the type attribute of each point in the second point cloud data, determining the first object point cloud corresponding to each type attribute; Determine a first object contour point cloud and a first object interior point cloud according to the first object point cloud; Sampling the first object contour point cloud according to a third sampling density corresponding to the first object contour point cloud to obtain a second object contour point cloud; According to the distance between each point in the first object internal point cloud and the first object contour point cloud, sampling the first object internal point cloud according to the density gradient rule to obtain a second object internal point cloud; Determine a thinned second object point cloud corresponding to the first object point cloud based on the second object contour point cloud and the second object interior point cloud; Based on the second object point cloud, the third point cloud data is determined.

7. The method according to any one of claims 2 to 6, wherein: The step of determining the point cloud data of the driving environment of the vehicle at the first moment based on the target fused point cloud data includes: Determine the state attribute of each point based on the type attribute of each point in the target fused point cloud data; the state attribute includes at least one of a size attribute, a color attribute and a transparency attribute; Based on the state attributes of each point in the target fused point cloud data, point cloud data of the driving environment at the first moment is determined.

8. The method according to claim 7, wherein: The determining of the state attribute of each point based on the type attribute of each point in the target fused point cloud data includes: Based on the type attribute of each point in the target fused point cloud data, determine the point sets corresponding to each type; For each point set corresponding to each type, based on the distance between each point in the point set and the vehicle and the color system corresponding to the type, the point set is subjected to color gradient processing to determine the color attributes corresponding to each point in the point set; based on the color attributes corresponding to each point in each point set, the color attributes of each point in the target fused point cloud data are obtained; and / or, Determine the height of each point in the target fused point cloud data relative to a preset plane; perform transparency gradient processing on the target fused point cloud data based on the height of each point relative to the preset plane to obtain a transparency attribute corresponding to each point; and / or, Determine a first position of a virtual camera for observing a driving environment; perform a size gradient process on each point based on the distance between each point in the target fused point cloud data and the first position, and obtain a size attribute corresponding to each point; The state attribute of each point is determined based on at least one of a color attribute, a transparency attribute, and a size attribute of each point in the target fused point cloud data.

9. The method according to claim 8, wherein: Also includes: Determining the viewing angle range of the virtual camera; Based on the viewing angle range, determining a first sub-point cloud covering a road surface from the target fused point cloud data; According to a preset transparency rule, a transparency attribute of each point in the first sub-point cloud is determined.

10. The method according to claim 9, wherein: Determining the transparency attribute of each point in the first sub-point cloud according to a preset transparency rule includes: Determining a third object point cloud of each object based on the type attribute of each point in the target fused point cloud data; Based on the third object point cloud and the first sub-point cloud of each object, determining a target object point cloud distributed in the first sub-point cloud and an object integrity state corresponding to the target object point cloud; For each of the target object point clouds, in response to the object integrity state corresponding to the target object point cloud being intact, determining the transparency attribute of each point in the target object point cloud to be a preset transparency value; or, In response to the object completeness status corresponding to the target object point cloud being incomplete, obtaining an object sub-point cloud that belongs to the same object as the target object point cloud and is not distributed in the first sub-point cloud; The target object point cloud and the object sub-point cloud are subjected to transparency gradient transition processing to obtain transparency attributes of each point in the target object point cloud and the object sub-point cloud.

11. The method according to any one of claims 3 to 6, wherein: The determining the target fused point cloud data based on the third point cloud data and the at least one point cloud data at the second moment includes: For each point cloud data at the second moment, based on the type attribute of each point in the point cloud data at the second moment, determine the static point cloud data at the second moment; Determine the target fused point cloud data based on the static point cloud data at each of the second moments and the third point cloud data; or, The determining of fused point cloud data based on the second point cloud data and the at least one point cloud data at the second moment includes: For each point cloud data at the second moment, based on the type attribute of each point in the point cloud data at the second moment, determine the static point cloud data at the second moment; The fused point cloud data is determined based on the static point cloud data at each of the second moments and the second point cloud data.

12. A display device for a vehicle driving environment, comprising: An acquisition module, used for acquiring first point cloud data at a first moment; A first processing module, used for classifying the first point cloud data to obtain second point cloud data containing type attributes of each point; A second processing module, configured to determine point cloud data of a driving environment of the vehicle at a first moment based on the second point cloud data and at least one point cloud data at a second moment; The second moment is a moment before the first moment; The third processing module is used to display the driving environment at the first moment based on the point cloud data of the driving environment.

13. A computer-readable storage medium, wherein the storage medium stores a computer program, wherein the computer program is used to execute the method according to any one of claims 1 to 11.

14. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-11.

Images