Environment display method and device in vehicle driving process, medium and electronic equipment

Abstract

The embodiment of the invention discloses an environment display method and device in the vehicle driving process, a medium and electronic equipment. The method comprises the steps that an image, collected by a sensor of a vehicle, of the surrounding environment of the vehicle is acquired; based on the image, attribute information corresponding to each voxel used for representing the surrounding environment of the vehicle is determined; based on the attribute information corresponding to each voxel, virtual point data corresponding to each voxel is determined; generating a virtual environment corresponding to the surrounding environment of the vehicle based on a virtual point cloud formed by the virtual point data corresponding to each voxel; and displaying the virtual environment through the display equipment. According to the embodiment of the invention, on the premise that the cost of the sensor is not increased, a more refined sensing result can be obtained, and the sensing result can be effectively presented to the user, so that the user can know the surrounding environment of the vehicle through visual, vivid and aesthetic picture display, and the use experience of the user can be improved.

Description

Technical Field

[0001] The present disclosure relates to driving technologies, and in particular, to a method, device, medium, and electronic device for displaying the environment during vehicle driving. Background Art

[0002] Vehicles are an important means of transportation, which can provide great convenience for users' travel. During the driving process of a vehicle, the surrounding environment of the vehicle can be perceived based on the data collected by sensors of the vehicle (which are generally low-cost sensors, such as cameras), and the perception results can be presented to the user.

[0003] In the related art, the perception results often can only roughly reflect the surrounding environment of the vehicle, and the degree of refinement is insufficient, which will reduce the user experience. How to obtain more refined perception results and effectively present the perception results to the user without increasing the sensor cost is a technical problem worthy of attention for those skilled in the art. Summary of the Invention

[0004] To solve the above technical problems, the present disclosure provides a method, device, medium, and electronic device for displaying the environment during vehicle driving.

[0005] According to one aspect of the embodiments of the present disclosure, there is provided a method for displaying the environment during vehicle driving, including:

[0006] Obtaining an image of the surrounding environment of the vehicle collected by a sensor of the vehicle;

[0007] Based on the image, determining attribute information corresponding to each voxel for characterizing the surrounding environment of the vehicle;

[0008] Based on the attribute information corresponding to each voxel, determining virtual point data corresponding to each voxel;

[0009] Based on the virtual point cloud formed by the virtual point data corresponding to each voxel, generating a virtual environment corresponding to the surrounding environment of the vehicle;

[0010] Displaying the virtual environment through a display device.

[0011] According to another aspect of the embodiments of the present disclosure, there is provided a device for displaying the environment during vehicle driving, including:

[0012] An obtaining module, configured to obtain an image of the surrounding environment of the vehicle collected by a sensor of the vehicle;

[0013] A first determination module, configured to determine, based on the image obtained by the obtaining module, attribute information corresponding to each voxel for characterizing the surrounding environment of the vehicle;

[0014] A second determination module, configured to determine virtual point data corresponding to each voxel based on the attribute information corresponding to each voxel determined by the first determination module;

[0015] A generation module, configured to generate a virtual environment corresponding to the vehicle surrounding environment based on the virtual point cloud formed by the virtual point data corresponding to each voxel determined by the second determination module;

[0016] A display module, configured to display the virtual environment generated by the generation module through a display device.

[0017] According to another aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing a computer program for executing the environment display method during vehicle driving as described above.

[0018] According to another aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0019] A processor;

[0020] A memory for storing executable instructions of the processor;

[0021] The processor is configured to read the executable instructions from the memory and execute the instructions to implement the environment display method during vehicle driving as described above.

[0022] According to another aspect of the embodiments of the present disclosure, there is provided a computer program product, when the instructions in the computer program product are executed by a processor, the environment display method during vehicle driving as described above is executed.

[0023] Based on the environmental display method, device, medium, electronic device, and program product provided in the above embodiments of the present disclosure, during the driving of a vehicle, images of the vehicle's surrounding environment can be collected through the vehicle's sensors, and based on the images, the attribute information corresponding to each voxel representing the vehicle's surrounding environment can be determined, thus obtaining a voxelized representation of the vehicle's surrounding environment. It should be noted that the voxelized representation of the vehicle's surrounding environment can carry the perception results of the vehicle's surrounding environment. For example, the voxelized representation of the vehicle's surrounding environment can reflect the objects existing around the vehicle, the object categories of these objects, the distribution positions of these objects, etc. Based on the attribute information corresponding to each voxel, the virtual point data corresponding to each voxel can be determined, so as to obtain a virtual point cloud formed by the virtual point data corresponding to each voxel, thus converting the voxelized representation of the vehicle's surrounding environment into a point cloud representation. According to the point cloud representation of the vehicle's surrounding environment, a virtual environment corresponding to the vehicle's surrounding environment can be generated, so as to display the virtual environment through a display device. It should be noted that similar to the voxelized representation of the vehicle's surrounding environment, the point cloud representation of the vehicle's surrounding environment can also carry the perception results of the vehicle's surrounding environment, and compared with the voxelized representation of the vehicle's surrounding environment, the point cloud representation of the vehicle's surrounding environment is more refined in visual effect, is more conducive to reflecting the shape of the object, and can facilitate the user's observation. Since the point cloud representation of the vehicle's surrounding environment is used for the generation of the virtual environment, through the display of the virtual environment, the refined perception results of the vehicle's surrounding environment can be presented to the user. Therefore, in the embodiments of the present disclosure, based on a relatively low-cost visual sensor (such as a camera), by obtaining the voxelized representation of the vehicle's surrounding environment, combined with the conversion from the voxelized representation to the point cloud representation and the simulated display of the environment, a more refined perception result can be obtained without increasing the sensor cost, and the perception result can be effectively presented to the user. In this way, through an intuitive, vivid, and aesthetically pleasing screen display, the user can understand the vehicle's surrounding environment, which is beneficial to improving the user's experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flowchart of an environmental display method during vehicle driving provided by some exemplary embodiments of the present disclosure.

[0025] Figure 2 is a flowchart of a method for determining the virtual point data corresponding to each voxel based on the attribute information corresponding to each voxel provided by some exemplary embodiments of the present disclosure.

[0026] Figure 3 is a flowchart of a method for generating a virtual environment corresponding to the vehicle's surrounding environment based on a virtual point cloud formed by the virtual point data corresponding to each voxel provided by some exemplary embodiments of the present disclosure.

[0027] Figure 4 It is a schematic flowchart of a method for determining display parameters corresponding to each virtual point data in a virtual point cloud provided by some exemplary embodiments of the present disclosure.

[0028] Figure 5 It is a schematic flowchart of a method for determining display parameters corresponding to each virtual point data in a virtual point cloud provided by some other exemplary embodiments of the present disclosure.

[0029] Figure 6 It is a schematic flowchart of a method for determining display parameters corresponding to each virtual point data in a virtual point cloud provided by still some other exemplary embodiments of the present disclosure.

[0030] Figure 7 It is a schematic flowchart of a method for generating a virtual environment corresponding to the vehicle surrounding environment based on a virtual point cloud formed by virtual point data respectively corresponding to each voxel provided by some other exemplary embodiments of the present disclosure.

[0031] Figure 8 It is a schematic flowchart of a method for updating a virtual point cloud based on a first virtual sub-point cloud, orientation information, and relative position relationship provided by some exemplary embodiments of the present disclosure.

[0032] Figure 9 It is a schematic diagram of the principle for obtaining a second virtual sub-point cloud in some exemplary embodiments of the present disclosure.

[0033] Figure 10-1 It is a schematic flowchart of a method for generating a virtual environment corresponding to the vehicle surrounding environment based on a virtual point cloud formed by virtual point data respectively corresponding to each voxel provided by still some other exemplary embodiments of the present disclosure.

[0034] Figure 10-2 It is a schematic flowchart of a method for determining densification parameters adapted to respective multiple virtual sub-point clouds provided by some exemplary embodiments of the present disclosure.

[0035] Figure 11 It is a schematic diagram of the structure of an environment display device during vehicle driving provided by some exemplary embodiments of the present disclosure.

[0036] Figure 12 It is a schematic diagram of the structure of a second determination module in some exemplary embodiments of the present disclosure.

[0037] Figure 13 It is a schematic diagram of the structure of a generation module in some exemplary embodiments of the present disclosure.

[0038] Figure 14 It is a schematic diagram of the structure of a generation module in some other exemplary embodiments of the present disclosure.

[0039] Figure 15It is a schematic structural diagram of a generation module in some other exemplary embodiments of the present disclosure.

[0040] Figure 16 It is a schematic structural diagram of an electronic device provided in some exemplary embodiments of the present disclosure. Detailed implementation manners

[0041] To explain the present disclosure, exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. It should be understood that the present disclosure is not limited by the exemplary embodiments.

[0042] It should be noted that: unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions and values set forth in these embodiments do not limit the scope of the present disclosure.

[0043] Overview of the Application

[0044] During the driving process of the vehicle (which may also be referred to as the vehicle hereinafter), the surrounding environment of the vehicle can be perceived based on the data collected by the sensors of the vehicle to obtain a perception result. For example, based on the images collected by the vehicle's camera, static objects, dynamic objects, etc. in the surrounding environment of the vehicle can be perceived. Static objects may include, for example, buildings, traffic lights, etc. Dynamic objects may include, for example, pedestrians, cyclists, other vehicles (vehicles other than the vehicle, which may also be referred to as other vehicles hereinafter), etc.

[0045] In the related art, the perception result often can only roughly reflect the surrounding environment of the vehicle, and the degree of refinement is insufficient, which will reduce the user experience. How to obtain a more refined perception result and effectively present the perception result to the user without increasing the cost of sensors is a technical problem worthy of attention for those skilled in the art.

[0046] Exemplary System

[0047] It should be noted that the "vehicle" without additional attributives hereinafter refers to the vehicle itself.

[0048] In the embodiments of the present disclosure, the camera of the vehicle can collect images of the surrounding environment of the vehicle. Based on the images of the surrounding environment of the vehicle, a voxelized expression of the surrounding environment of the vehicle can be generated. Next, the voxelized expression of the surrounding environment of the vehicle can be converted into a point cloud expression. Based on the point cloud expression of the surrounding environment of the vehicle, a virtual environment corresponding to the surrounding environment of the vehicle can be displayed. It should be noted that the virtual environment is a simulation of the surrounding environment of the vehicle. By displaying the virtual environment, the perception result of the surrounding environment of the vehicle can be presented to the user. In this way, the surrounding environment of the vehicle can be presented to the user through an intuitive, vivid and aesthetically pleasing display.

[0049] Exemplary Method

[0050] Figure 1 It is a schematic flow chart of an environmental display method during vehicle driving provided by some exemplary embodiments of the present disclosure. Figure 1 The method shown may include step 110, step 120, step 130, step 140, and step 150.

[0051] Step 110, obtain an image of the vehicle's surrounding environment collected by the vehicle's sensors.

[0052] Optionally, the vehicle's sensors may include the vehicle's cameras, and the number of the vehicle's cameras may be one or more than one. The vehicle's cameras may collect images of the vehicle's surrounding environment regularly or irregularly to obtain an image of the vehicle's surrounding environment.

[0053] Step 120, based on the image, determine the attribute information corresponding to each voxel used to characterize the vehicle's surrounding environment.

[0054] Optionally, based on the image, a three-dimensional reconstruction algorithm may be used to perform three-dimensional reconstruction on the vehicle's surrounding environment. If the number of the vehicle's cameras is one, the image collected by this camera may be used as the input data of the three-dimensional reconstruction algorithm. If the number of the vehicle's cameras is multiple, the images collected by the multiple cameras may be stitched together to obtain a panoramic image, and the panoramic image may be used as the input data of the three-dimensional reconstruction algorithm. After completing the three-dimensional reconstruction of the vehicle's surrounding environment, a voxel occupancy network (OCC) may be used to divide the reconstructed vehicle's surrounding environment into multiple cubes (equivalent to voxelization) respectively representing the corresponding objects in the vehicle's surrounding environment. The cube may also be referred to as a volume element or a voxel. On this basis, the position of each voxel in space may be determined to obtain the spatial position information corresponding to each voxel. Among them, the spatial position information corresponding to the voxel may include, but is not limited to, the position coordinates of the centroid of the voxel, the position coordinates of each of the eight corner points of the voxel, etc. Here, the vehicle may have a corresponding ego-vehicle coordinate system. The ego-vehicle coordinate system may be, for example, a three-dimensional coordinate system constructed with the centroid of the vehicle, the center of the chassis, etc. as the origin. The position coordinates of the centroid of the voxel may be the three-dimensional position coordinates of the voxel in the ego-vehicle coordinate system, and the position coordinates of the corner points of the voxel may be the three-dimensional position coordinates of the corner points of the voxel in the ego-vehicle coordinate system.

[0055] Optionally, based on the image, a neural network algorithm can be used to determine the semantic category, risk level, etc. of each of multiple objects in the vehicle's surrounding environment. For example, a first neural network for identifying semantic categories can be pre-trained. The image can be used as the input data for the first neural network, and the first neural network can perform operations based on the input data to determine the semantic category of each of the multiple objects in the vehicle's surrounding environment. For another example, a second neural network for identifying risk levels can be pre-trained. The image or an image sequence composed of images corresponding to multiple moments can be used as the input data for the second neural network, and the second neural network can perform operations based on the input data to determine the risk level of each of the multiple objects in the vehicle's surrounding environment.

[0056] It should be noted that in the embodiments of the present disclosure, the semantic category can also be referred to as the object category, which refers to the classification label of the objects in the vehicle's surrounding environment. The object category can include but is not limited to building category, traffic light category, overpass category, tree category, pedestrian category, cyclist category, vehicle category, cone category, stone category, etc. The risk level is used to characterize the risk degree brought to the vehicle. The higher the risk level, the higher the risk degree brought to the vehicle, and the lower the risk level, the lower the risk degree brought to the vehicle.

[0057] It should be noted that voxels can be used to represent the vehicle's surrounding environment. For multiple objects in the vehicle's surrounding environment, each object can be represented by some voxels in the voxels. Then, the semantic category of the object can be assigned to each voxel in this part of the voxels, and the risk level of the object can be assigned to each voxel in this part of the voxels. In this way, the semantic category and the corresponding risk level corresponding to each voxel can be determined. Then, combined with the spatial position information corresponding to each voxel, the attribute information corresponding to each voxel can be determined. Here, the attribute information corresponding to the voxel can include the spatial position information corresponding to the voxel and the semantic category corresponding to the voxel. In some embodiments, the attribute information corresponding to the voxel can further include the risk level corresponding to the voxel.

[0058] Step 130, based on the attribute information corresponding to each voxel, determine the virtual point data corresponding to each voxel.

[0059] Optionally, the attribute information corresponding to each voxel can jointly form a voxelized representation of the vehicle's surrounding environment. For each voxel, based on the attribute information corresponding to the voxel, the virtual point data corresponding to the voxel can be determined. The virtual point data corresponding to the voxel can correspond to a virtual point, that is, the voxel can correspond to a virtual point. The virtual point corresponding to the voxel can be understood as: when converting the voxelized representation of the vehicle's surrounding environment to a point cloud representation later, the virtual point used to replace the voxel. The virtual point data corresponding to the voxel can be data for describing the attributes of the virtual point corresponding to the voxel, including but not limited to the semantic category of the virtual point corresponding to the voxel, the virtual point coordinates of the virtual point corresponding to the voxel, the risk level of the virtual point corresponding to the voxel, etc. Here, the virtual point coordinates of the virtual point can be the three-dimensional position coordinates of the virtual point in the vehicle's own coordinate system.

[0060] Step 140, generate a virtual environment corresponding to the vehicle's surrounding environment based on the virtual point cloud formed by the virtual point data corresponding to each voxel.

[0061] Optionally, the virtual point data corresponding to each voxel can jointly form a virtual point cloud, and the virtual point cloud can be regarded as a point cloud representation converted from the voxelized representation of the vehicle's surrounding environment.

[0062] Optionally, after obtaining the virtual point cloud, a rendering engine can be used to render the virtual point cloud to generate a frame to be displayed; alternatively, some update methods can be used to update the virtual point cloud first, and then the rendering engine can be used to render the updated virtual point cloud to generate a frame to be displayed. The display content of the frame to be displayed can be used as the virtual environment corresponding to the vehicle's surrounding environment. Through the virtual environment, the virtual point cloud or the updated virtual point cloud can be presented. In some embodiments, through the virtual environment, a virtual road surface (used to simulate the road surface where the vehicle is located), a virtual self-vehicle (used to simulate the self-vehicle), etc. can also be presented.

[0063] It should be noted that the relationship between the vehicle's surrounding environment and the virtual environment can be understood as: the vehicle's surrounding environment is the real environment around the vehicle in the physical world, and the virtual environment is not the real environment around the vehicle in the physical world, but a simulation of the vehicle's surrounding environment. By observing the virtual environment, the vehicle's surrounding environment can be understood.

[0064] Step 150, display the virtual environment through a display device.

[0065] Optionally, the display device can include but not limited to the vehicle's central control screen, instrument screen, etc. Through the display device, the frame to be displayed generated by the rendering engine can be displayed. Since the display content of the frame to be displayed can be used as the virtual environment corresponding to the vehicle's surrounding environment, displaying the frame to be displayed through the display device is equivalent to displaying the virtual environment through the display device.

[0066] In an embodiment of the present disclosure, during the driving process of a vehicle, images of the vehicle surrounding environment can be collected by sensors of the vehicle, and based on the images, attribute information corresponding to each voxel for characterizing the vehicle surrounding environment can be determined, thus obtaining a voxelized representation of the vehicle surrounding environment. It should be noted that the voxelized representation of the vehicle surrounding environment can carry the perception results of the vehicle surrounding environment. For example, the voxelized representation of the vehicle surrounding environment can reflect the objects existing around the vehicle, the object categories of these objects, the distribution positions of these objects, etc. Based on the attribute information corresponding to each voxel, virtual point data corresponding to each voxel can be determined, so as to obtain a virtual point cloud formed by the virtual point data corresponding to each voxel, thus converting the voxelized representation of the vehicle surrounding environment into a point cloud representation. According to the point cloud representation of the vehicle surrounding environment, a virtual environment corresponding to the vehicle surrounding environment can be generated, so as to display the virtual environment through a display device. It should be noted that similar to the voxelized representation of the vehicle surrounding environment, the point cloud representation of the vehicle surrounding environment can also carry the perception results of the vehicle surrounding environment, and compared with the voxelized representation of the vehicle surrounding environment, the point cloud representation of the vehicle surrounding environment is more refined in visual effect, is more conducive to reflecting the shape of the object, and can facilitate the user's observation. Since the point cloud representation of the vehicle surrounding environment is used for the generation of the virtual environment, through the display of the virtual environment, the refined perception results of the vehicle surrounding environment can be presented to the user. Therefore, in the embodiment of the present disclosure, based on a relatively low-cost visual sensor (such as a camera), by obtaining the voxelized representation of the vehicle surrounding environment, combined with the conversion from the voxelized representation to the point cloud representation and the simulated display of the environment, a more refined perception result can be obtained without increasing the sensor cost, and the perception result can be effectively presented to the user. In this way, through an intuitive, vivid and aesthetically pleasing picture display, the user can understand the vehicle surrounding environment, which is beneficial to improving the user experience.

[0067] Figure 2 It is a flowchart showing a method for determining virtual point data corresponding to each voxel based on the attribute information corresponding to each voxel provided by some exemplary embodiments of the present disclosure. Figure 2 The method shown may include step 210, step 220, and step 230. Optionally, the combination of step 210 to step 230 may be an alternative implementation of step 130 of the present disclosure.

[0068] Step 210, obtain the semantic category and the corresponding spatial position information corresponding to each voxel from the attribute information corresponding to each voxel.

[0069] As introduced above, for each voxel, the attribute information corresponding to the voxel may include the spatial position information corresponding to the voxel and the semantic category corresponding to the voxel. Then, from the attribute information corresponding to the voxel, the semantic category corresponding to the voxel and the spatial position information corresponding to the voxel can be extracted.

[0070] Step 220: Based on the spatial position information respectively corresponding to each voxel, determine the virtual point coordinates respectively corresponding to each voxel.

[0071] Optionally, for each voxel, if the spatial position information corresponding to the voxel includes the position coordinates of the centroid of the voxel, the position coordinates of the centroid of the voxel can be determined as the virtual point coordinates corresponding to the voxel. If the spatial position information corresponding to the voxel includes the position coordinates of each of the eight corner points of the voxel, geometric calculations can be performed based on the position coordinates of each of the eight corner points of the voxel to determine the position coordinates of the centroid of the voxel, and the position coordinates of the centroid of the voxel can be determined as the virtual point coordinates corresponding to the voxel.

[0072] Step 230: Integrate the semantic category respectively corresponding to each voxel and the virtual point coordinates respectively corresponding to each voxel to obtain the virtual point data respectively corresponding to each voxel.

[0073] Optionally, for each voxel, the semantic category corresponding to the voxel and the virtual point coordinates corresponding to the voxel can be integrated to obtain the virtual point data corresponding to the voxel. In this way, the virtual point data corresponding to the voxel can include the semantic category and the virtual point coordinates.

[0074] In an example, the semantic category corresponding to a certain voxel is the tree category, and the virtual point coordinates corresponding to the voxel are three-dimensional position coordinates (x, y, z). Then, the virtual point data corresponding to the voxel can include the tree category and the three-dimensional position coordinates (x, y, z).

[0075] In the embodiments of the present disclosure, for each voxel, by assigning the semantic category corresponding to the voxel to the virtual point corresponding to the voxel and assigning the position coordinates of the centroid of the voxel to the virtual point corresponding to the voxel, the virtual point data corresponding to the voxel can be determined efficiently and quickly. On this basis, the voxelized expression of the vehicle surrounding environment can be converted into a point cloud expression. Since when converting the voxelized expression of the vehicle surrounding environment to a point cloud expression, the semantic category corresponding to the virtual point and the corresponding virtual point coordinates are respectively inherited from the semantic category corresponding to the voxel and the corresponding position coordinates of the centroid, it can better ensure the consistency of the perception results carried by the point cloud expression and the voxelized expression. In addition, compared with the voxelized expression, the point cloud expression is more refined in visual effect, is more conducive to reflecting the shape of the object, and is convenient for users to observe.

[0076] In some embodiments, for each voxel, the attribute information corresponding to the voxel may further include the risk level corresponding to the voxel. The risk level corresponding to the voxel may be assigned to the virtual point corresponding to the voxel, so that the virtual point data corresponding to the voxel includes not only the semantic category and the virtual point coordinates, but also the risk level.

[0077] In one example, the semantic category corresponding to a certain voxel is the vehicle category. And based on the analysis of the images collected by the vehicle's camera, it is determined that the vehicle represented by the voxel has abnormal behaviors such as swaying left and right and frequently crossing the line. Then the risk level corresponding to the voxel may be a major risk level, which is used to represent a very high degree of risk. The risk level in the virtual point data corresponding to the voxel is also the major risk level. In another example, the semantic category corresponding to a certain voxel is the vehicle category. And based on the analysis of the images collected by the vehicle's camera, it is determined that the vehicle represented by the voxel has no abnormal behaviors. Then the risk level corresponding to the voxel may be a general risk level, which is used to represent a relatively low degree of risk. The risk level in the virtual point data corresponding to the voxel is also the general risk level. In still another example, the semantic category corresponding to a certain voxel is the traffic light category or the overpass category. Since traffic lights and overpasses are stationary in the physical world and generally do not pose risks to vehicles, the risk level corresponding to the voxel may be a low risk level, which is used to represent a very low degree of risk. The risk level in the virtual point data corresponding to the voxel is also the low risk level.

[0078] It should be noted that in the case where the risk level is introduced into both the attribute information corresponding to the voxel and the virtual point data corresponding to the voxel, in the embodiments of the present disclosure, both the voxelized expression and the point cloud expression of the vehicle surrounding environment can carry the perception and prediction results of the vehicle surrounding environment, wherein the prediction results are specifically reflected by the risk level.

[0079] Figure 3 It is a schematic flowchart of a method for generating a virtual environment corresponding to a vehicle surrounding environment based on a virtual point cloud formed by virtual point data respectively corresponding to each voxel provided by some exemplary embodiments of the present disclosure. Figure 3 The method shown may include step 310, step 320, and step 330. Optionally, the combination of step 310 to step 330 may be an alternative implementation of step 140 of the present disclosure.

[0080] Step 310, determine the display parameters corresponding to each virtual point data in the virtual point cloud.

[0081] Optionally, the display parameters corresponding to each virtual point data in the virtual point cloud may include, but are not limited to, display color, display transparency, display size, etc. The display color refers to the color presented during display, including but not limited to red, yellow, blue, purple, etc. The display transparency refers to the transparency presented during display, which may be in the form of a percentage (specifically between 0% and 100%). The display size refers to the size presented during display, for example, it may include the radius. The display parameters corresponding to each virtual point data in the virtual point cloud can be determined based on the virtual point data. The specific determination methods are diverse and will be exemplified later, so no further elaboration will be made here.

[0082] Step 320: Add corresponding display parameters to each virtual point data in the virtual point cloud to update the virtual point cloud.

[0083] Optionally, for each virtual point data in the virtual point cloud, the display parameters corresponding to the virtual point data can be added to the virtual point data. Then, before performing the addition operation, the virtual point data may only include the semantic category and the virtual point coordinates. After performing the addition operation, the virtual point data may include the semantic category, the virtual point coordinates, and the display parameters. Or, before performing the addition operation, the virtual point data may include the semantic category, the virtual point coordinates, and the risk level. After performing the addition operation, the virtual point data may include the semantic category, the virtual point coordinates, the risk level, and the display parameters. In this way, the update of each virtual point data in the virtual point cloud can be achieved, thereby achieving the update of the virtual point cloud.

[0084] Step 330: Generate a virtual environment corresponding to the vehicle's surrounding environment based on the updated virtual point cloud.

[0085] Optionally, a rendering engine can be used to render the updated virtual point cloud to generate a frame to be displayed, and the content of the frame to be displayed can be used as the virtual environment corresponding to the vehicle's surrounding environment. Assume that the updated virtual point cloud includes M virtual point data, then there can be M virtual points in the frame to be displayed that correspond one-to-one to the M virtual point data; among them, each virtual point can be presented in the display color, display transparency, display size, etc. of the virtual point data corresponding to the virtual point.

[0086] In the embodiments of the present disclosure, by adding corresponding display parameters to each virtual point data in the virtual point cloud to update the virtual point cloud and using the updated virtual point cloud for the generation of the virtual environment, when the virtual environment is displayed through a display device, each virtual point can present corresponding colors, corresponding transparencies, corresponding sizes, etc. In this way, from the user's visual perspective, the virtual environment can have a sense of the world, be more vivid and lifelike, which is beneficial to improving the user's experience.

[0087] Figure 4It is a schematic flowchart of a method for determining display parameters corresponding to each virtual point data in a virtual point cloud provided by some exemplary embodiments of the present disclosure. Figure 4 The method shown may include step 410, step 420, and step 430. Optionally, the combination of step 410 to step 430 may be an alternative implementation of step 310 of the present disclosure. Figure 4 In the method shown, each virtual point data in the virtual point cloud includes a virtual point coordinate, a semantic category, and a risk level, and the risk level is used to characterize the degree of risk brought to the vehicle.

[0088] Step 410, for each virtual point data in the virtual point cloud, based on the semantic category in the virtual point data, according to the pre-set correspondence between the semantic category and the display color, determine the target display color, and / or, based on the risk level in the virtual point data, according to the pre-set correspondence between the risk level and the display color, determine the target display color.

[0089] Optionally, for each virtual point data in the virtual point cloud, the semantic category may be extracted from the virtual point data, and according to the pre-set correspondence between the semantic category and the display color, the display color corresponding to the extracted semantic category may be determined, and the determined display color may be used as the target display color corresponding to the virtual point data.

[0090] Optionally, for each virtual point data in the virtual point cloud, the risk level may be extracted from the virtual point data, and according to the pre-set correspondence between the risk level and the display color, the display color corresponding to the extracted risk level may be determined, and the determined display color may be used as the target display color corresponding to the virtual point data.

[0091] In some embodiments, for each virtual point data in the virtual point cloud, the target display color corresponding to the virtual point data may also be determined by combining the semantic category and the corresponding risk level corresponding to the virtual point data. For example, the correspondence between the semantic category, the risk level, and the display color may be pre-set. For each virtual point data, according to the pre-set correspondence, the display color corresponding to both the semantic category and the risk level in the virtual point data may be determined, and the determined display color may be used as the target display color corresponding to the virtual point data.

[0092] Step 420: For each piece of virtual point data in the virtual point cloud, based on the virtual point coordinates in the virtual point data, determine the first distance between the virtual point that conforms to the virtual point data and the vehicle. Based on the first distance, determine the target display transparency according to the pre-set correspondence between the distance range and the display transparency, and / or, based on the virtual point coordinates in the virtual point data, determine the position height of the virtual point that conforms to the virtual point data. Based on the position height, determine the target display transparency according to the pre-set correspondence between the height range and the display transparency.

[0093] Optionally, for each piece of virtual point data in the virtual point cloud, the virtual point that conforms to the virtual point data can also be referred to as the virtual point corresponding to the virtual point data. The virtual point that conforms to the virtual point data can be a spatial point in the vehicle's coordinate system. By performing geometric calculations based on the virtual point coordinates in the virtual point data, the distance between this spatial point and the origin of the vehicle's coordinate system can be obtained, and this distance can be used as the first distance between the virtual point that conforms to the virtual point data and the vehicle. The pre-set correspondence between the distance range and the display transparency can involve multiple distance ranges. By comparing the first distance with the multiple distance ranges, the distance range to which the first distance belongs can be determined. According to the pre-set correspondence, the display transparency corresponding to this distance range can be determined, and the determined display transparency can be used as the target display transparency corresponding to the virtual point data.

[0094] Optionally, in the vehicle's coordinate system, the plane where the road surface where the vehicle is located is located (subsequently referred to as the reference plane) can be determined. For each piece of virtual point data in the virtual point cloud, based on the plane equation of the reference plane and the virtual point coordinates in the virtual point data, through geometric calculations, the vertical distance between the virtual point that conforms to the virtual point data and the road surface can be determined, and this vertical distance can be used as the position height of the virtual point that conforms to the virtual point data. The pre-set correspondence between the height range and the display transparency can involve multiple height ranges. By comparing the position height with the multiple height ranges, the height range to which the position height belongs can be determined. According to the pre-set correspondence, the display transparency corresponding to this height range can be determined, and the determined display transparency can be used as the target display transparency corresponding to the virtual point data.

[0095] In some embodiments, for each piece of virtual point data in the virtual point cloud, the target display transparency corresponding to the virtual point data can be determined by combining the first distance corresponding to the virtual point data and the corresponding position height. For example, the corresponding relationship among the distance range, height range, and display transparency can be preset in advance. For each piece of virtual point data, the distance range to which the first distance corresponding to the virtual point data belongs can be determined, and the height range to which the position height corresponding to the virtual point data belongs can be determined. Then, according to the preset corresponding relationship, the display transparency corresponding to both the distance range and the height range can be determined, and the determined display transparency can be used as the target display transparency corresponding to the virtual point data.

[0096] Step 430: For each piece of virtual point data in the virtual point cloud, integrate the target display color and the corresponding target display transparency corresponding to the virtual point data to obtain the display parameter corresponding to the virtual point data.

[0097] Optionally, for each piece of virtual point data in the virtual point cloud, the display parameter corresponding to the virtual point data may include the target display color and the corresponding target display transparency corresponding to the virtual point data.

[0098] In the embodiments of the present disclosure, for each piece of virtual point data in the virtual point cloud, the target display color can be adaptively determined based on the semantic category and / or the risk level in the virtual point data, and the target display transparency can be adaptively determined based on the first distance between the virtual point conforming to the virtual point data and the vehicle and / or the position height of the virtual point conforming to the virtual point data. In this way, it can provide an effective reference for determining the display parameter corresponding to the virtual point data, which is beneficial to ensuring the rationality of the determined display parameter. On this basis, the virtual environment can have a better sense of the world, being more vivid and lifelike. For example, from the perspective of the user's vision, the color of the virtual point farther away from the virtual self-vehicle can be lighter, the virtual point corresponding to the semantic category of trees can be green, and the virtual point farther away from the virtual road surface can be more transparent, etc.

[0099] Figure 5 It is a schematic flowchart of a method for determining the display parameter corresponding to each piece of virtual point data in the virtual point cloud provided by some other exemplary embodiments of the present disclosure. Figure 5 The method shown may include Step 510, Step 520, Step 530, and Step 540. Optionally, the combination of Step 510 to Step 540 can be used as an alternative implementation of Step 310 of the present disclosure. Figure 5 In the method shown, each piece of virtual point data in the virtual point cloud includes virtual point coordinates and a semantic category.

[0100] Step 510: Based on the semantic categories in each virtual point data in the virtual point cloud, filter out the virtual point data in the virtual point cloud whose represented object is a static object.

[0101] Optionally, multiple objects in the environment where the vehicle is located can be divided into two categories, namely static objects and dynamic objects. If the semantic category in a certain virtual point data is a building category, a traffic light category, a pedestrian overpass category, a tree category, etc., since buildings, traffic lights, pedestrian overpasses, trees, etc. are generally stationary in the physical world, it can be determined that the object represented by this virtual point data is a static object. If the semantic category in a certain virtual point data is a pedestrian category, a cyclist category, a vehicle category, etc., since pedestrians, cyclists, other vehicles, etc. are generally moving in the physical world, it can be determined that the object represented by this virtual point data is a dynamic object. In this way, referring to the semantic categories in each virtual point data in the virtual point cloud, the virtual point data whose represented object is a static object can be efficiently and reliably filtered out from the virtual point cloud.

[0102] Step 520: Based on the virtual point coordinates in the filtered virtual point data, determine the projection covering relationship between the virtual points that match the filtered virtual point data and the road surface where the vehicle is located.

[0103] Optionally, the number of the filtered virtual point data can be multiple. For each virtual point data among the multiple filtered virtual point data, the projection covering relationship between the virtual points that match this virtual point data and the road surface where the vehicle is located can be determined based on the virtual point coordinates in this virtual point data; among them, the projection covering relationship corresponding to any virtual point data can be used to represent whether the projection point will cover the road surface where the vehicle is located when the virtual points that match this virtual point data are orthogonally projected onto the reference plane. Here, based on the virtual point coordinates in this virtual point data and the plane equation of the reference plane, the position coordinates of the projection point can be determined through geometric calculations. By comparing the position coordinates of the projection point with the area range occupied by the road surface where the vehicle is located, it can be determined whether the projection point will cover the road surface where the vehicle is located. If the projection point falls on the sidewalk (not belonging to the road surface where the vehicle is located), the projection point will not cover the road surface where the vehicle is located. If the projection point falls on the road surface where the vehicle is located, the projection point will cover the road surface where the vehicle is located.

[0104] Step 530: For the filtered virtual point data, determine the target display transparency adapted to the projection covering relationship.

[0105] Optionally, for each piece of virtual point data among the multiple pieces of filtered virtual point data, if the projection covering relationship corresponding to the virtual point data is used to represent that the corresponding projection point will cover the road surface where the vehicle is located, a preset transparency (such as 50%, 55%, etc.) can be used as the target display transparency adapted to the projection covering relationship. If the projection covering relationship corresponding to the virtual point data is used to represent that the corresponding projection point will not cover the road surface where the vehicle is located, it can be determined according to the introduction in the embodiment shown in Figure 4 Based on the first distance between the virtual point that conforms to the virtual point data and the vehicle and / or the position height of the virtual point that conforms to the virtual point data, the target display transparency is determined, and the determined target display transparency can be used as the target display transparency adapted to the projection covering relationship.

[0106] Step 540, based on the target display transparency corresponding to the filtered virtual point data, determine the display parameters corresponding to the filtered virtual point data.

[0107] Optionally, for each piece of virtual point data among the multiple pieces of filtered virtual point data, the display parameters corresponding to the virtual point data may include the target display transparency corresponding to the virtual point data. In addition, the display parameters corresponding to the virtual point data may further include a target display color, and the determination method of the target display color may refer to the introduction in the embodiment shown in Figure 4 and will not be elaborated here.

[0108] It should be noted that objects such as cyclists and other vehicles are usually located on the road surface, and these objects are the objects that users need to focus on. If these objects are covered, it will affect the user's observation and reduce the user's experience. In view of this, in the embodiments of the present disclosure, virtual point data whose corresponding projection points may cover the road surface (which can also be considered as virtual point data that may cover objects such as cyclists and other vehicles) can be filtered out from the virtual point cloud, and the target display transparency is adaptively determined for the filtered virtual point data. For example, 50% is used as the target display transparency corresponding to the filtered virtual point data. In this way, when the virtual environment is displayed, the virtual points corresponding to the filtered virtual point data can be semi-transparent and will not affect the user's observation of objects such as cyclists and other vehicles, which is beneficial to improving the user's experience.

[0109] Figure 6 It is a schematic flowchart of a method for determining the display parameters corresponding to each piece of virtual point data in the virtual point cloud provided by some other exemplary embodiments of the present disclosure. Figure 6 The method shown may include step 610 and step 620. Optionally, the combination of step 610 to step 620 may be used as an alternative implementation of step 310 of the present disclosure. Figure 6 In the method shown, each piece of virtual point data in the virtual point cloud includes virtual point coordinates.

[0110] Step 610: For each piece of virtual point data in the virtual point cloud, based on the virtual point coordinates in the virtual point data, determine the second distance between the virtual point that conforms to the virtual point data and the virtual camera set for the display of the virtual environment. Based on the second distance, determine the target display size according to the corresponding relationship between the preset distance range and the display size.

[0111] Optionally, for the display of the virtual environment, the pose of the virtual camera in the ego-vehicle coordinate system can be preset in advance. The virtual camera can be regarded as the front-view camera of the virtual ego-vehicle, and the to-be-displayed image in the above text can be regarded as the image that the virtual camera can capture in this pose. For each piece of virtual point data in the virtual point cloud, based on the virtual point coordinates in the virtual point data and the position in this pose, the second distance between the virtual point that conforms to the virtual point data and the virtual camera can be determined through geometric calculation. The corresponding relationship between the preset distance range and the display size can involve multiple distance ranges. By comparing the second distance with the multiple distance ranges, the distance range to which the second distance belongs can be determined. According to the preset corresponding relationship, the display size corresponding to this distance range can be determined, and the determined display size can be used as the target display size corresponding to the virtual point data.

[0112] Step 620: For each piece of virtual point data in the virtual point cloud, determine the display parameter corresponding to the virtual point data based on the target display size corresponding to the virtual point data.

[0113] Optionally, for each piece of virtual point data in the virtual point cloud, the display parameter corresponding to the virtual point data may include the target display size corresponding to the virtual point data. In addition, the display parameter corresponding to the virtual point data may further include the target display color and / or the target display transparency. The target display color and the target display transparency can be determined, for example, in the manner introduced in the Figure 4 illustrated embodiment, which will not be elaborated here.

[0114] In the embodiments of the present disclosure, for each piece of virtual point data in the virtual point cloud, based on the second distance between the virtual point that conforms to the virtual point data and the virtual camera set for the display of the virtual environment, the target display size can be adaptively determined. In this way, it can provide an effective reference for the determination of the display parameter corresponding to the virtual point data, which is beneficial to ensuring the rationality of the determined display parameter. On this basis, the virtual environment can have a better sense of the world, and be more vivid and lifelike. For example, from the perspective of the user's vision, the virtual points farther away from the virtual ego-vehicle can be smaller.

[0115] Figure 7It is a schematic flowchart of a method for generating a virtual environment corresponding to the vehicle surrounding environment based on a virtual point cloud formed by virtual point data respectively corresponding to each voxel provided by some other exemplary embodiments of the present disclosure. Figure 7 The method shown may include step 710, step 720, step 730, and step 740. Optionally, the combination of step 710 to step 740 may be an alternative implementation of step 140 of the present disclosure.

[0116] Step 710, determine a first virtual sub-point cloud from the virtual point cloud for characterizing a target object in the vehicle surrounding environment.

[0117] Optionally, among multiple objects in the vehicle surrounding environment, there may be at least one dynamic object, and each dynamic object in the at least one dynamic object may be used as a target object. As an example, the target object may be a pedestrian, a cyclist, another vehicle, etc.

[0118] Optionally, each virtual point data in the virtual point cloud may include a semantic category. Based on these semantic categories, the respective virtual point data for characterizing the target object in the vehicle surrounding environment can be determined from the virtual point cloud, and these virtual point data can jointly form the first virtual sub-point cloud.

[0119] Step 720, obtain the orientation information of the target object and the relative position relationship between the vehicle and the target object.

[0120] Optionally, the orientation information of the target object may be characterized by the pose of the target object in the vehicle coordinate system. In one example, the orientation information of the target object may be in the form of a rotation matrix. The relative position relationship between the vehicle and the target object may include but is not limited to the front-back relationship and the left-right relationship between the vehicle and the target object in the vehicle coordinate system. Here, the orientation information of the target object and the relative position relationship between the vehicle and the target object can both be obtained by performing operations on the images collected by the vehicle's camera or by performing operations on the virtual point data in the first virtual sub-point cloud.

[0121] Step 730, update the virtual point cloud based on the first virtual sub-point cloud, the orientation information, and the relative position relationship.

[0122] Optionally, based on the orientation information and the relative position relationship, some new virtual sub-point clouds may be added on the basis of the first virtual sub-point cloud to jointly characterize the target object with the first virtual sub-point cloud, thereby forming an updated virtual point cloud including some new virtual sub-point clouds.

[0123] In some alternative implementations of the present disclosure, as Figure 8 shown, step 730 may include step 810, step 820, and step 830.

[0124] Step 810, in response to the relative position relationship indicating that the vehicle and the target object meet the preset position condition, determine the target plane; wherein, the extending direction of the target plane is consistent with the orientation represented by the orientation information, and the target plane is used to divide the target object into two parts.

[0125] Optionally, the preset position condition may refer to: the two objects are neither aligned front to back nor left to right. If the relative position relationship is used to indicate that the target object is located in the left front, left rear, right front or right rear of the vehicle, since the vehicle and the target object are neither aligned front to back nor left to right, it can be determined that the vehicle and the target object meet the preset position condition. In this case, the target plane can be determined.

[0126] Take Figure 9 as an example, the target object can be another vehicle C1, and the other vehicle C1 is located in the right front of the own vehicle C2, that is, the other vehicle C1 and the own vehicle C2 meet the preset position condition. Assuming that the orientation information of the other vehicle C1 is in the direction shown by the arrow F, and the rear surface S of the target object is within the field of view of the camera of the own vehicle C2, then a plane P parallel to the direction shown by the arrow F and capable of dividing (for example, evenly dividing) the rear surface S into left and right parts can be determined, and the plane P can be used as the target plane.

[0127] Optionally, if the relative position relationship is used to indicate that the target object is located directly to the left, directly to the right, directly in front or directly behind the vehicle, since the vehicle and the target object are aligned left to right or front to back, it can be determined that the vehicle and the target object do not meet the preset position condition. In this case, there is no need to determine the target plane.

[0128] Step 820, determine the target virtual sub - point cloud distributed on the side of the target plane close to the vehicle in the first virtual sub - point cloud.

[0129] Continue to refer to Figure 9 , the target plane can divide the first virtual sub - point cloud into left and right parts, and the part of the first virtual sub - point cloud located on the left side of the target plane can be considered as the target virtual sub - point cloud distributed on the side of the target plane close to the vehicle in the first virtual sub - point cloud.

[0130] Step 830, take the target plane as the mirror reference plane, perform mirror processing on the target virtual sub - point cloud to obtain the second virtual sub - point cloud, and form an updated virtual point cloud including the second virtual sub - point cloud.

[0131] Continue to refer to Figure 9, the target plane can be used as the mirror reference plane to mirror the target virtual sub-point cloud, so that the mirror image result of the target virtual sub-point cloud can be formed on the right side of the target plane, that is, the second virtual sub-point cloud is formed. Thus, the second virtual sub-point cloud can be added to the virtual point cloud to obtain an updated virtual point cloud including the second virtual sub-point cloud. It should be noted that the first virtual sub-point cloud and the second virtual sub-point cloud in the updated virtual point cloud can jointly represent the target object.

[0132] Figure 8 In the illustrated embodiment, if the target object and the vehicle meet the preset position condition, some areas of the target object cannot be captured by the vehicle's camera (for example Figure 9 certain areas located on the right side of the plane P), then, for the target object, the target plane can be determined, and the target virtual sub-point cloud distributed on the side of the target plane close to the vehicle in the first virtual sub-point cloud used to represent the target object can be determined. Further, with the target plane as the mirror reference plane, the target virtual sub-point cloud is mirrored. Since dynamic objects are generally left-right symmetric, the second virtual sub-point cloud obtained by mirroring can more accurately represent the area of the target object that is not captured. In this way, through the first virtual sub-point cloud and the second virtual sub-point cloud, the target object can be more completely represented, facilitating the user to observe the target object and improving the user experience.

[0133] Of course, based on the first virtual sub-point cloud, the orientation information, and the relative position relationship, the method for updating the virtual point cloud is not limited to this. For example, in some cases, if the relative position relationship indicates that the target object is always located directly to the right of the vehicle, then the vehicle's camera can only capture the left side area of the vehicle. Then, based on the image collected by the vehicle's camera, the width of the target object can be determined. Additionally, referring to the semantic category of the target object, the length ratio of the target object can be estimated. Combining with the width of the target object, the length of the target object can be inferred. On this basis, the target plane can also be determined, and the extension direction of the target plane can be consistent with the orientation represented by the orientation information. The target plane can be used to divide the target object into two equal left and right parts. After that, with the target plane as the mirror reference plane, the target virtual sub-point cloud is mirrored to obtain the third virtual sub-point cloud, forming an updated virtual point cloud including the third virtual sub-point cloud. It should be noted that the first virtual sub-point cloud and the third virtual sub-point cloud in the updated virtual point cloud can jointly represent the target object.

[0134] Step 740, generate a virtual environment corresponding to the vehicle surrounding environment based on the updated virtual point cloud.

[0135] Optionally, the specific implementation of step 740 can refer to the relevant introduction to step 330 above and will not be elaborated here.

[0136] In embodiments of the present disclosure, the virtual point cloud can be updated by combining a first virtual sub-point cloud for characterizing a target object in the vehicle surrounding environment, the orientation information of the target object, and the relative position relationship between the vehicle and the target object, so as to optimize the virtual point cloud. For example, through the application of mirror processing, the updated virtual point cloud can more completely characterize the target object. In this way, when the virtual environment is displayed through a display device, from the perspective of the user's vision, the virtual environment can more completely reflect the shape of the target object, which is beneficial to improving the user experience.

[0137] Figure 10-1 FIG. 5 is a schematic flowchart of a method for generating a virtual environment corresponding to the vehicle surrounding environment based on a virtual point cloud formed by virtual point data respectively corresponding to each voxel provided in some exemplary embodiments of the present disclosure. Figure 10-1 The method shown may include step 1010, step 1020, step 1030, and step 1040. Optionally, the combination of step 1010 to step 1040 may be an alternative implementation of step 140 of the present disclosure.

[0138] Step 1010, determining a plurality of virtual sub-point clouds in the virtual point cloud; wherein, different virtual sub-point clouds among the plurality of virtual sub-point clouds are used to characterize different objects in the environment where the vehicle is located.

[0139] Optionally, the virtual point cloud can be divided into a plurality of virtual sub-point clouds for characterizing different objects according to the semantic category in each virtual point data in the virtual point cloud; wherein, each virtual sub-point cloud among the plurality of virtual sub-point clouds includes several virtual point data in the virtual point cloud, and the semantic categories of the several virtual point data included in each virtual sub-point cloud can be the same.

[0140] Step 1020, determining the densification parameters respectively adapted to the plurality of virtual sub-point clouds.

[0141] Optionally, for each virtual sub-point cloud among the plurality of virtual sub-point clouds, the densification parameter adapted to the virtual sub-point cloud can be determined; wherein, the densification parameter adapted to the virtual sub-point cloud can be a parameter for changing the virtual sub-point cloud from sparse to dense.

[0142] In some alternative embodiments of the present disclosure, as Figure 10-2 shown, step 1020 may include step 10201, step 10203, and step 10205.

[0143] Step 10201, determining the object volume of the object respectively characterized by the plurality of virtual sub-point clouds.

[0144] Optionally, for each virtual sub-point cloud among multiple virtual sub-point clouds, based on the virtual point coordinates in each virtual point data included in the virtual sub-point cloud, the spatial volume occupied by the virtual sub-point cloud can be determined through geometric calculation, and this spatial volume can be used as the object volume of the object represented by the virtual sub-point cloud.

[0145] Step 10203: Determine the volume levels to which the object volumes respectively corresponding to the multiple virtual sub-point clouds belong.

[0146] Optionally, multiple volume levels can be preset. As an example, the multiple volume levels can be five volume levels, which can be respectively, in ascending order of level: extremely small, small, medium, large, and extremely large. The multiple volume levels can be in one-to-one correspondence with multiple volume ranges. For each virtual sub-point cloud among the multiple virtual sub-point clouds, the object volume corresponding to the virtual sub-point cloud can be compared with the multiple volume ranges to determine the volume range to which the object volume corresponding to the virtual sub-point cloud belongs, and the volume level corresponding to this volume range can be used as the volume level to which the object volume belongs.

[0147] Step 10205: Based on the volume levels respectively corresponding to the multiple virtual sub-point clouds, determine the densification parameters respectively adapted to the multiple virtual sub-point clouds according to the corresponding relationship between the preset volume levels and the densification parameters.

[0148] Optionally, the corresponding relationship between the volume levels and the densification parameters can be preset; among them, any densification parameter can be in numerical form and is used to represent the number of virtual points to be inserted between two adjacent virtual points. The higher the volume level, the smaller the corresponding densification parameter, and the lower the volume level, the larger the corresponding densification parameter. For each virtual sub-point cloud among the multiple virtual sub-point clouds, after determining the volume level corresponding to the virtual sub-point cloud, the densification parameter corresponding to this volume level can be determined according to the preset corresponding relationship, and this densification parameter can be used as the densification parameter adapted to the virtual sub-point cloud.

[0149] Figure 10-2 In the illustrated embodiment, during the densification process, different objects can adopt corresponding densification parameters according to the volume levels. For example, for an object with a larger volume, a smaller densification parameter can be adopted, that is, relatively fewer virtual points can be inserted for this object, while for an object with a smaller volume, a larger densification parameter can be adopted, that is, relatively more virtual points can be inserted for this object. In this way, the shapes of objects of various volumes can be better expressed, facilitating user observation and being beneficial to improving the user experience.

[0150] Of course, the method for determining the densification parameters suitable for each of the multiple virtual sub-point clouds is not limited to this. For example, a target function with the independent variable being volume and the dependent variable being the densification parameter can be preset; among them, the dependent variable and the independent variable can be negatively correlated. For example, the target function can be a linear function with a slope less than 0, an exponential function with a base greater than 0 and less than 1, etc. For each virtual sub-point cloud among the multiple virtual sub-point clouds, the volume of the object corresponding to the virtual sub-point cloud can be used as the value of the independent variable and substituted into the target function, and the target function can obtain the corresponding densification parameter through calculation, and the densification parameter can be used as the densification parameter suitable for the virtual sub-point cloud.

[0151] Step 1030: Densify the multiple virtual sub-point clouds according to the densification parameters suitable for each of them, and form an updated virtual point cloud including the densified sub-point clouds corresponding to the multiple virtual sub-point clouds respectively.

[0152] Optionally, for each virtual sub-point cloud among the multiple virtual sub-point clouds, if the densification parameter suitable for it is used to represent that the number of virtual points to be inserted between two adjacent virtual points is K, then for any two adjacent virtual points in the virtual sub-point cloud, K virtual points can be inserted between them. Here, the insertion method can be either random insertion or equally spaced insertion. In this way, the densification processing result of the virtual sub-point cloud can be obtained, that is, the densified sub-point cloud corresponding to the virtual sub-point cloud can be obtained. On this basis, an updated virtual point cloud including the densified sub-point clouds corresponding to the multiple virtual sub-point clouds respectively can be formed.

[0153] In some embodiments, the densification parameter suitable for the first virtual sub-point cloud may include: (1) the number of virtual points to be inserted between two adjacent virtual points along the X-axis direction of the vehicle coordinate system; (2) the number of virtual points to be inserted between two adjacent virtual points along the Y-axis direction of the vehicle coordinate system; (3) the number of virtual points to be inserted between two adjacent virtual points along the Z-axis direction of the vehicle coordinate system. When densifying the virtual sub-point cloud, virtual points can be inserted along the X-axis direction, Y-axis direction, and Z-axis direction respectively according to the corresponding numbers.

[0154] Step 1040: Generate a virtual environment corresponding to the vehicle surrounding environment based on the updated virtual point cloud.

[0155] Optionally, for the specific implementation manner of step 1040, reference can be made to the relevant introduction of step 330 above, which will not be elaborated here.

[0156] It should be noted that by converting the voxelized representation of the vehicle's surrounding environment, a relatively sparse point cloud representation (i.e., virtual point cloud) can be obtained. In the embodiments of the present disclosure, multiple virtual sub-point clouds in the virtual point cloud can be determined, and the multiple virtual sub-point clouds can be densified according to the densification parameters adapted to each of the multiple virtual sub-point clouds, so as to obtain a denser point cloud representation, so as to represent the vehicle's surrounding environment with more virtual points. In this way, the embodiments of the present disclosure can obtain a more refined perception result and present the perception result to the user, thereby facilitating the user's observation and being beneficial to improving the user's experience.

[0157] In some alternative examples, the vehicle's camera can capture images of the vehicle's surrounding environment. Based on the images of the vehicle's surrounding environment, the attribute information corresponding to each voxel used to characterize the vehicle's surrounding environment can be determined, that is, the voxelized representation of the vehicle's surrounding environment is obtained. Next, the voxelized representation of the vehicle's surrounding environment can be converted into a point cloud representation, that is, a virtual point cloud is obtained.

[0158] For the virtual point cloud, first, Figure 8 perform mirror processing according to the Figure 10-1 shown implementation manner, and then perform densification processing according to the

[0159] shown implementation manner to obtain an updated virtual point cloud.

[0159] For the updated virtual point cloud, corresponding display parameters can be determined for each virtual point data therein. For example, the display parameters can be determined according to the following rules:

[0160] (1) Based on the origin of the ego-vehicle coordinate system, the color of the virtual point farther away from this reference is lighter;

[0161] (2) Based on the ground, perform a gradient transparency processing from bottom to top, and the higher the position, the more transparent;

[0162] (3) Based on the virtual camera, perform a gradient processing of the size of the virtual points from near to far, and the farther the position, the smaller;

[0163] (4) For the virtual points whose corresponding projection points will cover objects such as cyclists and other vehicles on the road surface, perform semi-transparency processing on them to avoid such covering;

[0164] (5) Virtual points of different semantic categories are distinguished by different colors;

[0165] (6) Virtual points of different risk levels are distinguished by different colors.

[0166] Finally, corresponding display parameters can be added to each piece of virtual point data in the updated virtual point cloud to further update the virtual point cloud, and a rendering engine can be used to render the further updated virtual point cloud to generate a virtual environment corresponding to the vehicle surrounding environment, so as to display the virtual environment through a center control screen or an instrument screen. Optionally, based on the images collected by the vehicle's camera, each processing operation in the process of generating the virtual environment can be executed by the vehicle's intelligent driving system.

[0167] In summary, in the embodiments of the present disclosure, based on a low-cost vision sensor (such as a camera), by obtaining a voxelized representation of the vehicle surrounding environment, and then combining the conversion from the voxelized representation to the point cloud representation, as well as the simulated display of the environment, it is possible to perform high-precision point cloud display of the vehicle surrounding environment without increasing the sensor cost, obtain a more refined perception result, and effectively present the perception result to the user. In this way, through an intuitive, vivid, and aesthetically pleasing picture display, the user can understand the vehicle surrounding environment, which is beneficial to improving the user experience.

[0168] Exemplary Device

[0169] Figure 11 It is a schematic structural diagram of an environment display device during vehicle driving provided by some exemplary embodiments of the present disclosure. Figure 11 The shown device may include:

[0170] An acquisition module 1110, configured to acquire images of the vehicle surrounding environment collected by the vehicle's sensors;

[0171] A first determination module 1120, configured to determine attribute information corresponding to each voxel used to characterize the vehicle surrounding environment based on the images acquired by the acquisition module 1110;

[0172] A second determination module 1130, configured to determine virtual point data corresponding to each voxel based on the attribute information corresponding to each voxel determined by the first determination module 1120;

[0173] A generation module 1140, configured to generate a virtual environment corresponding to the vehicle surrounding environment based on the virtual point cloud formed by the virtual point data corresponding to each voxel determined by the second determination module 1130;

[0174] A display module 1150, configured to display the virtual environment generated by the generation module 1140 through a display device.

[0175] In some alternative examples, as Figure 12 shown, the second determination module 1130 includes:

[0176] The first acquisition sub-module 1210 is configured to acquire the semantic category and the spatial position information corresponding to each voxel from the attribute information corresponding to each voxel determined by the first determination module 1120;

[0177] The first determination sub-module 1220 is configured to determine the virtual point coordinates corresponding to each voxel based on the spatial position information corresponding to each voxel acquired by the first acquisition sub-module 1210;

[0178] The integration sub-module 1230 is configured to integrate the semantic category corresponding to each voxel acquired by the first acquisition sub-module 1210 and the virtual point coordinates corresponding to each voxel determined by the first determination sub-module 1220 to obtain the virtual point data corresponding to each voxel.

[0179] In some alternative examples, as Figure 13 shown, the generation module 1140 includes:

[0180] The second determination sub-module 1310 is configured to determine a first virtual sub-point cloud for characterizing a target object in the vehicle surrounding environment from the virtual point cloud;

[0181] The second acquisition sub-module 1320 is configured to acquire the orientation information of the target object and the relative position relationship between the vehicle and the target object;

[0182] The first update sub-module 1330 is configured to update the virtual point cloud based on the first virtual sub-point cloud determined by the second determination sub-module 1310 and the orientation information and the relative position relationship acquired by the second acquisition sub-module 1320;

[0183] The first generation sub-module 1340 is configured to generate a virtual environment corresponding to the vehicle surrounding environment based on the virtual point cloud updated by the first update sub-module 1330.

[0184] In some alternative examples, the first update sub-module 1330 includes:

[0185] The first determination unit is configured to determine a target plane in response to the relative position relationship acquired by the second acquisition sub-module 1320 indicating that the vehicle and the target object satisfy a preset position condition; wherein, the extension direction of the target plane is consistent with the orientation represented by the orientation information, and the target plane is used to divide the target object into two parts;

[0186] The second determination unit is configured to determine a target virtual sub-point cloud distributed on the side of the target plane determined by the first determination unit and close to the vehicle in the first virtual sub-point cloud;

[0187] A mirror processing unit, configured to perform mirror processing on the target virtual sub-point cloud determined by the second determination unit with the target plane determined by the first determination unit as the mirror reference plane, to obtain a second virtual sub-point cloud, and form an updated virtual point cloud including the second virtual sub-point cloud.

[0188] In some alternative examples, such as Figure 14 shown, the generation module 1140 includes:

[0189] A third determination sub-module 1410, configured to determine a plurality of virtual sub-point clouds in the virtual point cloud; wherein, different virtual sub-point clouds in the plurality of virtual sub-point clouds are used to represent different objects in the environment where the vehicle is located;

[0190] A fourth determination sub-module 1420, configured to determine the densification parameters respectively adapted to the plurality of virtual sub-point clouds determined by the third determination sub-module 1410;

[0191] A second update sub-module 1430, configured to perform densification processing on the plurality of virtual sub-point clouds according to the densification parameters respectively adapted to the plurality of virtual sub-point clouds determined by the fourth determination sub-module 1420, and form an updated virtual point cloud including the densified sub-point clouds respectively corresponding to the plurality of virtual sub-point clouds;

[0192] A second generation sub-module 1440, configured to generate a virtual environment corresponding to the environment around the vehicle based on the virtual point cloud updated by the second update sub-module 1430.

[0193] In some alternative examples, the fourth determination sub-module 1420 includes:

[0194] A third determination unit, configured to determine the object volume of the object represented by each of the plurality of virtual sub-point clouds determined by the third determination sub-module 1410;

[0195] A fourth determination unit, configured to determine the volume level to which the object volumes respectively corresponding to the plurality of virtual sub-point clouds determined by the third determination unit belong;

[0196] A fifth determination unit, configured to determine the densification parameters respectively adapted to the plurality of virtual sub-point clouds according to the corresponding relationship between the volume levels and the densification parameters set in advance based on the volume levels respectively corresponding to the plurality of virtual sub-point clouds determined by the fourth determination unit.

[0197] In some alternative examples, such as Figure 15 shown, the generation module 1140 includes:

[0198] A fifth determination sub-module 1510, configured to determine the display parameters corresponding to each virtual point data in the virtual point cloud;

[0199] The third update sub-module 1520 is used to add the corresponding display parameters determined by the fifth determination sub-module 1510 to each piece of virtual point data in the virtual point cloud, so as to update the virtual point cloud;

[0200] The third generation sub-module 1530 is used to generate a virtual environment corresponding to the vehicle surrounding environment based on the virtual point cloud updated by the third update sub-module 1520.

[0201] In some optional examples, each piece of virtual point data in the virtual point cloud includes: virtual point coordinates, semantic category, and risk level, where the risk level is used to characterize the degree of risk brought to the vehicle;

[0202] The fifth determination sub-module 1510 includes:

[0203] The sixth determination unit is used to, for each piece of virtual point data in the virtual point cloud, based on the semantic category in the virtual point data, determine the target display color according to the pre-set correspondence between the semantic category and the display color, and / or, based on the risk level in the virtual point data, determine the target display color according to the pre-set correspondence between the risk level and the display color;

[0204] The seventh determination unit is used to, for each piece of virtual point data in the virtual point cloud, based on the virtual point coordinates in the virtual point data, determine the first distance between the virtual point that conforms to the virtual point data and the vehicle, and based on the first distance, determine the target display transparency according to the pre-set correspondence between the distance range and the display transparency, and / or, based on the virtual point coordinates in the virtual point data, determine the position height of the virtual point that conforms to the virtual point data, and based on the position height, determine the target display transparency according to the pre-set correspondence between the height range and the display transparency;

[0205] The integration unit is used to, for each piece of virtual point data in the virtual point cloud, integrate the target display color corresponding to the virtual point data determined by the sixth determination unit and the target display transparency corresponding to the virtual point data determined by the seventh determination unit, to obtain the display parameter corresponding to the virtual point data.

[0206] In some optional examples, each piece of virtual point data in the virtual point cloud includes: virtual point coordinates and semantic category;

[0207] The fifth determination sub-module 1510 includes:

[0208] The screening unit is used to, based on the semantic category in each piece of virtual point data in the virtual point cloud, screen out the virtual point data whose represented object is a static object from the virtual point cloud;

[0209] An eighth determination unit, configured to determine a projection covering relationship between a virtual point that conforms to the filtered virtual point data and the road surface where the vehicle is located based on the virtual point coordinates in the virtual point data filtered by the filtering unit;

[0210] A ninth determination unit, configured to determine a target display transparency adapted to the projection covering relationship determined by the eighth determination unit for the filtered virtual point data;

[0211] A tenth determination unit, configured to determine a display parameter corresponding to the filtered virtual point data based on the target display transparency corresponding to the filtered virtual point data determined by the ninth determination unit.

[0212] In some alternative examples, each piece of virtual point data in the virtual point cloud includes: virtual point coordinates;

[0213] The fifth determination sub-module 1510 includes:

[0214] An eleventh determination unit, configured to, for each piece of virtual point data in the virtual point cloud, determine a second distance between a virtual point that conforms to the virtual point data and a virtual camera for the display setting of the virtual environment based on the virtual point coordinates in the virtual point data, and determine a target display size based on the second distance according to a correspondence relationship between a preset distance range and a display size;

[0215] A twelfth determination unit, configured to, for each piece of virtual point data in the virtual point cloud, determine a display parameter corresponding to the virtual point data based on the target display size corresponding to the virtual point data determined by the eleventh determination unit.

[0216] In the device of the present disclosure, the above-described various alternative embodiments, alternative implementation manners, and alternative examples can be flexibly selected and combined as needed to achieve corresponding functions and effects, and the present disclosure does not list them one by one.

[0217] Exemplary Electronic Device

[0218] Figure 16 The block diagram of an electronic device according to an embodiment of the present disclosure is illustrated. The electronic device 1600 includes one or at least one processor 1610 and a memory 1620.

[0219] The processor 1610 may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1600 to perform desired functions.

[0220] The memory 1620 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1610 may run one or more computer program instructions to implement the methods of the various embodiments of the present disclosure described above and / or other desired functions.

[0221] In one example, the electronic device 1600 may further include: an input device 1630 and an output device 1640, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0222] The input device 1630 may further include, for example, a keyboard, a mouse, various sensors, a touch screen, and so on.

[0223] The output device 1640 may output various information to the outside, which may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, and so on.

[0224] Of course, for simplicity, Figure 16 only some of the components related to the present disclosure in the electronic device 1600 are shown, and components such as a bus, an input / output interface, and so on are omitted. In addition, according to specific application scenarios, the electronic device 1600 may further include any other appropriate components.

[0225] Exemplary Computer Program Product and Computer Readable Storage Medium

[0226] In addition to the above methods and devices, the embodiments of the present disclosure may also be computer program products, which include computer program instructions that, when run by a processor, cause the processor to execute the steps in the methods according to the various embodiments of the present disclosure described in the "Exemplary Methods" section above of this specification.

[0227] The computer program products may be written in any combination of one or more programming languages to write program codes for performing the operations of the embodiments of the present disclosure. The programming languages include object-oriented programming languages, such as Java, C++, etc., and also include conventional procedural programming languages, such as the "C" language or similar programming languages. The program codes may be executed entirely on the user computing device, partially on the user device, executed as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0228] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of the present disclosure described in the "Exemplary Methods" section above of this specification.

[0229] The computer-readable storage medium may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0230] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present disclosure. The above-described specific details are only for the purpose of illustration and easy understanding, and not for limitation. The above details do not limit the present disclosure to necessarily adopt the above specific details for implementation.

[0231] Those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present disclosure and their equivalent technologies, the present disclosure is also intended to include these modifications and variations.

Claims

1. An environmental display method during vehicle driving, comprising: Obtaining an image of the vehicle surrounding environment collected by sensors of the vehicle; Based on the image, determining attribute information respectively corresponding to each voxel for characterizing the vehicle surrounding environment; Based on the attribute information respectively corresponding to each voxel, determining virtual point data respectively corresponding to each voxel; Based on the virtual point cloud formed by the virtual point data respectively corresponding to each voxel, generating a virtual environment corresponding to the vehicle surrounding environment; Displaying the virtual environment through a display device.

2. The method according to claim 1, wherein, The determining the virtual point data respectively corresponding to the voxels based on the attribute information respectively corresponding to each voxel includes: Obtaining the semantic category respectively corresponding to each voxel and the spatial position information respectively corresponding to each voxel from the attribute information respectively corresponding to each voxel; Based on the spatial position information respectively corresponding to each voxel, determining the virtual point coordinates respectively corresponding to each voxel; Integrating the semantic category respectively corresponding to each voxel and the virtual point coordinates respectively corresponding to each voxel to obtain the virtual point data respectively corresponding to each voxel.

3. The method according to claim 1 or 2, wherein, The generating the virtual environment corresponding to the vehicle surrounding environment based on the virtual point cloud formed by the virtual point data respectively corresponding to each voxel includes: Determining a first virtual sub-point cloud from the virtual point cloud for characterizing a target object in the vehicle surrounding environment; Obtaining the orientation information of the target object and the relative position relationship between the vehicle and the target object; Based on the first virtual sub-point cloud, the orientation information and the relative position relationship, updating the virtual point cloud; Based on the updated virtual point cloud, generating a virtual environment corresponding to the vehicle surrounding environment.

4. The method according to claim 3, wherein The updating the virtual point cloud based on the first virtual sub-point cloud, the orientation information and the relative position relationship includes: In response to the relative position relationship indicating that the vehicle and the target object satisfy a preset position condition, determining a target plane; wherein, the extending direction of the target plane is consistent with the orientation characterized by the orientation information, and the target plane is used to divide the target object into two parts; Determining a target virtual sub-point cloud distributed on the side of the target plane close to the vehicle in the first virtual sub-point cloud; Using the target plane as a mirror reference plane, performing mirror processing on the target virtual sub-point cloud to obtain the second virtual sub-point cloud, and forming the updated virtual point cloud including the second virtual sub-point cloud.

5. The method according to claim 1 or 2, wherein, The generating the virtual environment corresponding to the vehicle surrounding environment based on the virtual point cloud formed by the virtual point data respectively corresponding to each voxel includes: Determining a plurality of virtual sub-point clouds in the virtual point cloud; wherein, different virtual sub-point clouds among the plurality of virtual sub-point clouds are used to characterize different objects in the environment where the vehicle is located; Determining the densification parameters respectively adapted to the plurality of virtual sub-point clouds; Densify the multiple virtual sub - point clouds according to the densification parameters adapted to each of the multiple virtual sub - point clouds, to form an updated virtual point cloud including densified sub - point clouds corresponding to the multiple virtual sub - point clouds respectively; Generate a virtual environment corresponding to the vehicle surrounding environment based on the updated virtual point cloud.

6. The method according to claim 5, wherein The determining the densification parameters adapted to each of the multiple virtual sub - point clouds includes: Determine the object volume of the object represented by each of the multiple virtual sub - point clouds; Determine the volume level to which the object volumes corresponding to the multiple virtual sub - point clouds belong; Based on the volume levels corresponding to the multiple virtual sub - point clouds respectively, determine the densification parameters adapted to each of the multiple virtual sub - point clouds according to the corresponding relationship between the pre - set volume levels and the densification parameters.

7. The method according to claim 1 or 2, wherein The generating a virtual environment corresponding to the vehicle surrounding environment based on the virtual point cloud formed by the virtual point data corresponding to each voxel includes: Determine the display parameters corresponding to each virtual point data in the virtual point cloud; Add the corresponding display parameters to each virtual point data in the virtual point cloud to update the virtual point cloud; Generate a virtual environment corresponding to the vehicle surrounding environment based on the updated virtual point cloud.

8. The method according to claim 7, wherein, Each virtual point data in the virtual point cloud includes: virtual point coordinates, semantic category, and risk level, where the risk level is used to characterize the degree of risk brought to the vehicle; The determining the display parameters corresponding to each virtual point data in the virtual point cloud includes: For each virtual point data in the virtual point cloud, based on the semantic category in the virtual point data, determine the target display color according to the corresponding relationship between the pre - set semantic category and the display color, and / or, based on the risk level in the virtual point data, determine the target display color according to the corresponding relationship between the pre - set risk level and the display color; For each virtual point data in the virtual point cloud, based on the virtual point coordinates in the virtual point data, determine the first distance between the virtual point that conforms to the virtual point data and the vehicle, and based on the first distance, determine the target display transparency according to the corresponding relationship between the pre - set distance range and the display transparency, and / or, based on the virtual point coordinates in the virtual point data, determine the position height of the virtual point that conforms to the virtual point data, and based on the position height, determine the target display transparency according to the corresponding relationship between the pre - set height range and the display transparency; For each virtual point data in the virtual point cloud, integrate the corresponding target display color and the corresponding target display transparency of the virtual point data to obtain the display parameters corresponding to the virtual point data.

9. The method according to claim 7, wherein Each virtual point data in the virtual point cloud includes: virtual point coordinates and semantic category; The determining the display parameters corresponding to each virtual point data in the virtual point cloud includes: Based on the semantic category in each virtual point data in the virtual point cloud, screen out the virtual point data whose represented object is a static object from the virtual point cloud; Based on the virtual point coordinates in the filtered virtual point data, determine the projection covering relationship between the virtual points that conform to the filtered virtual point data and the road surface where the vehicle is located; For the filtered virtual point data, determine a target display transparency adapted to the projection covering relationship; Based on the target display transparency corresponding to the filtered virtual point data, determine the display parameters corresponding to the filtered virtual point data.

10. The method according to claim 7, wherein, Each virtual point data in the virtual point cloud includes: virtual point coordinates; The determining the display parameters corresponding to each virtual point data in the virtual point cloud includes: For each virtual point data in the virtual point cloud, based on the virtual point coordinates in the virtual point data, determine a second distance between the virtual points that conform to the virtual point data and a virtual camera for the display setting of the virtual environment. Based on the second distance, determine a target display size according to the corresponding relationship between the preset distance range and the display size; For each virtual point data in the virtual point cloud, based on the target display size corresponding to the virtual point data, determine the display parameters corresponding to the virtual point data.

11. An environmental display device during vehicle driving, comprising: An acquisition module, configured to acquire an image of the environment around the vehicle collected by a sensor of the vehicle; A first determination module, configured to determine, based on the image acquired by the acquisition module, the attribute information corresponding to each voxel for characterizing the environment around the vehicle; A second determination module, configured to determine the virtual point data corresponding to each voxel respectively based on the attribute information corresponding to each voxel determined by the first determination module; A generation module, configured to generate a virtual environment corresponding to the environment around the vehicle based on the virtual point cloud formed by the virtual point data corresponding to each voxel determined by the second determination module; A display module, configured to display the virtual environment generated by the generation module through a display device.

12. A computer-readable storage medium, where the storage medium stores a computer program, and the computer program is used to execute the environmental display method during vehicle driving according to any one of the above claims 1-10.

13. An electronic device, the electronic device includes: A processor; A memory for storing executable instructions of the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the environmental display method during vehicle driving according to any one of the above claims 1-10.

Images