Point cloud rendering method and device, electronic equipment and storage medium
By distinguishing the frequency processing of dynamic and static point clouds, the overhead of point cloud data transmission and rendering is reduced, and the problem of excessive bandwidth and rendering overhead in point cloud rendering is solved, achieving a smoother rendering effect.
Patent Information
- Application Number
- CN202311808879.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-26
- Publication Date
- 2025-07-04
AI Technical Summary
The transmission and rendering of point cloud data requires a large bandwidth cost and rendering overhead, resulting in the rendering side being unable to respond in real time, and problems such as stuck or crashes occur.
Different frequencies are used to receive and render dynamic point clouds and static point clouds. Dynamic point clouds are used for point clouds with pavement areas or moving targets, and static point clouds are used for point clouds with non-pavement areas or other point clouds with low frequencies, and render them according to the position information of the point cloud.
It reduces the bandwidth cost and rendering overhead of point cloud data transmission, improves rendering fluency, and ensures that the rendered picture conforms to the actual scene.
Smart Images
Figure CN120259519A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the technical field of point cloud processing, and particularly relates to a point cloud rendering method, apparatus, electronic device, and storage medium. Background Art
[0002] In fields such as driving or surveying and mapping, point cloud data of the environment scene to be rendered can usually be collected by a radar or an indirect time-of-flight sensor. Then, the point cloud data is sent to the rendering end in real time for real-time rendering.
[0003] However, point cloud data is a data set composed of a large number of points in three-dimensional space, which is used to describe the shape and structure of the scene to be rendered. Therefore, when there is a large amount of point cloud data, transmitting a large amount of point cloud data not only requires a large bandwidth cost, but also the rendering end will occupy more rendering overhead during real-time rendering. Furthermore, it will cause the rendering end to be unable to respond in real time, resulting in problems such as freezing or crashing. Summary of the Invention
[0004] Embodiments of this application provide a point cloud rendering method, apparatus, electronic device, and storage medium, which can solve the problem that when transmitting and rendering point clouds, it not only requires a large bandwidth cost, but also requires a large amount of rendering overhead.
[0005] In a first aspect, embodiments of this application provide a point cloud rendering method, which includes:
[0006] Receiving and rendering dynamic point clouds at a first frequency, and displaying the rendered dynamic point clouds on a display interface according to the position information of the dynamic point clouds; wherein, the dynamic point clouds are point clouds of a road surface area collected by a radar device or point clouds of moving targets in the road surface area; the display interface is used to display point clouds of the detection area of the radar device, and the position points in the display interface correspond to the position points of the detection area; the detection area includes a road surface area and a non-road surface area;
[0007] Receiving and rendering static point clouds at a second frequency, and displaying the rendered static point clouds on the display interface according to the position information of the static point clouds; wherein, the static point clouds are point clouds of a non-road surface area collected by the radar device or point clouds other than the point clouds of moving targets in the road surface area, and the second frequency is less than the first frequency.
[0008] In a second aspect, embodiments of this application provide a point cloud rendering apparatus, which is applied to an electronic device, and the apparatus includes:
[0009] A dynamic point cloud processing module is configured to receive dynamic point clouds at a first frequency, render them, and display the rendered dynamic point clouds on a display interface according to the position information of the dynamic point clouds; wherein, the dynamic point clouds are point clouds of a road surface area collected by a radar device or point clouds of moving targets in the road surface area; the display interface is configured to display point clouds of a detection area of the radar device, and the position points in the display interface correspond to the position points of the detection area; the detection area includes a road surface area and a non-road surface area;
[0010] A static point cloud processing module is configured to receive static point clouds at a second frequency, render them, and display the rendered static point clouds on the display interface according to the position information of the static point clouds; wherein, the static point clouds are point clouds of a non-road surface area collected by the radar device or point clouds other than the point clouds of moving targets in the road surface area, and the second frequency is less than the first frequency.
[0011] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The electronic device is connected to the radar device. When the processor executes the computer program, the method of the first aspect as described above is implemented in cooperation with the radar device.
[0012] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the method of the first aspect as described above is implemented.
[0013] In a fifth aspect, an embodiment of the present application provides a computer program product, and when the computer program product runs on an electronic device, the electronic device is caused to execute the method of the first aspect as described above.
[0014] The beneficial effects of the embodiments of the present application compared with the prior art are as follows: After collecting the static point cloud and the dynamic point cloud in the scene to be rendered, since the dynamic point cloud is the point cloud of the road surface area collected by the radar device or the point cloud of the moving target in the road surface area, it can be considered that the object information represented by the dynamic point cloud is relatively sensitive to spatial changes over time (the changes are obvious). Therefore, the dynamic point cloud can be received at a relatively high first frequency. Moreover, since the point cloud of the non-road surface area collected by the radar device or the point cloud other than the point cloud of the moving target in the road surface area, it can be considered that the object information represented by the static point cloud is not sensitive to spatial changes over time (no change or insignificant change). Therefore, the static point cloud can be received at a relatively low second frequency. And, since the position points in the display interface correspond to the position points in the detection area, the dynamic point cloud can be directly rendered at a high frequency according to the first frequency for receiving the dynamic point cloud, and the static point cloud can be rendered at a low frequency according to the second frequency for receiving the static point cloud. Based on this, on the basis of being able to render the information of each object in the detection area based on the dynamic and static characteristics of the static and dynamic objects themselves, it is also possible to ensure that the rendered picture conforms to the actual scene. Furthermore, by reducing the transmission frequency and rendering frequency of the static point cloud, not only can the bandwidth cost required for transmitting the point cloud data and the rendering overhead required for rendering be reduced, but also because the amount of point cloud data rendered each time is small, the smoothness of the rendered picture can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.
[0016] Figure 1 is a flowchart of the implementation of a point cloud rendering method provided by an embodiment of the present application;
[0017] Figure 2 is a schematic diagram of the application scenario of the scene to be rendered in a point cloud rendering method provided by an embodiment of the present application;
[0018] Figure 3 is a schematic diagram of the application scenario of the scene to be rendered in a point cloud rendering method provided by another embodiment of the present application;
[0019] Figure 4 is a schematic diagram of the application scenario of the scene to be rendered in a point cloud rendering method provided by still another embodiment of the present application;
[0020] Figure 5It is a schematic diagram of the application scenario of the scene to be rendered in a point cloud rendering method provided by another embodiment of the present application;
[0021] Figure 6 It is a schematic structural diagram of a point cloud rendering device provided by an embodiment of the present application;
[0022] Figure 7 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners
[0023] In the following description, specific details such as specific system structures and technologies are proposed for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.
[0024] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.
[0025] In addition, in the description of the specification of the present application and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.
[0026] In fields such as driving or surveying and mapping, point cloud data of the environment scene to be rendered can usually be collected in real time through a radar or an indirect Time Of Flight (TOF) sensor. Then, the point cloud data is sent to the rendering end in real time for real-time rendering. Generally, the number of point cloud data collected in real time is large.
[0027] Moreover, point cloud data is a data set composed of a large number of points in three-dimensional space, which is used to describe the shape and structure of the scene to be rendered. Therefore, when the number of point cloud data is large, transmitting a large amount of point cloud data not only requires a large bandwidth cost, but also the rendering end will occupy more rendering overhead during real-time rendering. As a result, the rendering end cannot respond in real time, and problems such as freezing or crashing may occur.
[0028] Based on this, in order to reduce the bandwidth cost required for transmitting point cloud data during the rendering process and be able to smoothly render the point cloud data, the embodiments of the present application provide a point cloud rendering method. This method can be applied to electronic devices such as radar devices, camera devices, tablet computers, laptop computers, Ultra-Mobile Personal Computers (UMPCs), and netbooks. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.
[0029] Please refer to Figure 1 , Figure 1 which shows the implementation flowchart of a point cloud rendering method provided by the embodiments of the present application. The method includes the following steps:
[0030] S101. Receive dynamic point clouds at a first frequency and perform rendering, and display the rendered dynamic point clouds on the display interface according to the position information of the dynamic point clouds; wherein, the dynamic point clouds are the point clouds of the road surface area collected by the radar device or the point clouds of moving targets in the road surface area; the display interface is used to display the point clouds of the detection area of the radar device, and the position points in the display interface correspond to the position points of the detection area; the detection area includes the road surface area and the non-road surface area.
[0031] S102. Receive static point clouds at a second frequency and perform rendering, and display the rendered static point clouds on the display interface according to the position information of the static point clouds; wherein, the static point clouds are the point clouds of the non-road surface area collected by the radar device or the point clouds other than the point clouds of moving targets in the road surface area, and the second frequency is less than the first frequency.
[0032] In one embodiment, the detection area includes but is not limited to the road during the automatic driving process, the point cloud map, or the point cloud scene in the game, and no limitation is imposed thereon. For the sake of explanation, in this embodiment, the above detection area may be a road. The above first frequency and second frequency can be set according to the actual situation, and no limitation is imposed thereon.
[0033] Among them, the dynamic point clouds are the point clouds of the road surface area collected by the radar device or the point clouds of moving targets in the road surface area. The static point clouds are the point clouds of the non-road surface area collected by the radar device or the point clouds other than the point clouds of moving targets in the road surface area.
[0034] As an example, the radar device may pre-store a high-precision map, and the detection area of the radar device is marked in the high-precision map. In addition, the dynamic point clouds and static point clouds in the detection area can be pre-divided by the staff. At this time, the radar device can determine the point clouds collected from the road surface area as dynamic point clouds, and determine the point clouds from the non-road surface area as static point clouds.
[0035] It should be noted that since the road surface area in a road is usually the area where pedestrians and vehicles pass, it can be considered that the road surface area will often change (high change frequency), and / or the change amplitude is relatively large. Also, since the non-road surface area in a road usually has no pedestrians and vehicles passing, it can be considered that the non-road surface area will not often change (low change frequency), and / or the change amplitude is relatively small.
[0036] Exemplarily, referring to Figure 2 , Figure 2 is a schematic diagram of an application scenario of a detection area in a point cloud rendering method provided by an embodiment of the present application. Among them, Figure 2 the detection area in can be a road scenario. The area P1 surrounded by the white line can be considered as the road surface area, and the area P2 not surrounded by the white line can be considered as the non-road surface area.
[0037] From Figure 2 it can be seen that the white line can be considered as the boundary of the road. The area inside the road is usually the road surface area where pedestrians walk and vehicles drive. Therefore, it can be determined that when collecting the point cloud of the area inside the road (area P1), its point cloud usually changes. Based on this, the radar device can determine the point cloud collected from the area inside the road (area P1) as dynamic point cloud.
[0038] Also, the outside of the road is usually a non-road surface area, and people and vehicles usually cannot walk and drive in the outside area of the road. Therefore, it can be considered that the outside area of the road usually does not change. Based on this, the electronic device can determine the point cloud collected from the outside area of the road (area P2) as static point cloud.
[0039] It should be noted that determining all the point clouds collected from the road surface area as dynamic point clouds for rendering can enable the electronic device to detect the road state of the road surface area according to the rendered picture. Exemplarily, after rendering the dynamic point cloud corresponding to the road surface area in the current frame, the rendered picture of the current frame can be compared with the rendered picture of the previous frame. If it is determined that the road is damaged according to the comparison result, a detection result of the road damage state is generated and a reminder is given to the staff. Furthermore, the damage of the road surface can be discovered in time.
[0040] Among them, the above-mentioned road state can also include various states such as road waterlogging or snow accumulation state, road congestion state, etc., which are not limited herein.
[0041] As another example, the radar device can also extract the point cloud of the moving target from each point cloud collected from the road surface area as the dynamic point cloud, and determine the point cloud other than the point cloud of the moving target in the road surface area as the static point cloud.
[0042] Exemplarily, the radar device may be pre - set with point cloud detection software or a target category recognition model to identify the dynamic point cloud of a moving target. Among them, the training method of the target category recognition model is an existing method, which will not be described in detail herein.
[0043] Exemplarily, the radar device may determine the point cloud corresponding to each target from the respective point clouds collected from the road surface area. Then, the target category recognition model is used to identify the point cloud corresponding to the target to determine the category of the target. Finally, when it is determined that the category of the target is a preset category, the target is determined as a moving target, so as to determine the point cloud corresponding to the target as a dynamic point cloud.
[0044] Among them, the above - mentioned categories include, but are not limited to, various categories such as traffic signs, people, vehicles, animals, and plants. In this embodiment, the above - mentioned preset categories may be categories such as people, vehicles, and animals.
[0045] Exemplarily, referring to Figure 3 , Figure 3 is a schematic diagram of the application scenario of the detection area in a point cloud rendering method provided by another embodiment of the present application. Among them, Figure 3 the multiple point clouds corresponding to the respective rectangular areas surrounded by white lines in can be considered as the dynamic point clouds corresponding to the moving targets. At this time, the remaining point clouds (non - dynamic point clouds) in the overall area point cloud can be considered as static point clouds.
[0046] It can be understood that the dynamic point cloud recognized from the road surface area at this time is usually only a part of the point cloud in the road surface area. That is to say, the data volume of the dynamic point cloud recognized by the above - mentioned method is usually much smaller than the data volume of determining all the point clouds collected from the road surface area as dynamic point clouds in the previous embodiment. Furthermore, the number of point clouds that need to be transmitted at a high frequency can be reduced, and the resources required for the transmission and rendering of the dynamic point cloud can be reduced.
[0047] It should be noted that the position points in the display interface correspond one - to - one with the position points in the detection area. Furthermore, when the dynamic point cloud and the static point cloud are received, the dynamic point cloud and the static point cloud can be accurately rendered to display the detection area in the display interface.
[0048] It should be added that since the first frequency is higher than the second frequency, when rendering, the dynamic point cloud received in the current frame can be combined with the static point cloud received and rendered before to display the entire detection area in the display interface.
[0049] It can be understood that since the scenes corresponding to the static point clouds usually do not change frequently (low change frequency) and / or have a small change amplitude. Also, the scenes corresponding to the dynamic point clouds usually change frequently (high change frequency) and / or have a large change amplitude. Based on this, rendering the dynamic point clouds at the first frequency and the static point clouds at the second frequency can also enable the display screen to accurately display the information of each object in the detection area, making the rendered image conform to the actual scene and reducing the rendering overhead required during rendering.
[0050] In summary, in this embodiment, after collecting the static point clouds and dynamic point clouds in the scene to be rendered, since the dynamic point clouds are the point clouds of the road surface area collected by the radar device or the point clouds of moving targets in the road surface area, it can be considered that the object information represented by the dynamic point clouds is relatively sensitive to spatial changes over time (obvious changes). Therefore, the first frequency with a higher frequency can be used to receive the dynamic point clouds. Also, since the point clouds of the non-road surface area collected by the radar device or the point clouds other than the point clouds of moving targets in the road surface area, it can be considered that the object information represented by the static point clouds is not sensitive to spatial changes over time (no change or insignificant change). Therefore, the second frequency with a lower frequency can be used to receive the static point clouds. And, since the position points in the display interface correspond to the position points in the detection area, the dynamic point clouds can be directly rendered at a high frequency according to the first frequency for receiving the dynamic point clouds, and the static point clouds can be rendered at a low frequency according to the second frequency for receiving the static point clouds. Based on this, on the basis of being able to render the information of each object in the detection area based on the dynamic and static characteristics of the static and dynamic objects themselves, it can also be ensured that the rendered image conforms to the actual scene. Furthermore, by reducing the transmission frequency and rendering frequency of the static point clouds, not only can the bandwidth cost required for transmitting the point cloud data and the rendering overhead required during rendering be reduced, but also since the amount of point cloud data rendered each time is smaller, the smoothness of the rendered image can be improved.
[0051] In one embodiment, the number of the above-mentioned radar devices can be 1 or multiple, and there is no limitation in this regard. When there is 1 radar device, the radar device can collect the road surface area in the detection area at the first frequency every time to obtain dynamic point clouds. Also, it can collect the road surface area in the detection area at the second frequency every time to obtain static point clouds.
[0052] Also, when there are multiple radar devices, the multiple radar devices can be divided into radar devices for collecting the road surface area and radar devices for collecting the non-road surface area. And, when there are multiple radar devices, each of the above-mentioned radar devices can be pre-set with identification information that can be uniquely identified. Among them, the identification information can be in the form of numbers, letters, or a combination thereof, and there is no limitation in this regard. Also, the collection area corresponding to each radar device can be pre-set.
[0053] Exemplarily, referring to Figure 4 , Figure 4 is a schematic diagram of an application scenario of a scene to be rendered in a point cloud rendering method provided by another embodiment of the present application. Among them, Figure 4 Each rectangular area enclosed by white lines in can be considered as the acquisition area corresponding to each radar device. The point cloud collected by each radar device from the corresponding acquisition area can be considered as the local static point cloud within the detection area.
[0054] In this embodiment, taking the number of radar devices as multiple as an example, the methods for collecting, sending, receiving, and rendering dynamic point clouds and static point clouds can adopt the following multiple methods. Details are as follows:
[0055] For dynamic point clouds, since it is necessary to ensure that the rendered image conforms to the actual scene, therefore, for the dynamic point clouds collected by each radar device, the radar device can collect and send the dynamic point clouds to the electronic device at the first frequency. Then the electronic device can render the received dynamic point clouds in real time to ensure the real-time rendering of the moving targets corresponding to the dynamic point clouds.
[0056] However, for static point clouds, since the object information represented by the static point clouds is not sensitive to spatial changes over time, the second frequency at which the static point clouds are sent is relatively low. Therefore, there is a relatively long interval duration between collecting static point clouds and sending static point clouds. At this time, when each radar device collects the static point clouds in its respective corresponding acquisition area, it can also process the static point clouds during the above interval duration to select the sending method of the static point clouds. And, the electronic device can also select the rendering method of the static point clouds every time it receives the static point clouds at the second frequency. Specifically, details are as follows:
[0057] The first method:
[0058] Multiple radar devices are used to splice the local static point clouds collected by each of them into an overall static point cloud, and after equally dividing the overall static point cloud into a preset number of divided static point clouds, the divided static point clouds are sequentially sent to the electronic device at the second frequency.
[0059] The electronic device is used to splice the sequentially received divided static point clouds into an overall static point cloud and then render it; or, it is used to render the received divided static point clouds in real time.
[0060] It should be noted that there is a corresponding relationship between the sending frequency, receiving frequency, and rendering frequency of the static point clouds. For example, when the radar device sends the static point clouds at the second frequency, the frequency at which the electronic device receives and renders the static point clouds should also be the second frequency.
[0061] It should be added that multiple target frequencies can be set in the second frequency, and each static point cloud sending method can select different target frequencies for sending. Moreover, each target frequency in the second frequency should be less than the first frequency. Exemplarily, in the first method, when the radar device sends the segmented static point cloud, it can use the first target frequency in the second frequency for sending.
[0062] In one embodiment, since multiple point cloud acquisition devices respectively acquire the point clouds of different acquisition areas in the detection area, the radar device can splice the local static point clouds corresponding to each acquisition area to obtain the overall static point cloud of the detection area.
[0063] Among them, the above preset quantity can be set in advance according to the actual situation, and there is no limitation on this. Exemplarily, when the above preset quantity is 7, the overall static point cloud can be divided into 7 groups of segmented static point clouds according to the preset segmentation method. Then, the segmented static point clouds are respectively sent to the rendering device using the first target frequency.
[0064] Among them, the preset segmentation method includes but is not limited to the vertical segmentation method, the horizontal segmentation method, and there is no limitation on this.
[0065] Exemplarily, Figure 5 is a schematic diagram of the application scenario of the detection area in a point cloud rendering method provided by another embodiment of the present application. After obtaining the overall static point cloud, the vertical segmentation method can be used to evenly divide the overall static point cloud into 7 groups. Each group of static point clouds respectively represents a partial scene of different areas in the scene to be rendered. Among them, Figure 5 the sizes of the rectangular areas surrounded by the white lines in are the same, and the point clouds included in each rectangular area can be considered as the segmented static point clouds after equal-proportion segmentation.
[0066] After that, after the electronic device successively receives the segmented static point clouds at every first target frequency, it can first splice the segmented static point clouds into the overall static point cloud, and then render the overall static point cloud of the detection area simultaneously. Or, the electronic device can render the segmented static point clouds received at every first target frequency in real time.
[0067] It should be noted that the radar device can select the sending method of sending one group of segmented static point clouds to the rendering device at every first target frequency until each group of segmented static point clouds is sent in sequence, so as to further reduce the transmission bandwidth required for each sending while reducing the sending frequency of the static point cloud. Moreover, when the electronic device renders after splicing the segmented static point clouds into the overall static point cloud, the number of times the static point cloud needs to be rendered can be reduced. Or, when the electronic device renders the segmented static point clouds in real time, the rendering overhead required for each static point cloud rendering can be reduced.
[0068] The second method:
[0069] Multiple radar devices are used to sequentially send the local static point clouds collected by each radar device to the electronic device at a second frequency according to a preset sending order.
[0070] The electronic device is used to splice the sequentially received local static point clouds into an overall static point cloud and then perform rendering; or, it is used to perform real-time rendering on the received local static point clouds;
[0071] In one embodiment, as described above, each of the above radar devices may be pre-set with identification information capable of unique identification. Therefore, the radar device may set the above preset sending order based on the identification information. Or, the preset sending order is used as the identification information.
[0072] Among them, similar to the above first method, the radar devices may also collect and send the local static point clouds of their respective corresponding areas at the first target frequency in the second frequency. And, after the electronic device sequentially receives the local static point clouds at every first target frequency, it may first splice the local static point clouds into an overall static point cloud, and then simultaneously render the static point clouds of the detection area. Or, the electronic device may perform real-time rendering on the segmented static point clouds received at every first target frequency, which will not be elaborated here.
[0073] The third method:
[0074] Multiple radar devices are used to splice the local static point clouds they have collected into an overall static point cloud, and send the overall static point cloud to the electronic device at a second frequency.
[0075] The electronic device is used to perform real-time rendering on the received overall static point cloud.
[0076] In one embodiment, different from the above first method and second method, multiple radar devices may first splice the local static point clouds they have collected to obtain the overall static point cloud of the detection area. Then, it is sent to the electronic device at a second frequency. At this time, since the electronic device receives the overall static point cloud at every second frequency. Therefore, the electronic device can perform real-time rendering on the received overall static point cloud.
[0077] Among them, since the radar device sends the overall static point cloud, there is no need for the electronic device to perform secondary splicing. Based on this, in this embodiment, the radar device may send the static point cloud at the second target frequency in the second frequency. At this time, the second target frequency may be lower than the first target frequency of the above first method and second method.
[0078] It should be noted that although sending the overall static point cloud at a second target frequency with a lower frequency cannot reduce the transmission bandwidth required for each transmission, compared with the first method and the second method, the third method can further reduce the transmission frequency of the static point cloud (i.e., fewer transmission times).
[0079] In summary, by using the above-mentioned multiple methods to send the static point cloud at a low frequency, not only can the transmission bandwidth required when sending the point cloud be reduced, but also the rendering device can receive and render the detection area corresponding to the static point cloud at a low frequency, reducing the rendering overhead.
[0080] In another embodiment, after rendering the dynamic point cloud and the static point cloud, the electronic device can also store the rendered dynamic point cloud and static point cloud. At this time, in order to make the pictures rendered by the simultaneously stored dynamic point cloud and static point cloud conform to the actual scene and facilitate subsequent querying of the stored dynamic point cloud and static point cloud, the electronic device can determine the reception time of each overall static point cloud. Then, for any dynamic point cloud, the dynamic point cloud is spliced with the overall static point cloud corresponding to the latest reception time among the multiple reception times to obtain the overall area point cloud and store it.
[0081] It should be noted that since the electronic device needs to render and update the overall static point cloud after receiving it. Therefore, it can be considered that the overall static point cloud corresponding to the latest reception time among the multiple reception times is the latest overall static point cloud received and spliced by the electronic device at this time.
[0082] Moreover, since the update rendering frequency of the overall static point cloud is relatively low, the overall static point cloud will be spliced with the dynamic point clouds obtained at multiple moments to obtain the overall area point clouds at multiple moments. However, based on the dynamic and static characteristics of the static objects and dynamic objects described above, even if the dynamic point cloud and the overall static point cloud are not point clouds obtained at the same time, the spliced overall static point cloud can still make the rendered picture conform to the actual scene.
[0083] In another embodiment, the electronic device can also obtain the point cloud information of the dynamic point cloud and the static point cloud, and then store the dynamic point cloud and the static point cloud according to the point cloud information.
[0084] Among them, the point cloud information includes, but is not limited to, one or more of various information such as the original frame number of the point cloud, type (types such as dynamic point cloud or static point cloud), source (identification information of the point cloud acquisition device for acquiring the point cloud), and the acquisition method of the static point cloud.
[0085] Exemplarily, when the point cloud information only includes any one of the above, the electronic device may store the dynamic point cloud and the static point cloud with the same frame number in the point cloud data frame; or, store the dynamic point cloud and the static point cloud separately; or, store the dynamic point cloud and the static point cloud collected by the same point cloud acquisition device; or, store the local static point cloud, the segmented static point cloud, and the overall static point cloud separately according to the acquisition method of the static point cloud. In this embodiment, the method for storing the dynamic point cloud and the static point cloud is not limited.
[0086] Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of a point cloud rendering device provided by an embodiment of the present application. Each module included in the point cloud rendering device in this embodiment is used to execute Figure 1 the corresponding steps in the corresponding embodiment. Specifically, please refer to Figure 1 and Figure 1 the relevant descriptions in the corresponding embodiments. For the sake of convenience of description, only the parts related to this embodiment are shown. Refer to Figure 6 , the point cloud rendering device 600 may include: a dynamic point cloud processing module 610 and a static point cloud processing module 620, where:
[0087] The dynamic point cloud processing module 610 is used to receive the dynamic point cloud at a first frequency and perform rendering, and display the rendered dynamic point cloud on the display interface according to the position information of the dynamic point cloud; wherein, the dynamic point cloud is the point cloud of the road surface area collected by the radar device or the point cloud of the moving target in the road surface area; the display interface is used to display the point cloud of the detection area of the radar device, and the position points in the display interface correspond to the position points of the detection area; the detection area includes the road surface area and the non-road surface area.
[0088] The static point cloud processing module 620 is used to receive the static point cloud at a second frequency and perform rendering, and display the rendered static point cloud on the display interface according to the position information of the static point cloud; wherein, the static point cloud is the point cloud of the non-road surface area collected by the radar device or the point cloud other than the point cloud of the moving target in the road surface area, and the second frequency is less than the first frequency.
[0089] In one embodiment, the radar device is used to extract the point cloud of the moving target from each point cloud collected from the road surface area as the dynamic point cloud, and determine the point cloud other than the point cloud of the moving target in the road surface area as the static point cloud; or,
[0090] The radar device is used to determine the point cloud collected from the road surface area as the dynamic point cloud, and determine the point cloud from the non-road surface area as the static point cloud.
[0091] In one embodiment, the point cloud rendering device 600 is used for multi-radar device joint global detection, and there is a corresponding relationship between the transmission frequency, the reception frequency, and the rendering frequency of the static point cloud:
[0092] Multiple radar devices are used to splice the local static point clouds collected by each of them into an overall static point cloud, and after equally dividing the overall static point cloud into a preset number of segmented static point clouds, the segmented static point clouds are sequentially sent to the electronic device at a second frequency;
[0093] The electronic device is used to splice the sequentially received segmented static point clouds into an overall static point cloud and then perform rendering; or, it is used to perform rendering on the received segmented static point clouds in real time;
[0094] Multiple radar devices are used to send the dynamic point clouds collected by each of them to the electronic device at a first frequency;
[0095] The electronic device is used to perform rendering on the received dynamic point clouds in real time.
[0096] In one embodiment, the point cloud rendering device 600 is used for multi-radar device joint global detection, and there is a corresponding relationship among the transmission frequency, reception frequency, and rendering frequency of the static point cloud:
[0097] Multiple radar devices are used to sequentially send the local static point clouds collected by each radar device to the electronic device at a second frequency according to a preset transmission order;
[0098] The electronic device is used to splice the sequentially received local static point clouds into an overall static point cloud and then perform rendering; or, it is used to perform rendering on the received local static point clouds in real time;
[0099] Multiple radar devices are used to send the dynamic point clouds collected by each of them to the electronic device at a first frequency;
[0100] The electronic device is used to perform rendering on the received dynamic point clouds in real time.
[0101] In one embodiment, the point cloud rendering device 600 is used for multi-radar device joint global detection, and there is a corresponding relationship among the transmission frequency, reception frequency, and rendering frequency of the static point cloud:
[0102] Multiple radar devices are used to splice the local static point clouds collected by each of them into an overall static point cloud and send the overall static point cloud to the electronic device at a second frequency;
[0103] Multiple radar devices are used to send the dynamic point clouds collected by each of them to the electronic device at a first frequency;
[0104] The electronic device is used to perform rendering on the received overall static point cloud and dynamic point clouds in real time.
[0105] In one embodiment, the point cloud rendering device 600 further includes:
[0106] An acquisition module, configured to acquire point cloud information of dynamic point clouds and static point clouds.
[0107] A first storage module, configured to store dynamic point clouds and static point clouds according to the point cloud information.
[0108] In one embodiment, the point cloud rendering device 600 further includes:
[0109] A determination module, configured to determine the splicing moments of receiving respective overall static point clouds.
[0110] A splicing module, configured to, for any dynamic point cloud, splice the dynamic point cloud with the overall static point cloud corresponding to the latest receiving moment among multiple receiving moments, to obtain an overall area point cloud corresponding to a detection area.
[0111] A second storage module, configured to store the overall area point cloud.
[0112] It should be understood that Figure 6 in the structural schematic diagram of the shown point cloud rendering device, each module is configured to execute Figure 1 the respective steps in the corresponding embodiment, and for Figure 1 the respective steps in the corresponding embodiment, they have been explained in detail in the above embodiments. For details, please refer to Figure 1 and Figure 1 the relevant descriptions in the corresponding embodiment, which will not be elaborated here.
[0113] Figure 7 is a structural schematic diagram of an electronic device provided in an embodiment of the present application. As Figure 7 shown, the electronic device 700 in this embodiment includes: a processor 710, a memory 720, and a computer program 730 stored in the memory 720 and executable on the processor 710, such as a program for the point cloud rendering method. Specifically, the electronic device 700 is connected to a radar device 800. When the processor 710 executes the computer program 730, it implements the steps in the respective embodiments of the above various point cloud rendering methods, such as Figure 1 S101 to S102 shown. Or, when the processor 710 executes the computer program 730, it implements the functions of each module in the above Figure 6 corresponding embodiment, for example, Figure 6 the functions of each module shown. For details, please refer to Figure 6 the relevant descriptions in the corresponding embodiment.
[0114] Exemplarily, the computer program 730 can be divided into one or more modules. One or more modules are stored in the memory 720 and executed by the processor 710 to implement the point cloud rendering method provided by the embodiments of the present application. One or more modules can be a series of computer program instruction segments capable of completing specific functions, and these instruction segments are used to describe the execution process of the computer program 730 in the electronic device 700. For example, the computer program 730 can implement the point cloud rendering method provided by the embodiments of the present application.
[0115] The electronic device 700 may include, but is not limited to, the processor 710 and the memory 720. Those skilled in the art can understand that Figure 7 merely examples of the electronic device 700, which do not constitute a limitation on the electronic device 700, may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.
[0116] The so-called processor 710 may be a central processing unit, or may also be other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0117] The memory 720 may be an internal storage unit of the electronic device 700, such as the hard disk or memory of the electronic device 700. The memory 720 may also be an external storage device of the electronic device 700, such as a plug-in hard disk, a smart memory card, a flash memory card, etc. equipped on the electronic device 700. Further, the memory 720 may also include both the internal storage unit and the external storage device of the electronic device 700.
[0118] The embodiments of the present application provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and the computer program is executed by the processor to perform the point cloud rendering method in the above-mentioned various embodiments.
[0119] The embodiments of the present application provide a computer program product. When the computer program product runs on an electronic device, the electronic device is caused to execute the point cloud rendering method in the above-mentioned various embodiments.
[0120] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included within the protection scope of the present application.
Claims
1. A point cloud rendering method, characterized in that, The method is applied to an electronic device, and the method includes: Receiving a dynamic point cloud at a first frequency and performing rendering, and displaying the rendered dynamic point cloud on a display interface according to the position information of the dynamic point cloud; wherein, the dynamic point cloud is a point cloud of a road surface area collected by a radar device or a point cloud of a moving target in the road surface area; the display interface is used to display the point cloud of the detection area of the radar device, and the position points in the display interface correspond to the position points of the detection area; the detection area includes the road surface area and a non-road surface area; Receiving a static point cloud at a second frequency and performing rendering, and displaying the rendered static point cloud on the display interface according to the position information of the static point cloud; wherein, the static point cloud is a point cloud of the non-road surface area collected by the radar device or a point cloud other than the point cloud of the moving target in the road surface area, and the second frequency is less than the first frequency.
2. The method according to claim 1, characterized in that, The method further includes: The radar device is configured to extract the point cloud of the moving target from each point cloud collected from the road surface area as the dynamic point cloud, and determine the point cloud other than the point cloud of the moving target in the road surface area as the static point cloud; or, The radar device is configured to determine the point cloud collected from the road surface area as the dynamic point cloud, and determine the point cloud from the non-road surface area as the static point cloud.
3. The method according to claim 1, wherein For multi-radar device joint global detection, there is a corresponding relationship among the transmission frequency, reception frequency, and rendering frequency of the static point cloud. The method further includes: Multiple radar devices are configured to splice the local static point clouds collected by each of them into an overall static point cloud, and after equally dividing the overall static point cloud into a preset number of divided static point clouds, sequentially send the divided static point clouds to the electronic device at the second frequency; The electronic device is configured to splice the sequentially received divided static point clouds into the overall static point cloud and then perform rendering; or, is configured to perform rendering on the received divided static point cloud in real time; Multiple radar devices are configured to send the dynamic point clouds collected by each of them to the electronic device at the first frequency; The electronic device is configured to perform rendering on the received dynamic point cloud in real time.
4. The method according to claim 1, characterized in that, For multi-radar device joint global detection, there is a corresponding relationship among the transmission frequency, reception frequency, and rendering frequency of the static point cloud. The method further includes: Multiple radar devices are configured to sequentially send the local static point clouds collected by each radar device to the electronic device at the second frequency according to a preset transmission order; The electronic device is configured to splice the sequentially received local static point clouds into the overall static point cloud and then perform rendering; or, is configured to perform rendering on the received local static point cloud in real time; Multiple radar devices are configured to send the dynamic point clouds collected by each of them to the electronic device at the first frequency; The electronic device is configured to perform rendering on the received dynamic point cloud in real time.
5. The method according to claim 1, characterized in that, For joint all-domain detection of multiple radar devices, there is a corresponding relationship among the transmission frequency, reception frequency, and rendering frequency of the static point cloud. The method further includes: Multiple of the radar devices are configured to splice the locally acquired static point clouds into an overall static point cloud and transmit the overall static point cloud to the electronic device at the second frequency; Multiple of the radar devices are configured to transmit the acquired dynamic point clouds to the electronic device at the first frequency; The electronic device is configured to render the received overall static point cloud and dynamic point cloud in real time.
6. The method according to any one of claims 1-5, characterized in that, The method further includes: Obtaining the point cloud information of the dynamic point cloud and the static point cloud; Storing the dynamic point cloud and the static point cloud according to the point cloud information.
7. The method according to any one of claims 3 to 5, characterized in that, The method further includes: Determining the splicing time when each of the overall static point clouds is received; For any one of the dynamic point clouds, splicing the dynamic point cloud with the overall static point cloud corresponding to the latest reception time among the multiple reception times to obtain the overall regional point cloud corresponding to the detection area; Storing the overall regional point cloud.
8. A point cloud rendering device, characterized in that, The apparatus is applied to an electronic device. The apparatus includes: A dynamic point cloud processing module, configured to receive the dynamic point cloud at the first frequency and perform rendering, and display the rendered dynamic point cloud on the display interface according to the position information of the dynamic point cloud; wherein, the dynamic point cloud is the point cloud of the road surface area collected by the radar device or the point cloud of a moving target in the road surface area; the display interface is used to display the point cloud of the detection area of the radar device, and the position points in the display interface correspond to the position points of the detection area; the detection area includes the road surface area and the non-road surface area; A static point cloud processing module, configured to receive the static point cloud at the second frequency and perform rendering, and display the rendered static point cloud on the display interface according to the position information of the static point cloud; wherein, the static point cloud is the point cloud of the non-road surface area collected by the radar device or the point cloud other than the point cloud of the moving target in the road surface area, and the second frequency is less than the first frequency.
9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The electronic device is connected to the radar device. When the processor executes the computer program, it implements the method according to any one of claims 1 to 7 with the radar device.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the method according to any one of claims 1 to 7.