A visualization method for heterogeneous variable-resolution point cloud imaging
By constructing a heterogeneous variable resolution point cloud imaging lidar model and combining with the Unity3D platform, real-time visual display of heterogeneous variable resolution point cloud imaging is realized, solving the problem of heterogeneous variable resolution point cloud imaging visualization in the existing technology. , realizes high-precision three-dimensional image fusion display, reducing the difficulty of development.
Patent Information
- Application Number
- CN202210257552.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-16
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-03-16
AI Technical Summary
The existing technology has not effectively realized the visualization of heterogeneous variable resolution point cloud imaging, especially the hardware configuration of laser three-dimensional imaging of human-like retinal structures is high and the development is difficult. Related software is lacking in the Visual support.
By constructing a heterogeneous variable resolution point cloud imaging lidar model, using the C# environment and Unity3D platform, combined with multi-layer rendering technology, the heterogeneous variable resolution point cloud imaging scanning of the target object and the fusion of visible light bands and point cloud data are achieved. Display, display high-precision three-dimensional images in real time.
Real-time visual display of heterogeneous variable resolution point cloud imaging is realized, which meets the data visualization needs of high-precision models, reduces the difficulty of design and development, and improves the efficiency of discovering design problems.
Smart Images

Figure CN114638942B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of photoelectric imaging, and in particular to a heterogeneous variable-resolution point cloud imaging visualization method. Background Art
[0002] With the continuous improvement of computer computing power, three-dimensional visualization technology has flourished in recent years. The Unity3D engine developed based on computer graphics has unique advantages in data display, scene roaming, visual simulation and other fields.
[0003] In recent years, laser 3D imaging technology has been widely used in visual navigation, aerospace, surveying and mapping and other fields due to its advantages of high resolution, simple principle and long detection distance. With the deepening of research, relevant researchers have proposed a laser 3D imaging method based on the structure of the human eye retina on the basis of studying human vision and laser 3D imaging technology. While taking into account a large field of view, it can also perform high-resolution imaging of the target of interest. Therefore, heterogeneous variable-resolution 3D point cloud imaging based on the human eye retina can be applied to large field of view, real-time, and high-resolution occasions, such as: unmanned driving, intelligent monitoring, and unmanned logistics. Conventional 3D point cloud imaging has a relatively mature visualization platform, but the current human eye laser radar has high hardware configuration requirements and is difficult to develop. It has not been widely used, and there is no relevant software to visualize heterogeneous variable-resolution point clouds. At the same time, visualization also has good support for the research and development of heterogeneous variable-resolution imaging equipment. By visualizing the hardware system to be designed, it is easier to experience the final design product effect, so that design problems can be discovered in time.
[0004] Therefore, it is crucial to develop heterogeneous variable-resolution three-dimensional point cloud imaging visualization. Summary of the invention
[0005] The purpose of the present invention is to provide a heterogeneous variable resolution point cloud imaging visualization method to solve the problems existing in the above-mentioned prior art, to construct a heterogeneous variable resolution point cloud imaging laser three-dimensional imaging system and to complete the real-time display of heterogeneous variable resolution point cloud imaging point clouds. In addition, based on a high-precision three-dimensional model, a three-dimensional image fusion visualization display of visible light bands and point cloud data is realized.
[0006] To achieve the above object, the present invention provides the following solution: The present invention provides a heterogeneous variable resolution point cloud imaging visualization method, comprising the following steps:
[0007] Construct a heterogeneous variable-resolution point cloud imaging lidar model;
[0008] Using the heterogeneous variable resolution point cloud imaging laser radar model, performing heterogeneous variable resolution point cloud imaging scanning on the target object, collecting point cloud data, and displaying the point cloud data;
[0009] The visible light band and the point cloud data are fused and displayed.
[0010] Optionally, a C# environment is used when constructing the heterogeneous variable-resolution point cloud imaging lidar model.
[0011] Optionally, constructing the heterogeneous variable-resolution point cloud imaging lidar model includes:
[0012] A mathematical and physical model of a heterogeneous variable-resolution point cloud imaging lidar is constructed and converted using the C# language to obtain a preliminary heterogeneous variable-resolution point cloud imaging lidar model. The preliminary heterogeneous variable-resolution point cloud imaging lidar model is mounted into the Unity3D environment to obtain the heterogeneous variable-resolution point cloud imaging lidar model.
[0013] Optionally, using the heterogeneous variable resolution point cloud imaging lidar model to perform heterogeneous variable resolution point cloud imaging scanning on the target object includes:
[0014] In the heterogeneous variable-resolution point cloud imaging lidar model, the number of scanning rings M and the number of scanning points N in each ring are set, and a scanning plan is calculated based on the number of rings M and the number of scanning points N, and the target object is scanned based on the scanning plan.
[0015] Optionally, scanning the target object based on the scanning scheme includes:
[0016] According to the scanning scheme, laser pulses are emitted to scan the target object, the depth information of the target point is obtained and the input vector group is saved, and it is determined whether the last scanning point has been executed. If yes, a frame of image scanning is completed, and if not, the laser beam is adjusted to the next scanning point to continue scanning.
[0017] Optionally, displaying the point cloud data includes:
[0018] According to the collected point cloud data, the size and color of the point cloud data are adjusted for real-time display.
[0019] Optionally, fusing and displaying the visible light band and the point cloud data includes:
[0020] Identify a rendering object, load the rendering object, and set a rendering material and texture corresponding to the rendering object;
[0021] Perform vertex coordinate transformation and clipping on the rendered primitives to obtain vertex coordinates and texture coordinates;
[0022] Convert the vertex coordinates and texture coordinates into pixel coordinates, generate pixels based on the pixel coordinates, and form a picture;
[0023] The rendering layer and the point cloud rendering layer are fused and calculated to realize the fusion display of the visible light band and the point cloud data.
[0024] The present invention discloses the following technical effects:
[0025] The present invention discloses a method for visualizing heterogeneous variable-resolution point cloud imaging. The method is based on the Unity3D platform and combines the C# programming language to realize real-time visualization of data. The multi-layer rendering of Unity3D is applied to realize the fusion display of the scanned high-precision model in the visible light band and the point cloud data, thus meeting the data visualization requirements in the research of heterogeneous variable-resolution point cloud imaging. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0027] Figure 1 Schematic diagram of the process of the heterogeneous variable resolution point cloud imaging visualization method in an embodiment of the present invention;
[0028] Figure 2 A scanning scene diagram of the heterogeneous variable-resolution point cloud imaging visualization method in an embodiment of the present invention;
[0029] Figure 3 This is a flow chart of target scanning of heterogeneous variable resolution point cloud imaging in an embodiment of the present invention;
[0030] Figure 4 It is a flow chart of visualization of individual point cloud imaging and fusion display of visible light band and point cloud data in heterogeneous variable resolution point cloud imaging in an embodiment of the present invention;
[0031] Figure 5 Schematic diagram of heterogeneous variable resolution point cloud imaging scanning based on the MOEMS scanning system in an embodiment of the present invention;
[0032] Figure 6 A separate point cloud display of the heterogeneous variable resolution point cloud imaging visualization method in an embodiment of the present invention;
[0033] Figure 7 It is a fusion display of the visible light band and point cloud data of the heterogeneous variable resolution point cloud imaging visualization method in an embodiment of the present invention. DETAILED DESCRIPTION
[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0035] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0036] The present invention provides a heterogeneous variable resolution point cloud imaging visualization method, such as Figure 1 As shown, including:
[0037] Step 1: Build a heterogeneous variable resolution point cloud imaging lidar model based on the C# environment.
[0038] like Figure 2 The figure shows a heterogeneous variable-resolution point cloud imaging lidar model and a scanned target vehicle model.
[0039] The laser echo power P returned from the target object is received at the photosensitive surface receiving end of the photoelectric detection element. r for:
[0040]
[0041] In the above formula, ρ is the reflectivity of the target object being detected, P L is the peak power of the transmitting laser, T E is the transmittance of the optical system used to collimate the laser beam, T R A is the transmittance of the optical system used by the receiving end to converge the echo signal on the photosensitive surface of the detection element. R is the area of the receiving unit at the receiving end, δ is the angle between the optical axis of the optical system lens at the transmitting end and the normal of the target object surface, σ is the attenuation coefficient of the atmosphere, and R is the distance between the laser radar and the surface of the detected target.
[0042] From the expression of echo power, we can know that the echo power of laser beam is affected by many factors. Under the condition of other constants, the echo power decreases rapidly with the increase of detection distance. When the echo power is as small as the minimum detectable power P of the receiver at the receiving end, r =P min When the distance between the target and the laser radar is the maximum detectable distance R of the laser radar system max :
[0043]
[0044] In the Unity3D engine, the scene supports a variety of externally imported 3D model formats and has very strong compatibility. Among them, the supported formats for networks, materials, animations, and skeletons include Maya's .mb and .mal formats, 3D Studio Max's .maxl format, Cheetah3D's .jasl format, Cinema 4D's .c4dl 2 format, Blender's .blendl format, etc.
[0045] In this embodiment, the present invention applies a heterogeneous variable resolution point cloud imaging lidar model built in 3D Studio Max and imports it into the scene. Then, a high-precision vehicle 3D is selected from the official AssetStore and the corresponding algorithm structure is mounted.
[0046] First, create a model for storing the specific parameter information of the lidar, and then set ρ (the reflectivity of the detected target object) at the port. L (peak power of the laser at the transmitting end), T E (Transmittance of the optical system used to collimate the laser beam), T R (transmittance of the optical system used by the receiving end to converge the echo signal on the photosensitive surface of the detection element), A R (area of the receiving unit at the receiving end), δ (angle between the optical axis of the optical system lens at the transmitting end and the normal of the target object surface), σ (atmospheric attenuation coefficient), R (distance between the laser radar and the detected target surface), the model receives the preset parameters and calculates the maximum detectable distance R of the laser radar system max , and use this detection distance as the threshold of the laser radar detection to filter the point cloud data. If the echo power is lower than the minimum detection power of the receiver, the signal will not be collected. In this embodiment, it is assumed that the minimum detectable power P of the photoelectric detection element is min 1×10 -11 W, the reflectivity of the detected object ρ = 0.2, the area of the receiving unit at the receiving end A R =0.22mm 2 ,δ=0.25, transmittance T of the optical system for laser beam collimation E =0.9, the transmittance T of the optical system used by the receiving end to converge the echo signal on the photosensitive surface of the detection element R =0.8. When the maximum detectable distance of the laser radar is 120m, e -σR = 0.9, from which we can calculate that the emission power of the laser diode is about 20W, and finally the maximum detectable distance R of the laser radar system is calculated maxTherefore, the laser radar model uses this detection distance as the threshold of the laser radar detection to filter the point cloud data. If the echo power is lower than the minimum detection power of the receiver, the signal will not be collected.
[0047] Step 2: Heterogeneous variable-resolution point cloud imaging scanning based on the MOEMS scanning system.
[0048] In this embodiment, in order to achieve coverage scanning of the target vehicle model, by combining Figure 5 The schematic diagram of heterogeneous variable resolution point cloud imaging scanning based on the MOEMS scanning system selects the corresponding M and N to scan the target vehicle.
[0049] In the environment of C# programming language and Unity3D, this embodiment converts the mathematical and physical model of the corresponding heterogeneous variable resolution point cloud imaging lidar model into a C# algorithm and inherits the MonoBehaviour parent class and loads it into the model.
[0050] To illustrate the scanning process of heterogeneous variable resolution point cloud imaging, combined with Figure 3 The flowchart of heterogeneous variable resolution point cloud imaging target scanning is further explained. After the algorithm starts running, the laser radar model is programmed according to the set M and N values and controlled to perform heterogeneous variable resolution point cloud imaging scanning according to the arrangement of photoreceptor cells in the human eye. The target is detected by simulating the emission of laser pulses, and the target depth information obtained by a single scan is stored in the Vector3 vector group. It is determined whether the last scanning point has been reached. If not, the laser beam is adjusted to the next scanning point until the scanning of a frame of image is completed.
[0051] The visualization method based on C# programming environment and Unity3D platform can achieve very good real-time performance. In the above steps, the heterogeneous variable resolution point cloud imaging laser radar model has completed the scanning process of the target vehicle. The next step is to display the point cloud data stored in the Vector3 vector group in real time. In order to realize real-time scanning and display of point cloud data, the present invention will display the scanned vehicle target point cloud data while the heterogeneous variable resolution point cloud imaging laser radar is scanning, instead of displaying it after completing a frame of the picture. In this embodiment, M represents the number of scanned circles, and N represents the number of points scanned in each circle, where M is set to 7 and N is set to 21. The laser radar will scan point by point from the innermost circle each time. After completing a circle of scanning, the scanning point will expand outward by one unit until the scanning of the last point of the last circle is completed. The algorithm will reset the scanning point to the initial point, and a frame of point cloud image is completed.
[0052] Step 3: Heterogeneous variable resolution point cloud imaging scanning visualization.
[0053] In order to realize the visualization of heterogeneous variable resolution point cloud imaging scanning, the target to be detected must first be scanned by the constructed laser radar model. The model forms a detection threshold according to the preset laser radar parameters to detect the target. Different from the traditional circular scanning laser radar, the laser radar model in this embodiment scans the target object in the form of heterogeneous variable rate, that is, the central area will be scanned with high resolution. In order to realize this form of scanning imaging, in this embodiment, a heterogeneous variable resolution scanning based on the MOEMS scanning form is constructed, and the point cloud data that can be displayed is screened out by the threshold and then visualized. The threshold in this step is determined by the maximum detection distance of the laser radar calculated in step 1, that is, the maximum detection distance of the laser radar calculated in step 1 is the threshold of point cloud acquisition. The virtual three-dimensional space in the Unity3D environment can obtain the distance between the detection point and the laser radar. By comparing this distance with the maximum detection distance of the laser radar, the point cloud data can be screened.
[0054] Specifically, when the LiDAR model algorithm is running, the LiDAR model is initialized first, that is, the script code will first execute the content in Start(), which contains some content of the initialization parameters of the heterogeneous variable resolution LiDAR, and in a complete execution cycle, this part of the code will only be executed once. The code for the mathematical and physical model of the heterogeneous variable resolution LiDAR model will be placed in FixedUpdate() for execution. FixedUpdate() will run stably according to the controlled frequency, while Update() will be executed once per frame. For different scenes, the execution pressure of the computer will also be different, and the operation of the visualization scene will often not be stable at a certain frame rate, but will have a drift, which will be extremely unfavorable for accurate physical operations.
[0055] After completing the construction of the heterogeneous variable resolution point cloud imaging lidar model, the next step is to save the target point cloud data scanned by the lidar each time and render the point cloud data in three-dimensional space. In the rendering of point cloud data, ParticleSystem inherited from Component in UnityEngine is applied.
[0056] Combination Figure 4 , further illustrating the visualization of individual point cloud imaging in heterogeneous variable resolution point cloud imaging, the algorithm starts running and the process Figure 3 The collected Vector3 type point cloud vector group will be loaded into the process Figure 4 In, as Figure 4The first input item, the point cloud vector group contains the XYZ position information of the target, and then the point cloud vector group will be loaded into the ParticleSystem. In the ParticleSystem, the properties of the particles are grouped according to modules, such as the particle noise system and the particle emission system. These properties are in the form of structures, but they are different from normal C# structures. The most critical difference is that when setting the properties on the module structure, there is no need to assign the structure back to the particle system component, because Unity3D will immediately assign the variables to the ParticleSystem. Therefore, pay attention to the difference when operating each point cloud data. In this embodiment, the visualization system used to display the point cloud can realize the adjustment of the size, color and other parameters of the point cloud data, which can be adjusted directly in the user operation panel. At the same time, the system will judge by the index. After completing the loading of the last target point data, the system will self-check and initialize the settings, and render the collected point cloud data in real time. Figure 6 shown.
[0057] Step 4: Fusion display of visible light band and point cloud data.
[0058] In Unity3D rendering, completing the display of each frame usually requires going through the application stage, geometry stage, and rasterization stage.
[0059] The application stage mainly identifies potential rendering objects, loads them into memory, and sets the materials and textures that need to be rendered.
[0060] The geometry stage is mainly responsible for performing vertex coordinate transformation and clipping on the rendering primitives transmitted from the CPU, and transmitting the processed vertex coordinates, texture coordinates, etc. to the rasterizer.
[0061] The rasterization stage receives the vertex coordinates, texture coordinates and other data processed in the geometry stage, and converts these geometric data into pixel coordinates on the screen, thereby generating pixels and forming a picture from these pixels.
[0062] By fusing the model rendering layer with the point cloud rendering layer in Unity3D, the fusion of the visible light band and the point cloud data can be finally achieved. Figure 7As shown. In this embodiment, when the laser radar model scans the target, it will also return the color information of the scanning point. In Unity3D, the color information of the object is represented by four channels, including three RGB channels and one channel for adjusting transparency. Each channel will be represented by a Byte value of 0-255, and then the color value will be deviated according to the set fusion model to finally obtain a visualized point cloud. If in the example, because the vehicle model is close to the laser radar, the weights of the RGB and transparency channels are set to 1 when performing point cloud fusion visualization. If facing a large scene, the visible light received from the sensor starts from the target and passes through a certain atmospheric medium, and the wavelength will change to a certain extent, then the weight can be changed accordingly.
[0063] The present invention discloses a method for visualizing heterogeneous variable-resolution point cloud imaging. A mathematical and physical model of a laser radar that simulates the circular scanning of human eyes is constructed in Unity3D based on C#, and the target object is scanned by using the laser radar model. At the same time, the point cloud data scanned to the target is rendered and displayed in real time by using the particle system of Unity3D, and the fusion display of visible light band and point cloud data can be realized.
[0064] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A method for visualizing heterogeneous variable-resolution point cloud imaging. It is characterized in that The following steps are involved: Construct a heterogeneous variable-resolution point cloud imaging lidar model; Using the heterogeneous variable resolution point cloud imaging laser radar model, performing heterogeneous variable resolution point cloud imaging scanning on the target object, collecting point cloud data, and displaying the point cloud data; fusing the visible light band with the point cloud data for display; When constructing the heterogeneous variable-resolution point cloud imaging lidar model, the C# environment is used; Constructing the heterogeneous variable resolution point cloud imaging lidar model includes: Construct a mathematical and physical model of a heterogeneous variable-resolution point cloud imaging laser radar, convert it using the C# language, obtain a preliminary heterogeneous variable-resolution point cloud imaging laser radar model, mount the preliminary heterogeneous variable-resolution point cloud imaging laser radar model into the Unity3D environment, and obtain the heterogeneous variable-resolution point cloud imaging laser radar model; Using the heterogeneous variable resolution point cloud imaging laser radar model to perform heterogeneous variable resolution point cloud imaging scanning on the target object includes: In the heterogeneous variable-resolution point cloud imaging laser radar model, the number of scanning rings M and the number of scanning points N of each ring are set, a scanning scheme is calculated based on the number of rings M and the number of scanning points N, and the target object is scanned based on the scanning scheme; The fusion display of the visible light band and the point cloud data includes: Identify a rendering object, load the rendering object, and set a rendering material and texture corresponding to the rendering object; Perform vertex coordinate transformation and clipping on the rendered primitives to obtain vertex coordinates and texture coordinates; Convert the vertex coordinates and texture coordinates into pixel coordinates, generate pixels based on the pixel coordinates, and form a picture; The rendering layer and the point cloud rendering layer are fused and calculated to realize the fusion display of the visible light band and the point cloud data.
2. The heterogeneous variable resolution point cloud imaging visualization method according to claim 1, It is characterized in that Scanning the target object based on the scanning scheme includes: According to the scanning scheme, laser pulses are emitted to scan the target object, the depth information of the target point is obtained and the input vector group is saved, and it is determined whether the last scanning point has been executed. If yes, a frame of image scanning is completed, and if not, the laser beam is adjusted to the next scanning point to continue scanning.
3. The heterogeneous variable resolution point cloud imaging visualization method according to claim 1, It is characterized in that Displaying the point cloud data comprises: According to the collected point cloud data, the size and color of the point cloud data are adjusted for real-time display.
