A method for building a software framework for visualizing and processing PCD format point cloud data

Abstract

This invention provides a method for building a software framework for PCD format point cloud data visualization processing. Addressing the limitation of existing technologies that generally only support simple reading of PCD format files, it provides a development method for point cloud data visualization processing programs that can be ported and run across platforms such as Linux and Windows. It can comprehensively realize various functions such as point cloud data visualization editing, modification, and storage, thereby greatly expanding the application scenarios and processing depth of PCD format point cloud data.

Description

Technical Field

[0001] This invention belongs to the field of 3D point cloud data processing and visualization technology, specifically relating to a method for building a PCD format point cloud data visualization processing software framework. Background Technology

[0002] In the field of 3D point cloud data processing, the PCD (Point Cloud Data) file format is widely used due to its advantages in storing ordered point cloud datasets, fast binary data type reading and writing, and support for different data types. However, the PCD file format has relatively limited functionality in visual editing. Currently, most PCD files only support basic creation and reading operations, lacking the ability to perform in-depth editing and manipulation of existing 3D point cloud data. For example, while users can create PCD files and store the XYZ coordinates, intensity information, and color information (RGB) of point sets, existing tools often fall short when it comes to more complex editing tasks, such as adding, deleting, modifying, and measuring distances in point clouds, as well as performing advanced point cloud filtering and coordinate transformations.

[0003] Furthermore, existing PCD file processing tools typically do not support dynamic visualization and real-time editing. This limits the user's interactive experience and the depth of application scenarios in point cloud data processing, forcing them to resort to complex and expensive professional software and hardware, which also hinders the popularization and application of 3D point cloud data. Therefore, it is necessary to develop a new software tool to overcome the limitations of existing technologies and provide richer and more in-depth point cloud data processing capabilities. Summary of the Invention

[0004] In view of this, and to address the technical problems existing in this field, the present invention provides a method for building a software framework for visualizing and processing PCD format point cloud data, specifically including the following steps:

[0005] Step 1: Design the software user interface (UI) based on the QT framework and QT components. The programming language is C++. The PCL and VTK libraries are used as point cloud data processing tools and point cloud visualization frameworks, and the Eigen library is used as the software's built-in algorithm mathematical solution tool. First, create a new Qt Widgets Application class project file under the QT framework, and use the .pro file as the project configuration file to configure QT, project files, and third-party libraries including PCL, VTK, Eigen, and Boost libraries, so as to realize the portability and operation of the overall framework between Linux and Windows platforms.

[0006] Step 2: Based on the multi-threaded optimization design, an instance of the StartThread class is created to start the software. First, in the StartThread class constructor, a QApplication class object is created to manage the control flow and main settings of the GUI program. Then, a CtrlWindow class is instantiated to register and manage the software's user interface (UI). Next, a PCDEditor class object is created for registering and managing point cloud processing functions. Following this, a PCDEditorThread class object is created to encapsulate the sub-threads of the point cloud processing program. Then, a ViewerThread class object is created to register and manage point cloud visualization functions, while also encapsulating sub-threads. Finally, the show() function of the CtrlWindow parent class QWidget is called to display the software UI, and the start() function of the QThread class is called to start the parallel execution of the two sub-threads.

[0007] Step 3: Design the UI window within the QT framework using the ctrlwindow.ui layout file. First, visualize the ctrlwindow.ui window in QTDesign, and build UI components according to actual functional requirements to implement the relevant functions of PCD, including operation, processing, and display.

[0008] Step 4: Establish a communication mechanism between components and threads based on the signal and slot mechanism. This includes binding the UI components in the ctrlwindow.ui layout file with slot functions in the CtrlWindow class of the main thread to ensure that the interaction between components is achieved through a predefined event triggering strategy.

[0009] Step 5: Design the PCDEditor class, which runs under the PCL, VTK, and Eigen library framework and serves as the core of the sub-thread for point cloud data processing. Enable it to communicate with the CtrlWindow main thread and the ViewerThread sub-thread during runtime to respond to user interaction operations and update the point cloud data in real time.

[0010] Step 6: Design the ViewerThread class, which serves as the core of the point cloud visualization and user interaction mechanism. The ViewerThread class enables bidirectional inter-thread communication by subscribing to CtrlWindow and PCDEditor class objects in the constructor. This allows for interactive viewing, modification, and editing of point cloud data within the visualization window.

[0011] Furthermore, the specific functions implemented in step three for building the UI components include: using the Push Button component for the design and application of functions such as loading PCD, saving PCD, global reset, view reset, screen snapshot, export, global downsampling, filtering discrete points, pass-through filtering, global coordinate transformation, two-point distance measurement reset, coordinate axis visualization, and clearing process printouts; using the Check Box component for the design and application of functions such as deleting point clouds, adding point clouds, viewing point clouds, two-point distance measurement, and fixed top-down view; using the Table Widget component for viewing point clouds and displaying the results of two-point distance measurement; using the Horizontal Slider component for the development and application of functions such as single-point size modification and coordinate axis visualization; using the SpinBox and Double Spin Box components for setting hyperparameters for global downsampling, filtering discrete points, and pass-through filtering; using the Combo Box component for selecting modes for pass-through filtering and global coordinate transformation; and using the Text Edit component for displaying UI window description text and process printouts for some functional modules.

[0012] Furthermore, in step four, the CtrlWindow class maintains a series of functional flags and corresponding get and set functions. These functions are used to query and update the flags and other class member variables, thereby enabling inter-thread communication.

[0013] Furthermore, in step five, the PCDEditor class declares a pcl::PointCloud <pointt>The member variable pe_cloud of type ::Ptr is used to store global point cloud data; the child thread encapsulation of the PCDEditor instance pe is implemented in the constructor of the StartThread class. At the same time, the Run() function of PCDEditor can respond to the start() method of the parent class QThread of PCDEditorThread and execute the Run() method of the PCDEditorThread class, so that the Run() function of PCDEditor can run in the child thread to achieve parallel operation.

[0014] The point cloud processing task is continuously executed through an infinite loop function implemented in the `Run()` function. Its point cloud processing capabilities include three data management functions: `LoadPCDFile()` for loading point clouds, `SavePCDFile()` for storing point clouds, and `SaveAsFile()` for exporting point clouds to multiple formats, including PLY, PCDBinary, OBJ, and VTK; and four point cloud filtering modes: `AllPointsDownsize()` for global downsampling, allowing users to set voxel edge lengths; `AllPointsOutlierRemoval()` for filtering out discrete points based on user-defined hyperparameters such as the estimated number of points based on the average distance and the standard deviation factor; and `AllPointsPassT`. `hrough()` is used to perform pass-through filtering based on the filtering direction and range specified by the user; `AllPointsRemoveGround()` is used to filter out ground points based on the ground height and elevation threshold specified by the user; the `TfTrans()` function provides global coordinate transformation capabilities, supporting user-defined three-axis transformations; based on the infinite loop nature of the `Run()` function, this sub-thread periodically calls a series of related function `get` functions in the `CtrlWindow` main thread, using the `Flag` flag to monitor user requests. Once a user request is detected, the sub-thread immediately executes the corresponding function to meet the user's needs, and resets the `Flag` by calling the `CtrlWindow` `set` function, thus completing the current communication cycle.

[0015] Furthermore, when AllPointsRemoveGround() performs ground point filtering, it specifically uses inter-thread communication to call the getAllPointsRemoveGroundFlag() function of the CtrlWindow class instance cw to detect the user request corresponding to this function in real time. When a user request is captured, the getUi() function is called to capture the user interface class instance ui_, and the user-set ground height groundZ and elevation threshold ground_remove_threshold parameters are directly received from ui_. Then, the boundary of the point cloud data is determined, and then rasterization is performed on the XY plane at a specified resolution, initializing the maximum and minimum Z values ​​of each raster cell. The process involves traversing the point cloud and updating the Z-values ​​of the raster cells, calculating the Z-value change for each cell. If the Z-value change of a cell is less than the elevation threshold ground_remove_threshold, and the height of the point is close to groundZ, then the cell is marked as ground. Next, ground and non-ground point clouds are separated based on the raster marking results, thus filtering out ground points. Finally, the setAllPointsRemoveGroundFlag() function is called to reset and update the ground point filtering flag, fulfilling the user's request for ground point filtering. For point cloud data with different densities and terrain features, the resolution and threshold are adjusted to adapt, taking into account the actual ground height of the sensor during data acquisition when setting groundZ.

[0016] Furthermore, in step six, the ViewerThread class, as a subclass of QThread, directly calls the start() method of its parent class QThread in StartThread to execute the run() method of the ViewerThread class, thereby achieving parallel execution of multiple threads. The ViewerThread class declares and maintains a member variable viewer__ of type pcl::visualization::PCLVisualizer, used to create a parameterized visualization window. In the run() method, the viewer__ object is first initialized, and basic visualization window parameters are configured, including window name, position, size, background color, the point cloud data to be presented, various point cloud attributes, view camera parameters, etc. Then, the sub-thread enters an infinite loop function, which loops at a frequency of 10Hz and checks the health of viewer__. When viewer__ stops running, the loop automatically ends to avoid wasting platform computing power on invalid visualization refreshes. In this infinite loop, the corresponding point cloud rendering and visualization control, user interaction processing, cross-thread parameter synchronization, and view export functions are implemented according to different user requests.

[0017] Furthermore, in the implementation of the point cloud rendering function, the point cloud rendering parameters are initialized in the run() method. When the user triggers a request to change the "single point size", ViewerThread captures the current user request flag and the "point size" parameter set by the user through the Horizontal Slider component via communication with the main thread, executes the changePointSize() function, and resets the point cloud rendering parameters according to the user-set parameters. After completing this function and other subsequent function functions, the next loop begins, and the VTK visualization window object viewer_ re-renders according to the newly set parameters, thereby realizing the dynamic point cloud rendering function. The visualization control is specifically the "show coordinate axis" function, which allows the user to dynamically adjust the coordinate axis size through the Horizontal Slider component in the UI interface. The function can also be triggered by the "show coordinate axis / hide coordinate axis" button, and the related functions are implemented through the showAxis() function. Its basic process is similar to that of point cloud rendering.

[0018] Furthermore, the user interaction processing functions include four types: adding point clouds, viewing point clouds, two-point distance measurement, and deleting point clouds. In implementing these functions, the interaction callback functions are first registered and bound in the `run()` function based on the PCL framework. These include `registerKeyboardCallback()`, `registerMouseCallback()`, `registerPointPickingCallback()`, and `registerAreaPickingCallback()`. These callback functions also utilize the PCL library's event triggering mechanism, which refers to a specified action triggered by a background event, such as a mouse double-click event. The events used in the user interaction processing functions include: `KeyboardEvent`, `MouseEvent`, `PointPickingEvent`, and `AreaPickingEvent`. Through different event response mechanisms and related callback functions, real-time interaction between the user and the point cloud data in the visualization interface is achieved.

[0019] For the point cloud deletion function, the KeyboardEvent event is used in the pclKeyBoardCallBack() callback function to bind the "X" and "x" keys to the on / off function of deleting the point cloud. Then, user requests are monitored through inter-thread communication. When a request is captured, the UI process information stream outputs a prompt message to assist the user in toggling the function on and off again using the "X" and "x" keys. When the user enables the function a second time, the user's mouse selection action will trigger the aforementioned area selection callback function, which will start executing the pclAreaPickingCallBack() function. The getPointsIndices() function of the AreaPickingEvent event can extract the sequence number of the points in the selected pe_cloud container, and then use the ExtractIndices filter to delete the selected point set. Finally, the UI process information stream is output and the relevant flags are reset and updated to fulfill the user request.

[0020] To add point cloud functionality, first call the PCL's getCameraParameters() function to obtain the camera parameters under the current view, then use the computeViewMatrix() and computeProjectionMatrix() functions to calculate the camera's view matrix M respectively. view and projection matrix M proj Among them, M view M represents the transformation from the point cloud coordinate system to the camera coordinate system. proj This represents the projection transformation from the camera coordinate system to the screen coordinate system; it is achieved through the product of the following two matrices, M. composite The complete transformation relationship from the point cloud coordinate system to the screen coordinate system is defined:

[0021] M composite =M view *M proj

[0022] Then, two reference points are read from the pe_cloud container across threads, obtaining their coordinates P1(x1,y1,z1) and P2(x2,y2,z2) in the point cloud coordinate system; these two points are then 2Dized to eliminate rotation and perspective effects, resulting in P1'(x1,y1,0) and P2'(x2,y2,0); the two points are then compared with M... composite The coordinate system is transformed to obtain the pixel coordinates P1"(a1,b1,0) and P2"(a2,b2,0) in the screen coordinate system; the clicked screen position P0"(a0,b0,0) is captured using the MouseEvent; based on the similarity principle, a proportional formula is used:

[0023]

[0024] Therefore, we get:

[0025]

[0026] This gives us the 2D coordinates of the mouse click location in the point cloud coordinate system:

[0027]

[0028] Subsequently, the intensity and elevation value z of the point are custom-completed to form a complete point cloud. The point is then added to the pe_cloud container and visualized in the next rendering cycle, thus fulfilling the user's request to add a point cloud.

[0029] Furthermore, the cross-thread parameter synchronization function includes two parts: view reset and global reset. The view reset is implemented using the doViewReset() function, which captures user requests through inter-thread communication, then keeps the view camera position unchanged, and only resets its observation angle completely in the opposite direction of elevation to achieve the same top-down view effect. When this function needs to be implemented once, a request is sent by clicking the "View Reset" button. If it needs to be executed frequently in a loop, a request is sent by checking the "Fixed Top View" checkbox component. The global reset is implemented using the doAllReset() function. First, the point cloud is reset across threads, replacing the current point cloud with the initially loaded point cloud. Then, the UI component parameters are reset across threads, resetting them all to the state when the software was first started. Finally, the view reset of this thread is executed to achieve the reset of the visualization window.

[0030] Furthermore, the view export function is specifically implemented using the saveScreenShot() function. After capturing the user request through inter-thread communication, this function uses the saveScreenshot() function provided by the VTK framework to capture the image information of the current visualization window, and sends a pop-up request to the main thread through the getScreenShotPathOut() function of the CtrlWindow class. The CtrlWindow class then invokes the QFileDialog pop-up to assist the user in specifying the image save path and naming the image. Finally, the set function is used to reset and update the function flag to meet the user's requirements.

[0031] The PCD format point cloud data visualization processing software framework construction method provided by the present invention addresses the limitation of existing technologies that generally only support simple reading of PCD format files. It provides a point cloud data visualization processing program development method that can be ported and run on platforms such as Linux and Windows. It can comprehensively realize various functions such as point cloud data visualization editing, modification and storage, thereby greatly expanding the application scenarios and processing depth of PCD format point cloud data. Attached Figure Description

[0032] Figure 1 A software system framework diagram constructed by the method provided in this invention;

[0033] Figure 2 This is a diagram of an optional software user interface built based on the method provided in this invention. Detailed Implementation

[0034] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] The PCD format point cloud data visualization processing software framework developed based on this invention is as follows: Figure 1 As shown, the specific steps for its construction include:

[0036] Step 1: Design the software user interface (UI) based on the QT framework and QT components. The programming language is C++. The PCL and VTK libraries are used as point cloud data processing tools and point cloud visualization frameworks, and the Eigen library is used as the software's built-in algorithm mathematical solution tool. First, create a new Qt Widgets Application class project file under the QT framework, and use the .pro file as the project configuration file to configure QT, project files, and third-party libraries including PCL, VTK, Eigen, and Boost libraries, so as to realize the portability and operation of the overall framework between Linux and Windows platforms.

[0037] Step 2: Based on the multi-threaded optimization design, an instance of the StartThread class is created to start the software. First, in the StartThread class constructor, a QApplication class object is created to manage the control flow and main settings of the GUI program. Then, a CtrlWindow class is instantiated to register and manage the software's user interface (UI). Next, a PCDEditor class object is created for registering and managing point cloud processing functions. Following this, a PCDEditorThread class object is created to encapsulate the sub-threads of the point cloud processing program. Then, a ViewerThread class object is created to register and manage point cloud visualization functions, while also encapsulating sub-threads. Finally, the show() function of the CtrlWindow parent class QWidget is called to display the software UI, and the start() function of the QThread class is called to start the parallel execution of the two sub-threads.

[0038] Step 3: Design the UI window within the QT framework using the ctrlwindow.ui layout file. First, visualize the ctrlwindow.ui window in QTDesign, and build UI components according to actual functional requirements to implement the relevant functions of PCD, including operation, processing, and display.

[0039] Step 4: Establish a communication mechanism between components and threads based on the signal and slot mechanism. This includes binding the UI components in the ctrlwindow.ui layout file with slot functions in the CtrlWindow class of the main thread to ensure that the interaction between components is achieved through a predefined event triggering strategy.

[0040] Step 5: Design the PCDEditor class, which runs under the PCL, VTK, and Eigen library framework and serves as the core of the sub-thread for point cloud data processing. Enable it to communicate with the CtrlWindow main thread and the ViewerThread sub-thread during runtime to respond to user interaction operations and update the point cloud data in real time.

[0041] Step 6: Design the ViewerThread class, which serves as the core of the point cloud visualization and user interaction mechanism. The ViewerThread class enables bidirectional inter-thread communication by subscribing to CtrlWindow and PCDEditor class objects in the constructor. This allows for interactive viewing, modification, and editing of point cloud data within the visualization window.

[0042] In a preferred embodiment of the present invention, the relevant functions implemented in step three for building the UI components specifically include: using the Push Button component to design and apply functions such as loading PCD, saving PCD, global reset, view reset, screen snapshot, export, global downsampling, filtering discrete points, pass-through filtering, global coordinate transformation, two-point distance measurement reset, coordinate axis visualization, and process print clearing; using the Check Box component to design and apply functions such as deleting point clouds, adding point clouds, viewing point clouds, two-point distance measurement, and fixed top view; using the Table Widget component to display the results of viewing point clouds and two-point distance measurement; using the Horizontal Slider component to develop and apply functions such as single-point size change and coordinate axis visualization; using the Spin Box and Double Spin Box components to set hyperparameters for global downsampling, filtering discrete points, and pass-through filtering; using the Combo Box component to select the mode for pass-through filtering and global coordinate transformation; and using the Text Edit component to display UI window description text and process print output for some functional modules. Figure 2 An optional software user interface designed using the steps described above is shown in the figure.

[0043] In a preferred embodiment of the present invention, in step four, the CtrlWindow class simultaneously maintains a series of functional flags and corresponding get and set functions. These functions enable the querying and updating of flags and other class member variables, thereby achieving inter-thread communication. Taking the PCD file reading function as an example: when the user clicks the "Load PCD" button (Push Button) on the UI interface, the slot function on_loadPCD_View_clicked() bound to this button in the CtrlWindow class will immediately respond and execute. First, a specific function is executed, namely, calling the QFileDialog component to pop up a window, allowing the user to select a PCD file under a certain path for loading. Then, a PCD file path logic check is performed. If the check passes, the absolute path is extracted, packaged into the class member variable pcdFilePathIn, and the flag loadPCDViewFlag for this function is updated to True. The UI process information stream is also output, allowing the user to know the current function execution status. If the above file path logic check fails, the flag is not updated, and relevant alarm information is output in the UI process information stream, prompting the user that the function has failed to execute successfully.

[0044] In the design of the corresponding read (get) and set (set) functions, taking the PCD file reading function as an example: the getLoadPCDViewFlag() function is designed so that the PCDEditor sub-thread can monitor the function flag loadPCDViewFlag. When the latter is activated and updated, the relevant program of the PCDEditor sub-thread starts to execute. The getPcdFilePathIn() function is used to capture the absolute path of the PCD file to be read. Then, the specific functions of reading PCD file, assigning values, logic checking, and outputting UI process information flow are executed. Finally, the PCDEditor sub-thread calls the setLoadPCDViewFlag() function to reset and update the function flag, thus completing the inter-thread communication and specific function implementation between the main thread and the sub-thread.

[0045] In a preferred embodiment of the present invention, in step five, the PCDEditor class declares a pcl::PointCloud. <pointt>The member variable pe_cloud of type ::Ptr is used to store global point cloud data; the child thread encapsulation of the PCDEditor instance pe is implemented in the constructor of the StartThread class. At the same time, the Run() function of PCDEditor can respond to the start() method of the parent class QThread of PCDEditorThread and execute the Run() method of the PCDEditorThread class, so that the Run() function of PCDEditor can run in the child thread to achieve parallel operation.

[0046] The point cloud processing task is continuously executed through an infinite loop function implemented in the `Run()` function. Its point cloud processing capabilities include three data management functions: `LoadPCDFile()` for loading point clouds, `SavePCDFile()` for storing point clouds, and `SaveAsFile()` for exporting point clouds to multiple formats, including PLY, PCDBinary, OBJ, and VTK; and four point cloud filtering modes: `AllPointsDownsize()` for global downsampling, allowing users to set voxel edge lengths; `AllPointsOutlierRemoval()` for filtering out discrete points based on user-defined hyperparameters such as the estimated number of points based on the average distance and the standard deviation factor; and `AllPointsPassT`. `hrough()` is used to perform pass-through filtering based on the filtering direction and range specified by the user; `AllPointsRemoveGround()` is used to filter out ground points based on the ground height and elevation threshold specified by the user; the `TfTrans()` function provides global coordinate transformation capabilities, supporting user-defined three-axis transformations; based on the infinite loop nature of the `Run()` function, this sub-thread periodically calls a series of related function `get` functions in the `CtrlWindow` main thread, using the `Flag` flag to monitor user requests. Once a user request is detected, the sub-thread immediately executes the corresponding function to meet the user's needs, and resets the `Flag` by calling the `CtrlWindow` `set` function, thus completing the current communication cycle.

[0047] In a preferred embodiment of the present invention, when AllPointsRemoveGround() performs ground point filtering, it specifically uses inter-thread communication to call the getAllPointsRemoveGroundFlag() function of the CtrlWindow class instance cw to detect the user request corresponding to the function in real time; when the user request is captured, the getUi() function is called to capture the user interface class instance ui_, and the user-set ground height groundZ and elevation threshold ground_remove_threshold parameters are directly received from ui_; then, the boundary of the point cloud data is determined, and then rasterization is performed on the XY plane at a specified resolution, initializing the maximum and minimum values ​​of each raster cell. The process involves: traversing the point cloud and updating the Z-values ​​of the raster cells, calculating the Z-value change for each cell; if the Z-value change of a cell is less than the elevation threshold `ground_remove_threshold`, and the point's height is close to `groundZ`, then marking the cell as ground; next, separating ground and non-ground point clouds based on the raster marking results to filter out ground points; finally, calling the `setAllPointsRemoveGroundFlag()` function to reset and update the ground point filtering flag, fulfilling the user's request for ground point filtering; adapting to point cloud data with different densities and terrain features by adjusting the resolution and threshold, where the actual ground height of the sensor during data acquisition is considered when setting `groundZ`.

[0048] In a preferred embodiment of the present invention, in step six, the ViewerThread class is a subclass of QThread. In StartThread, it directly calls the start() method of its parent class QThread to execute the run() method of the ViewerThread class, thereby achieving parallel execution of multiple threads. The ViewerThread class declares and maintains a member variable viewer__ of type pcl::visualization::PCLVisualizer, used to create a parameterized visualization window. In the run() method, the viewer__ object is first initialized, configuring basic visualization window parameters, including window name, position, size, background color, presented point cloud data, various point cloud attributes, and view camera parameters. Then, the sub-thread enters an infinite loop function, which loops at a frequency of 10Hz and checks the health of viewer__. When viewer__ stops running, the loop automatically ends to avoid wasting platform computing power on invalid visualization refreshes. Within this infinite loop, corresponding point cloud rendering and visualization control, user interaction processing, cross-thread parameter synchronization, and view export functions are implemented according to different user requests.

[0049] In a preferred embodiment of the present invention, the point cloud rendering function is implemented by initializing the point cloud rendering parameters in the run() method. When the user triggers a request to change the "single point size", ViewerThread captures the current user request flag and the "point size" parameter set by the user through the Horizontal Slider component via communication with the main thread, executes the changePointSize() function, and resets the point cloud rendering parameters according to the user-set parameters. After completing this function and other subsequent function functions, the next loop begins, and the VTK visualization window object viewer_ is re-rendered according to the newly set parameters, thereby realizing the dynamic point cloud rendering function. The visualization control is specifically the "show coordinate axis" function, which allows the user to dynamically adjust the coordinate axis size through the Horizontal Slider component in the UI interface. The function can also be triggered by the "show coordinate axis / hide coordinate axis" button, and the related functions are implemented through the showAxis() function. Its basic process is similar to that of point cloud rendering.

[0050] In a preferred embodiment of the present invention, the user interaction processing function includes four functions: adding point clouds, viewing point clouds, measuring distance between two points, and deleting point clouds. To implement these functions, firstly, based on the PCL framework, interactive callback functions are registered and bound in the `run()` function, including `keyboardCallback()`, `mouseCallback()`, `pointPickingCallback()`, and `areaPickingCallback()`. These callback functions also incorporate the event triggering mechanism of the PCL library, which refers to a specified action triggered by a certain background event, such as a mouse double-click event. The events applied by the user interaction processing function include: `KeyboardEvent`, `MouseEvent`, `PointPickingEvent`, and `AreaPickingEvent`. Through different event response mechanisms and related callback functions, real-time interaction between the user and the point cloud data in the visualization interface is achieved.

[0051] For the point cloud deletion function, the KeyboardEvent event is used in the pclKeyBoardCallBack() callback function to bind the "X" and "x" keys to the on / off function of deleting the point cloud. Then, user requests are monitored through inter-thread communication. When a request is captured, the UI process information stream outputs a prompt message to assist the user in toggling the function on and off again using the "X" and "x" keys. When the user enables the function a second time, the user's mouse selection action will trigger the aforementioned area selection callback function, which will start executing the pclAreaPickingCallBack() function. The getPointsIndices() function of the AreaPickingEvent event can extract the sequence number of the points in the selected pe_cloud container, and then use the ExtractIndices filter to delete the selected point set. Finally, the UI process information stream is output and the relevant flags are reset and updated to fulfill the user request.

[0052] To add point cloud functionality, first call the PCL's getCameraParameters() function to obtain the camera parameters under the current view, then use the computeViewMatrix() and computeProjectionMatrix() functions to calculate the camera's view matrix M respectively. view and projection matrix M proj Among them, M view M represents the transformation from the point cloud coordinate system to the camera coordinate system. proj This represents the projection transformation from the camera coordinate system to the screen coordinate system; it is achieved through the product of the following two matrices, M. composite The complete transformation relationship from the point cloud coordinate system to the screen coordinate system is defined:

[0053] M composite =M view *M proj

[0054] Then, two reference points are read from the pe_cloud container across threads, obtaining their coordinates P1(x1,y1,z1) and P2(x2,y2,z2) in the point cloud coordinate system; these two points are then 2Dized to eliminate rotation and perspective effects, resulting in P1'(x1,y1,0) and P2'(x2,y2,0); the two points are then compared with M... composite The coordinate system is transformed to obtain the pixel coordinates P1"(a1,b1,0) and P2"(a2,b2,0) in the screen coordinate system; the clicked screen position P0"(a0,b0,0) is captured using the MouseEvent; based on the similarity principle, a proportional formula is used:

[0055]

[0056] Therefore, we get:

[0057]

[0058] This gives us the 2D coordinates of the mouse click location in the point cloud coordinate system:

[0059]

[0060] Subsequently, the intensity and elevation value z of the point are custom-completed to form a complete point cloud. The point is then added to the pe_cloud container and visualized in the next rendering cycle, thus fulfilling the user's request to add a point cloud.

[0061] In a preferred embodiment of the present invention, the cross-thread parameter synchronization function includes two parts: view reset and global reset. The view reset is implemented using the doViewReset() function, which captures user requests through inter-thread communication, and then keeps the view camera position unchanged, resetting only its observation angle completely in the opposite direction of elevation to achieve a top-down view effect. When this function needs to be implemented once, a request is sent by clicking the "View Reset" button; when it needs to be executed frequently in a loop, a request is sent by checking the "Fixed Top View" checkbox component. The global reset is implemented using the doAllReset() function. First, a point cloud reset is performed across threads, replacing the current point cloud with the initially loaded point cloud; then, UI component parameter reset is performed across threads, resetting all of them to the state when the software was first started; finally, the view reset of the current thread is executed to achieve the reset of the visualization window.

[0062] In a preferred embodiment of the present invention, the view export function is specifically implemented using the saveScreenShot() function. After capturing the user request through inter-thread communication, the function uses the saveScreenshot() function provided by the VTK framework to capture the image information of the current visualization window, and sends a pop-up request to the main thread through the getScreenShotPathOut() function of the CtrlWindow class. The CtrlWindow class then invokes the QFileDialog pop-up to assist the user in specifying the image save path and naming the image. Finally, the set function is used to reset and update the function flag to meet the user's requirements.

[0063] It should be understood that the sequence number of each step in the embodiments of the present invention does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0064] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.< / pointt> < / pointt>

Claims

1. A method for constructing a software framework for visualizing and processing PCD format point cloud data, characterized in that: Specifically, the following steps are included: Step 1: Design the software user interface (UI) based on the QT framework and QT components. The programming language is C++. The PCL and VTK libraries are used as point cloud data processing tools and point cloud visualization frameworks, and the Eigen library is used as the software's built-in algorithm mathematical solution tool. First, create a new Qt Widgets Application class project file under the QT framework, and use the .pro file as the project configuration file to configure QT, project files, and third-party libraries including PCL, VTK, Eigen, and Boost libraries, so as to realize the portability and operation of the overall framework between Linux and Windows platforms. Step 2: Based on the multi-threaded optimization design scheme, an instance of the StartThread class is created to start the software. First, in the StartThread class constructor, a QApplication class object is created to manage the control flow and main settings of the GUI program; then, the CtrlWindow class is instantiated to register and manage the software user interface, i.e., the UI interface; then, a PCDEditor class object is created to register and manage the point cloud processing class functions; next, a PCDEditorThread class object is created to encapsulate the sub-threads of the point cloud processing program. Next, create a ViewerThread class object to register and manage point cloud visualization class functions, and encapsulate the child thread; finally, call the show() function of the CtrlWindow parent class QWidget to display the software UI, and call the start() function of the QThread class to start the parallel execution of the two child threads; Step 3: Design the UI window within the QT framework using the ctrlwindow.ui layout file. First, visualize the ctrlwindow.ui window in QT Design, and build UI components according to actual functional requirements to implement the relevant functions of PCD, including operation, processing, and display. Step 4: Establish a communication mechanism between components and threads based on the signal and slot mechanism. This includes binding the UI components in the ctrlwindow.ui layout file with slot functions in the CtrlWindow class of the main thread to ensure that the interaction between components is achieved through a predefined event triggering strategy. Step 5: Design the PCDEditor class, which runs under the PCL, VTK, and Eigen library framework and serves as the core of the sub-thread for point cloud data processing. Enable it to communicate with the CtrlWindow main thread and the ViewerThread sub-thread during runtime to respond to user interaction operations and update the point cloud data in real time. Step 6: Design the ViewerThread class as the core of the point cloud visualization and user interaction mechanism. The ViewerThread class enables bidirectional inter-thread communication by subscribing to CtrlWindow and PCDEditor class objects in the constructor. This allows for interactive viewing, modification, and editing of point cloud data in the visualization window.

2. The method as described in claim 1, characterized in that: Step 3 involves building and implementing UI components, specifically including: using the Push Button component for loading PCD, saving PCD, global reset, view reset, screen snapshot, export, global downsampling, filtering discrete points, pass-through filtering, global coordinate transformation, two-point distance measurement reset, coordinate axis visualization, and process print clearing functions; using the Check Box component for deleting point clouds, adding point clouds, viewing point clouds, two-point distance measurement, and fixed top-down view functions; using the Table Widget component for viewing point clouds and displaying the results of two-point distance measurement functions; using the Horizontal Slider component for developing and applying single-point size modification and coordinate axis visualization functions; using the Spin Box and Double Spin Box components for setting hyperparameters for global downsampling, filtering discrete points, and pass-through filtering; using the Combo Box component for selecting the mode for pass-through filtering and global coordinate transformation; and using the Text Edit component for displaying UI window descriptions and process printouts for some functional modules.

3. The method as described in claim 1, characterized in that: In step four, the CtrlWindow class maintains a series of function flags and corresponding get and set functions. These functions are used to query and update the flags and other class member variables, thereby enabling inter-thread communication.

4. The method as described in claim 1, characterized in that: In step five, the PCDEditor class declares a pcl::PointCloud. <pointt> The member variable pe_cloud of type ::Ptr is used to store global point cloud data; the child thread encapsulation of the PCDEditor instance pe is implemented in the constructor of the StartThread class. At the same time, the Run() function of PCDEditor can respond to the start() method of the parent class QThread of PCDEditorThread and execute the Run() method of the PCDEditorThread class, so that the Run() function of PCDEditor can run in the child thread to achieve parallel operation.< / pointt> The point cloud processing task is continuously executed through an infinite loop function implemented in the `Run()` function. Its point cloud processing capabilities include three data management functions: `LoadPCDFile()` for loading point clouds, `SavePCDFile()` for storing point clouds, and `SaveAsFile()` for exporting point clouds to multiple formats, including PLY, PCDBinary, OBJ, and VTK; and four point cloud filtering modes: `AllPointsDownsize()` for global downsampling, allowing users to set voxel edge lengths; `AllPointsOutlierRemoval()` for filtering out discrete points based on user-defined hyperparameters such as the estimated number of points based on the average distance and the standard deviation factor; and `AllPointsPassT`. `hrough()` is used to perform pass-through filtering based on the filtering direction and range specified by the user; `AllPointsRemoveGround()` is used to filter out ground points based on the ground height and elevation threshold specified by the user; the `TfTrans()` function provides global coordinate transformation capabilities, supporting user-defined three-axis transformations; based on the infinite loop nature of the `Run()` function, this sub-thread periodically calls a series of related function `get` functions in the `CtrlWindow` main thread, using the `Flag` flag to monitor user requests. Once a user request is detected, the sub-thread immediately executes the corresponding function to meet the user's needs, and resets the `Flag` by calling the `CtrlWindow` `set` function, thus completing the current communication cycle.

5. The method as described in claim 4, characterized in that: When AllPointsRemoveGround() performs ground point filtering, it specifically uses inter-thread communication to call the getAllPointsRemoveGroundFlag() function of the CtrlWindow class instance cw to detect the user request corresponding to this function in real time. When a user request is captured, the getUi() function is called to capture the user interface class instance ui, directly receiving the user-defined ground height groundZ and elevation threshold ground_remove_threshold parameters from ui_. Then, the boundary of the point cloud data is determined, and rasterization is performed on the XY plane at a specified resolution, initializing the maximum and minimum Z values ​​of each raster cell. This is done by iterating through... The point cloud is processed and the Z-value of the raster cells is updated, and the Z-value change of each cell is calculated. If the Z-value change of a cell is less than the elevation threshold ground_remove_threshold, and the height of the point is close to groundZ, the cell is marked as ground. Then, ground and non-ground point clouds are separated according to the raster marking results to filter out ground points. Finally, the setAllPointsRemoveGroundFlag() function is called to reset and update the ground point filtering function flag, realizing the user's request for ground point filtering function. For point cloud data with different densities and terrain features, the resolution and threshold are adjusted to adapt, and the actual ground height of the sensor during data acquisition is taken into account when setting groundZ.

6. The method as described in claim 1, characterized in that: In step six, the ViewerThread class, as a subclass of QThread, directly calls the start() method of its parent class QThread in StartThread to execute the run() method of the ViewerThread class, thereby achieving parallel execution of multiple threads. The ViewerThread class declares and maintains a member variable viewer__ of type pcl::visualization::PCLVisualizer, which is used to create a parameterized visualization window. In the run() method, the viewer__ object is first initialized, and basic visualization window parameters are configured, including window name, position, size, background color, point cloud data to be presented, various point cloud attributes, and view camera parameters. Then, the sub-thread enters an infinite loop function, which loops at a frequency of 10Hz and checks the health of viewer__. When viewer__ stops running, the loop automatically ends to avoid wasting platform computing power on invalid visualization refreshes. Within this infinite loop, corresponding point cloud rendering and visualization control, user interaction processing, cross-thread parameter synchronization, and view export functions are implemented according to different user requests.

7. The method as described in claim 6, characterized in that: In implementing the point cloud rendering function, the point cloud rendering parameters are initialized in the `run()` method. When the user triggers a request to change the "single point size", `ViewerThread` captures the current user request flag and the "point size" parameter set by the user through the Horizontal Slider component via communication with the main thread. It then executes the `changePointSize()` function to reset the point cloud rendering parameters according to the user's settings. After completing this function and other subsequent functions, it enters the next loop. The VTK visualization window object `viewer_` re-renders according to the newly set parameters, thus realizing the dynamic point cloud rendering function. The visualization control is specifically the "show coordinate axis" function, which allows users to dynamically adjust the coordinate axis size through the Horizontal Slider component in the UI interface. The function can be triggered by the "show coordinate axis / hide coordinate axis" button, and the related functions are implemented through the `showAxis()` function. Its basic process is similar to that of point cloud rendering.

8. The method as described in claim 6, characterized in that: The user interaction processing functions include four types: adding point clouds, viewing point clouds, two-point distance measurement, and deleting point clouds. In implementing these functions, the interaction callback functions are first registered and bound in the run() function based on the PCL framework, including registerKeyboardCallback(), registerMouseCallback(), registerPointPickingCallback(), and registerAreaPickingCallback(). These callback functions also introduce the event triggering mechanism of the PCL library, which refers to a specified action triggered by a certain background event. The events used in the user interaction processing function include: KeyboardEvent, MouseEvent, PointPickingEvent, and AreaPickingEvent; through different event response mechanisms and related callback functions, real-time interaction between the user and the point cloud data is achieved. For the point cloud deletion function, the KeyboardEvent event is used in the pclKeyBoardCallBack() callback function to bind the "X" and "x" keys to the on / off function of deleting the point cloud. Then, user requests are monitored through inter-thread communication. When a request is captured, the UI process information stream outputs a prompt message to assist the user in toggling the function on and off again using the "X" and "x" keys. When the user enables the function a second time, the user's mouse selection action will trigger the aforementioned area selection callback function, which will start executing the pclAreaPickingCallBack() function. The getPointsIndices() function of the AreaPickingEvent event can extract the sequence number of the points in the selected pe_cloud container, and then use the ExtractIndices filter to delete the selected point set. Finally, the UI process information stream is output and the relevant flags are reset and updated to fulfill the user request. To add point cloud functionality, first call the PCL's getCameraParameters() function to obtain the camera parameters under the current view, then use the computeViewMatrix() and computeProjectionMatrix() functions to calculate the camera's view matrix M respectively. view and projection matrix M proj Among them, M view M represents the transformation from the point cloud coordinate system to the camera coordinate system. proj This represents the projection transformation from the camera coordinate system to the screen coordinate system; it is achieved through the product of the following two matrices, M. composite The complete transformation relationship from the point cloud coordinate system to the screen coordinate system is defined: M composite =M view *M proj Then, two reference points are read from the pe_cloud container across threads, obtaining their coordinates P1(x1,y1,z1) and P2(x2,y2,z2) in the point cloud coordinate system; these two points are then 2Dized to eliminate rotation and perspective effects, resulting in P1'(x1,y1,0) and P2'(x2,y2,0); the two points are then compared with M... composite The coordinate system is transformed to obtain the pixel coordinates P1"(a1,b1,0) and P2"(a2,b2,0) in the screen coordinate system; the clicked screen position P0"(a0,b0,0) is captured using the MouseEvent; based on the similarity principle, a proportional formula is used: Therefore, we get: This gives us the 2D coordinates of the mouse click location in the point cloud coordinate system: Subsequently, the intensity and elevation value z of the point are custom-completed to form a complete point cloud. The point is then added to the pe_cloud container and visualized in the next rendering cycle, thus fulfilling the user's request to add a point cloud.

9. The method as described in claim 1, characterized in that: The cross-thread parameter synchronization function includes two parts: view reset and global reset. The view reset is implemented using the doViewReset() function. It captures user requests through inter-thread communication, then keeps the view camera position unchanged, and only resets its observation angle completely in the opposite direction of elevation to achieve the same top-down view effect. When this function needs to be implemented once, a request is sent by clicking the "View Reset" button. If it needs to be executed frequently in a loop, a request is sent by checking the "Fixed Top View" checkbox component. The global reset is implemented using the doAllReset() function. First, it performs a cross-thread point cloud reset, replacing the current point cloud with the initially loaded point cloud. Then, it performs a cross-thread UI component parameter reset, resetting all of them to the state when the software was first launched. Finally, it executes the view reset in the current thread to achieve the reset of the visualization window.

10. The method as described in claim 1, characterized in that: The view export function is implemented using the saveScreenShot() function. After capturing the user request through inter-thread communication, this function uses the saveScreenshot() function provided by the VTK framework to capture the image information of the current visualization window, and sends a pop-up request to the main thread through the getScreenShotPathOut() function of the CtrlWindow class. The CtrlWindow class then invokes the QFileDialog pop-up to help the user specify the image save path and name the image. Finally, the set function is used to reset and update the function flag to meet the user's requirements.

Images