Mobile three-dimensional laser scanning detection system for large castings

Abstract

The present application relates to the technical field of laser scanning detection, and particularly relates to a mobile three-dimensional laser scanning detection system for large castings. The system obtains a scanning difficulty evaluation index by analyzing the local geometric features of the point cloud in the point cloud grid on the fitting plane and the distribution difference of the point cloud on the fitting plane; the scanning difficulty evaluation index is obtained by covering and scoring the target grid and the non-target grid in different ways, avoiding meaningless distance and angle calculation, and improving the calculation efficiency; when optimization is performed, the spherical candidate area is obtained by clustering, which can effectively solve the problem of laser scanning shielding caused by irregular areas on the large castings; compared with the existing random candidate point generation method, the optimization is performed by generating candidate poses for the clustering cluster, which greatly reduces the optimization range. The present application finds the best scanning pose by using the guided local optimal decision algorithm, ensures to cover a larger area, improves the scanning efficiency, and improves the integrity of the scanning data.

Description

Technical Field

[0001] This invention relates to the field of laser scanning inspection technology, and more specifically to a mobile three-dimensional laser scanning inspection system for large castings. Background Technology

[0002] With the development of laser scanning technology, mobile 3D laser scanning technology can collect point cloud data of large castings in a non-contact manner, realize the rapid construction of high-precision 3D models, and compare the collected point cloud data with the original design model, which can greatly improve the inspection efficiency and integrity of large castings and realize flexible inspection.

[0003] Due to the complex structure of large castings, including deep cavities, concave areas, and irregular regions such as ribbed plates, these areas can physically obstruct the laser beam. This leads to blind spots and data gaps when using a pre-set single scanning path during 3D laser scanning, compromising the integrity of the 3D data and necessitating supplementary scanning. Current technologies, which dynamically plan the supplementary scanning path using local optimum decision algorithms, typically only consider the coverage of the scan results, neglecting the quality of the obtained point cloud data. This can mask the true information of these irregular areas, increasing the false detection rate and introducing detection errors when compared with the original design model. Summary of the Invention

[0004] To address the technical problem that existing technologies focus solely on scan coverage when planning the scan path, resulting in poor scan data quality and scanning detection errors, the present invention aims to provide a mobile three-dimensional laser scanning and inspection system for large castings. The specific technical solution adopted is as follows: This invention proposes a mobile three-dimensional laser scanning inspection system for large castings, the system comprising: The data acquisition module is used to acquire the pose data of the 3D laser scanning device and the point cloud data of the large casting scanned by the 3D laser scanning device in each pose; the point cloud data is processed into a grid to obtain several point cloud grids; the pose data includes the line of sight angle and device position of the 3D laser scanning device in each pose. The scanning difficulty assessment module is used to obtain the scanning difficulty evaluation index of the point cloud mesh at each pose, based on the information content of the point cloud in the point cloud mesh, the local geometric features of the point cloud on the fitting plane, and the distribution differences of the orientation of the point cloud on the fitting plane. The scan integrity assessment module is used to obtain the coverage score of the point cloud mesh based on the point cloud data within the point cloud mesh and the pose data of the 3D laser scanning device at each pose; and to obtain the scan integrity assessment index of the point cloud mesh based on the scan difficulty evaluation index and the coverage score. The scanning path planning module is used to obtain the optimal scanning pose for the next scan based on the current pose data, the coverage score, and the scan integrity evaluation index during the moving scan process, using a local optimal decision algorithm.

[0005] Preferably, for each point cloud grid, the point cloud density is obtained based on the number and area of ​​points within the grid, and is used as the information content; The rate of change of curvature is obtained based on the distribution of curvature of the point cloud within the point cloud mesh on the fitting plane, and is used as the distribution difference of the local geometric features. The discreteness evaluation index of the normal vector is obtained based on the distribution of the normal vector of the point cloud within the point cloud mesh on the fitting plane, which serves as the distribution difference of the orientation.

[0006] Preferably, the method for obtaining the coverage score of the point cloud mesh includes: For each pose, based on the pose data of the 3D laser scanning device and the scanned point cloud data, the point cloud mesh located within the field of view of the 3D laser scanning device is selected. Use any selected point cloud mesh as a target mesh; For each target grid, the coverage score of the target grid is obtained based on the spatial distance between the target grid and the 3D laser scanning device, as well as the surface normal vector of the target grid and the line of sight of the 3D laser scanning sensor. Set the coverage score of any non-target point cloud mesh to the preset coverage score.

[0007] Preferably, the method for obtaining the coverage score of the target grid includes: For each target grid, calculate the spatial distance between the target grid and the 3D laser scanning device; Calculate the angle between the surface normal vector of the target mesh and the sensor's line-of-sight angle during 3D laser scanning; The coverage score of the target grid is obtained based on the trigonometric function values ​​of the spatial distance and the included angle.

[0008] Preferably, the method for obtaining the optimal scanning pose for the next scan includes: Before starting the scan, the pose of the 3D laser scanning device is initialized; During the moving scan, a set of candidate poses for the current pose is obtained, and the objective function value is obtained based on the scan integrity evaluation index and coverage score of all target meshes under each candidate pose in the candidate pose set. Based on the point cloud data scanned in the current pose and the objective function value, the optimal pose for the next scan is obtained using a local optimum decision algorithm.

[0009] Preferably, the method for obtaining the objective function value is as follows: For each candidate pose, the scan observation value of each target grid is obtained based on the scan integrity evaluation index of each target grid under each candidate pose and the coverage score of each target grid. The product of the scan observations of all target grids under each candidate pose and the number of target grids is used as the objective function value for each candidate pose.

[0010] Preferably, the method for obtaining the candidate pose set is as follows: Based on the scanning difficulty evaluation index and coordinate information of point cloud mesh, a clustering algorithm is used to divide all point cloud mesh into several clusters; For each cluster, the geometric center of the cluster is obtained. A spherical region is determined with the geometric center as the center and a preset scanning distance as the radius. Several points are sampled on the surface of the spherical region as candidate observation positions. For each candidate observation location, obtain the orientation angle that makes the optical axis of the laser sensor point towards the geometric center; Each candidate pose includes a candidate observation position and the orientation angle at that candidate observation position; The set of all candidate poses is called the candidate pose set.

[0011] Preferably, the method for obtaining the optimal pose for the next scan is to take the candidate pose with the largest objective function value in the candidate pose set as the optimal pose for the next scan.

[0012] Preferably, the preset scanning distance is less than the upper limit of the effective scanning distance of the three-dimensional laser scanning device.

[0013] Preferably, the sampling generation method is random sampling generation or uniform sampling generation.

[0014] The present invention has the following beneficial effects: This invention provides a scanning difficulty evaluation index by comprehensively analyzing the information content of the point cloud within the point cloud mesh at each pose, the local geometric features on the fitting plane, and the distribution differences of the point cloud in two dimensions on the fitting plane. This process accurately quantifies the difficulty of acquiring complete data at each pose by analyzing the impact of irregular structures on the large casting. Secondly, based on the point cloud data within the mesh and the pose data of the 3D laser scanning device, the coverage score of the point cloud mesh is determined. This process quantifies the assessment of the completeness of the point cloud mesh at each pose by analyzing the coverage of the 3D laser scanning device on the point cloud mesh at each pose. Finally, combined with the scanning difficulty evaluation index, a scanning integrity evaluation index for the point cloud mesh is determined. This index comprehensively considers both the completeness of the scan and the scanning difficulty, ensuring that when the local optimal decision algorithm is guided by the coverage score and the scanning integrity evaluation index to find the optimal scanning pose, the optimal scanning pose can guarantee a larger coverage area and improve scanning efficiency while ensuring that the point cloud mesh with irregular structures on the large casting, where scanning is difficult, is scanned, thus guaranteeing the integrity of the scanned data. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a structural block diagram of a mobile three-dimensional laser scanning inspection system for large castings, provided as an embodiment of the present invention. Detailed Implementation

[0017] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the mobile three-dimensional laser scanning inspection system for large castings proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] The following description, in conjunction with the accompanying drawings, details a specific solution for a mobile three-dimensional laser scanning inspection system for large castings provided by the present invention.

[0020] Please see Figure 1 The diagram shows a structural block diagram of a mobile three-dimensional laser scanning inspection system for large castings provided by an embodiment of the present invention. The system includes a data acquisition module 101, a scanning difficulty assessment module 102, a scanning integrity assessment module 103, and a scanning path planning module 104.

[0021] In this embodiment of the invention, the data acquisition module 101 acquires the pose data of the 3D laser scanning device and the point cloud data of the large casting scanned by the 3D laser scanning device at a preset scanning speed at each pose during the movement process. The preset scanning speed is automatically set based on the size of the large casting and the performance of the 3D laser scanning device, and will not be elaborated upon here.

[0022] Specifically, a mobile support platform is used to achieve smooth movement of the 3D laser scanning equipment. The mobile support platform refers to a multi-axis robotic arm mounted on a mobile device, including but not limited to unmanned transport vehicles and automated guided vehicles (AGVs); the multi-axis robotic arm includes, but is not limited to, four-axis and six-axis robotic arms. The 3D laser scanning equipment is mounted on the multi-axis robotic arm, and the pose of the 3D laser scanning equipment is adjusted through the movements of the multi-axis robotic arm.

[0023] Pose data includes the viewing angle and device position of the 3D laser scanning device in each pose; that is, in each pose, there exists a conical field of view and multi-degree-of-freedom pose data for the 3D laser scanning device. In this embodiment of the invention, in each pose, the sensor viewing direction of the 3D laser scanning device is acquired through a multi-axis robotic arm controller or driver program; and multi-degree-of-freedom pose data is obtained through a laser tracker.

[0024] Simultaneously, at each pose, a large casting is scanned using a 3D laser scanning device to obtain point cloud data. Since the scan yields discrete point clouds of the large casting's surface, subsequent analysis using 3D voxels would result in a large number of point clouds not belonging to the casting's surface, such as hollow parts of the casting or the background space, affecting the analysis results and making it difficult to reflect whether the casting's surface scan is complete. Therefore, in this embodiment of the invention, the point cloud data is meshed to obtain several point cloud meshes. The meshing process is as follows: based on the CAD design model surface of the large casting, the point cloud data is discretized into surface meshes of a preset size, resulting in several point cloud meshes. The meshing of point cloud data is a well-known technique, and the specific process will not be elaborated further.

[0025] In one implementation of this invention, the preset size is 10mm × 10mm.

[0026] It should be noted that the embodiments of the present invention aim to plan the scanning path of a 3D laser scanning device during the mobile scanning process. Therefore, the system proposed in the embodiments of the present invention is based on a mobile carrier platform mounted on the 3D laser scanning device. The embodiments of the present invention adjust the scanning path by adjusting the pose of the 3D laser scanning device; therefore, it is necessary to collect the pose data of the 3D laser scanning device and the scanned point cloud data. By analyzing the pose data already collected during the current scan and the point cloud data features in the point cloud mesh, the optimal scanning pose of the 3D laser scanning device for the next scan is determined. Therefore, after the data acquisition module 101 collects the pose data of the 3D laser scanning device and the point cloud data of the large casting for each scan, the collected data is transmitted to subsequent modules for further analysis.

[0027] The scanning difficulty assessment module 102 is used to obtain the scanning difficulty evaluation index of the point cloud mesh at each pose, based on the information content of the point cloud in the point cloud mesh, the local geometric features of the point cloud on the fitting plane, and the distribution differences of the orientation of the point cloud on the fitting plane.

[0028] In this embodiment of the invention, the object being scanned is a large casting with irregular areas such as deep cavities, concave areas, and ribbed regions. During each scan, these irregular areas easily obstruct the scanning laser, resulting in missing point cloud data. Therefore, the likelihood of missing data can be assessed by analyzing the amount of information in the point cloud data within the point cloud grid. The less information there is, the greater the possibility of the laser being obstructed during the scan, and the more difficult it is to obtain complete scan data.

[0029] Furthermore, it is considered that not only data loss due to occlusion during scanning leads to a smaller amount of information in the point cloud mesh, but also that the complex geometry of irregular areas further increases the difficulty of obtaining complete scan data. Moreover, the lack of scan data reflects scanning anomalies in large castings, while scanning of complex geometry in irregular areas still possesses authenticity and can be used for scanning and inspecting large castings.

[0030] The scanning difficulty assessment module can further quantify and evaluate the local geometric features of the point cloud within the point cloud mesh on the fitting plane, as well as the differences in the orientation distribution of the point cloud on the fitting plane. Based on the assessment results and the amount of information, a scanning difficulty evaluation index is obtained for each point cloud mesh. The scanning difficulty evaluation index characterizes the difficulty of accurately scanning complete point cloud data when performing laser scanning on the casting surface corresponding to the point cloud mesh at each pose. The higher the value of the scanning difficulty evaluation index, the more difficult it is to obtain complete point cloud data for that point cloud mesh.

[0031] Based on the results obtained from the scanning difficulty assessment module 102, the scan integrity assessment module 103 further analyzes the coverage integrity of the point cloud mesh by the scanning range of the 3D laser scanning device at each pose. In each pose, the field of view of the 3D laser scanning device is shaped like a cone; only point cloud meshes located within this cone can be scanned. That is, not all point cloud meshes are within the cone-shaped field of view of the 3D laser scanning device at that pose. The scan integrity assessment module first uses an occlusion culling algorithm to filter point cloud meshes within the cone-shaped field of view to obtain the target mesh. For each target mesh, a coverage score is obtained based on the point cloud data within the target mesh and the pose data of the 3D laser scanning device. The higher the coverage score, the higher the coverage integrity of the target mesh by the cone-shaped field of view of the 3D laser scanning device at the current pose.

[0032] It should be noted that for any point cloud mesh that is not the target mesh, the coverage score of such point cloud mesh is set to a preset coverage score. In one implementation of this invention, the preset coverage score is set to 0.

[0033] After obtaining the coverage score for each point cloud mesh, the scan integrity assessment module further combines the scan difficulty evaluation index to obtain the scan integrity evaluation index for the point cloud mesh. The scan integrity evaluation index characterizes the difficulty of accurately obtaining complete scan data when scanning a large casting at each pose, provided that the conical field of view of the scanning device covers each point cloud mesh. A higher scan integrity evaluation index indicates higher coverage integrity of the point cloud mesh by the conical field of view of the 3D laser scanning device at the current pose, and lower difficulty in obtaining complete point cloud data.

[0034] Based on the results obtained from the scan integrity evaluation module 103, the scan path planning module 104 determines the optimal scan pose for the next scan during the moving scan process. For the scanning and inspection of large castings, the pose of the mobile 3D laser scanning device needs to be adjusted in real time to ensure that the point cloud data of the large casting obtained under each scan pose is as complete as possible. By analyzing the coverage score of the target mesh and the scan integrity evaluation index obtained under different scan poses, the local optimum decision algorithm is guided to obtain the optimal scan pose for the next scan.

[0035] In the above scheme, the embodiments of the present invention acquire pose data of the 3D laser scanning device and point cloud data of the large casting at each pose, providing a data foundation for subsequent scanning path planning. By comprehensively analyzing the local geometric features of the point cloud within the point cloud grid on the fitting plane and the distribution differences of the point cloud in the two dimensions of orientation on the fitting plane, a scanning difficulty evaluation index is obtained, realizing the partitioned quantification of the difficulty of obtaining complete data for each point cloud grid. Based on the point cloud data within the point cloud grid and the pose data of the 3D laser scanning device, target grids are screened, and coverage scores are obtained for target grids and non-target grids in different ways, avoiding meaningless distance and angle calculations and improving computational efficiency. During optimization, spherical candidate regions are obtained through clustering, which can effectively solve the problem of irregular areas on the large casting obstructing the scanning laser. A set of candidate poses is generated within the spherical candidate regions. Compared with the existing method of randomly generating candidate points, the optimization of candidate poses generated for clusters greatly narrows the optimization range. In this embodiment of the invention, an objective function is constructed by using the coverage score of the point cloud mesh and the scanning integrity evaluation index of scanning the point cloud mesh to obtain complete data under different poses. When guiding the local optimal decision algorithm to find the best scanning pose, the target mesh number ensures that a larger area is covered, thereby improving scanning efficiency. At the same time, it ensures that meshes with high scanning difficulty are supplemented, thus ensuring the integrity of the scanning data.

[0036] Preferably, in this embodiment of the invention, for each point cloud grid, the information content is obtained based on the number and area of ​​point clouds within the point cloud grid to obtain the point cloud density.

[0037] Here, "information content" represents the number of points in each point cloud grid obtained by the 3D scanning device when scanning a large casting at the current pose. More information content indicates less influence from irregular areas of the casting when scanning the corresponding area of ​​the point cloud grid, resulting in a larger number of points within the grid and more data information contained therein.

[0038] Specifically, the ratio of the number of points in each point cloud grid to the area of ​​the point cloud grid is used as the point cloud density. The point cloud density is used as the information content. The higher the point cloud density, the more points there are in the same size point cloud grid, and the greater the information content.

[0039] Secondly, the rate of change of curvature is obtained based on the distribution of curvature of the point cloud within the point cloud mesh on the fitting plane, which serves as the distribution difference of the local geometric features.

[0040] During each scan, when an irregular region is scanned, the geometric features on the surface of the irregular region have obvious differences, such as irregular geometric structural lines. Curvature can be used to describe the smoothness and curvature of the point cloud surface. Therefore, in this embodiment of the invention, the curvature of the point cloud is used to obtain the distribution differences of local geometric features.

[0041] Among them, the rate of change index characterizes the distribution difference of the curvature of the point cloud within the point cloud mesh on the fitting plane. The larger the value of the rate of change index, the worse the consistency of the local geometric features of adjacent point clouds on the fitting plane, and the greater the difference between the curvatures obtained by adjacent point clouds on the fitting plane.

[0042] Specifically, for each point cloud in the point cloud grid, k nearest neighbor point clouds are obtained using the k-nearest neighbor algorithm, centered on each point cloud and based on the Euclidean distance between point clouds. Surface fitting is then performed based on the point cloud and its k nearest neighbors to obtain the curvature of the point cloud on the fitted plane. In one specific implementation of this invention, the value of k is set to an empirical value of 30. Surface fitting is a commonly used technique in the field of point cloud data processing; the specific process will not be elaborated further.

[0043] Based on the above method, the curvature of each point cloud in the point cloud grid on the fitting plane is obtained, the distribution difference of all curvatures is evaluated, and the evaluation result is used as the rate of change index.

[0044] In one specific implementation of this invention, for each point cloud grid, the absolute value of the difference between the curvatures of any two adjacent point clouds is calculated. The ratio of the absolute value of the difference to the average curvature of all point clouds is taken as the rate of change of curvature between the two adjacent point clouds. The sum of all the rates of change of curvature is taken as the rate of change index.

[0045] In other implementations of this invention, the degree of dispersion of all curvatures can be evaluated, and the evaluation index can be used as a rate of change index. The evaluation index includes, but is not limited to, the variance of all curvatures and the standard deviation of all curvatures. These are specific technical means well-known to those skilled in the art, and the specific process will not be elaborated further.

[0046] Subsequently, the discreteness evaluation index of the normal vector is obtained based on the distribution of the normal vectors of the point cloud within the point cloud mesh on the fitting plane, which serves as the distribution difference of the orientation.

[0047] Among them, the discreteness evaluation index characterizes the distribution difference of the orientation of the point cloud in the point cloud mesh on the fitting plane. The larger the discreteness evaluation index, the higher the degree of dispersion of the orientation distribution, and the greater the difference in the directional distribution between the normal vectors of the point cloud on the fitting plane.

[0048] Specifically, for each point cloud in the point cloud mesh, a normal vector on the fitted surface is obtained based on the point cloud and its nearest neighbors. The angle between any two normal vectors is calculated, and the dispersion evaluation index of all angle values ​​is used as the distribution difference of orientation. The dispersion evaluation index includes, but is not limited to, variance and coefficient of variation.

[0049] In the above scheme, considering the complex structure of large castings, with irregular structures such as deep cavities and concavities on the surface, these areas will block the laser during the 3D laser scanning process, resulting in data loss in the scanned 3D point cloud data, a decrease in point cloud density within the point cloud mesh, and a small amount of information. At the same time, considering that the scanning of complex areas on the surface of large castings may be affected by factors such as scanning angle or scanning speed under each scanning pose, there may also be real low-density point cloud meshes, multiple evaluation indicators were quantified from different perspectives to facilitate a comprehensive evaluation of the scanning difficulty of each point cloud mesh under the scanning pose.

[0050] Preferably, in this embodiment of the invention, the method for obtaining the coverage score of the point cloud mesh includes: First, for each pose, based on the pose data of the 3D laser scanning device and the scanned point cloud data, the point cloud mesh located within the field of view of the 3D laser scanning device is selected.

[0051] In this system, the field of view of a 3D laser scanning device is cone-shaped, and only meshes located within this cone can be scanned. Therefore, at each pose, the pose data of the 3D laser scanning device and the scanned point cloud data are used as inputs to the occlusion culling algorithm. A 3D view cone model of the 3D laser scanning device is constructed based on the pose data. Each 3D view cone model defines a cone-shaped field of view, and the point cloud mesh is then filtered based on this model.

[0052] As a specific example, in one implementation of this invention, a frustum culling algorithm is used to determine whether each point cloud mesh is located within the 3D frustum model of the 3D laser scanning device, and point cloud meshes located within the frustum field of view are selected as target meshes. The frustum culling algorithm is a commonly used occlusion culling algorithm in the field of point cloud data processing, and the specific process will not be described in detail.

[0053] Secondly, for each target grid, the coverage score of the target grid is obtained based on the spatial distance between the target grid and the 3D laser scanning device, as well as the surface normal vector of the target grid and the line of sight of the 3D laser scanning sensor.

[0054] The coverage score characterizes the completeness of the cone-shaped field of view (CFD) defined by the 3D laser scanning device for each target mesh at each pose. The smaller the angle between the surface normal vector of the target mesh and the line-of-sight direction of the 3D laser scanning sensor, the more complete the coverage of the target mesh by the CCD. Considering that the distance between the target mesh and the 3D laser scanning device also affects the completeness of the target mesh coverage, a distance weight for the target mesh is determined based on the spatial distance between the target mesh and the 3D laser scanning device and the upper limit of the effective working distance of the sensor in the laser scanning device. The greater the spatial distance, the smaller the distance weight. In other words, the closer the spatial distance between the target mesh and the 3D laser scanning device, and the smaller the angle, the higher the coverage score of the target mesh by the scanning range of the 3D laser device at the current pose, the higher the completeness of the target mesh coverage by the CCD at that pose, and the better the scanning effect.

[0055] Specifically, for each target mesh, the surface normal vector represents the orientation of the entire target mesh. The smaller the angle between this orientation and the sensor's line of sight, the closer the target mesh is to the center region of the 3D frustum model, and the greater the probability of it being completely covered. Since the angle between the surface normal vector of the target mesh and the sensor's line of sight ranges from 0° to 180° at each pose, a cosine function is used to quantize the angle. The smaller the angle, the larger the cosine function value, and the higher the coverage score.

[0056] In one specific implementation of this invention, the product of the distance weight and the cosine of the included angle is used as the coverage score of the target grid.

[0057] In one specific implementation of this invention, the ratio of the spatial distance between the target mesh and the three-dimensional laser scanning device to the upper limit of the effective working distance is calculated, and the difference between 1 and the ratio is used as the distance weight.

[0058] In one specific implementation of this invention, the average vector of the normal vectors of all point clouds within the target mesh on the fitting plane is used as the surface normal vector of the target mesh.

[0059] In other implementations of this invention, plane fitting can be performed on all point clouds within the target mesh, and the normal vector of the fitted plane can be used as the surface normal vector of the target mesh. Plane fitting methods include, but are not limited to, least squares fitting and principal component analysis. The specific fitting process is a well-known technique in the art, and will not be described in detail here.

[0060] In the above scheme, based on the point cloud data and pose data within the point cloud mesh, an occlusion algorithm is used to filter the target mesh. Coverage scores are obtained for both target and non-target meshes using different methods, avoiding meaningless distance and angle calculations and improving computational efficiency. The coverage score reflects the actual coverage capability of the 3D laser scanning equipment when scanning the surface of a large casting in the current pose, achieving accurate quantification of the coverage integrity during scanning in the current pose.

[0061] Preferably, in this embodiment of the invention, the method for obtaining the optimal scanning pose for the next scan includes: First, the pose of the 3D laser scanning device is initialized before scanning begins.

[0062] Specifically, before scanning and inspecting large castings, the 3D laser scanning equipment is placed on the widest side at both ends of the large casting to ensure that the initial position of the 3D scanning equipment can directly observe the surface of the casting.

[0063] Secondly, during the moving scan, a set of candidate poses for the current pose is obtained, and the objective function value is obtained based on the scan integrity evaluation index and coverage score of all target meshes under each candidate pose in the candidate pose set.

[0064] Since scanning path planning using local optimum decision-making requires first obtaining an optimization range for the current pose, i.e., a candidate set, to plan the optimal pose for the next scan, this embodiment of the invention uses a clustering algorithm to obtain a candidate pose set based on the point cloud data obtained from the scan under the current pose and the pose data of the 3D laser device. The objective function value characterizes the suitability of each candidate pose in the candidate pose set as the optimal candidate pose for the next scan. The larger the objective function value of a candidate pose, the more the 3D laser scanning device focuses on areas with high scanning difficulty and the more complete the obtained point cloud data when scanning large castings under that candidate pose.

[0065] In some possible implementations of the embodiments of this application, the specific process of obtaining the target function value may include the following two steps: (1) For each candidate pose, the scan observation value of each target grid is obtained based on the scan integrity evaluation index of each target grid under each candidate pose and the coverage score of each target grid.

[0066] Among them, the scan observation value represents the expected scanning benefit when scanning each target mesh in each candidate pose. For the target mesh, the larger the scan observation value, the more complete the coverage of the target mesh by the cone field of view of the 3D laser scanning device during scanning, and the smaller the scan integrity evaluation index, the greater the expected benefit when scanning the target mesh in the candidate pose.

[0067] In one specific implementation of this invention, the specific calculation formula for obtaining the scan observation value of each target grid is as follows:

[0068] In the formula, This represents the scan observation value of the i-th target grid. This represents the scan integrity evaluation index for the i-th target grid. This represents the coverage score of the i-th target grid.

[0069] (2) The product of the scan observations of all target grids under each candidate pose and the number of target grids is used as the objective function value of each candidate pose.

[0070] The purpose of scanning observations is to guide the 3D laser scanning equipment to prioritize areas that are difficult to scan and to ensure coverage integrity, and to supplement the scanned results rather than scan simple areas. At the same time, in order to ensure scanning efficiency, the size of the scanning area should also be considered. The larger the scanning area in each scan, the more target meshes should be, and the greater the possibility that the candidate pose will become the best candidate pose in the next scan.

[0071] Specifically, for each candidate pose, the sum of scan observations of all target grids is calculated, and the product of the sum and the number of target grids is used as the objective function value for each candidate pose.

[0072] Then, based on the point cloud data scanned in the current pose and the objective function value, the optimal pose for the next scan is obtained using a local optimum decision algorithm.

[0073] Specifically, after obtaining the candidate pose set for the current pose, this set serves as the optimization range for determining the optimal scanning pose for the next scan. The objective function value for each candidate pose is obtained, and the candidate pose with the largest objective function value in the set is taken as the optimal pose for the next scan. A mobile platform and a multi-axis robotic arm are used to move the 3D laser scanning equipment to the optimal pose to scan the point cloud data of the large casting, and the process is iterated to obtain the optimal pose for the next scan. In one specific implementation of this invention, the local optimal decision-making algorithm is a greedy algorithm.

[0074] It should be noted that if the objective function values ​​of multiple candidate poses are all maximum values, then the candidate pose closest to the current scan pose is selected as the best candidate pose for the next scan.

[0075] In some possible implementations of this invention, the iterative optimization process is as follows: (1) After obtaining the best candidate pose for the next scan, the robotic arm moves to the best candidate pose to perform the scan; (2) The point cloud data obtained by scanning in (1) is processed into a grid. The information content, curvature change rate index and normal vector discreteness evaluation index of the point cloud grid are updated according to the above steps. The scanning difficulty evaluation index of the point cloud grid is recalculated. (3) Recalculate the coverage score of the point cloud mesh and the scan integrity evaluation index, substitute them into the calculation formula of the objective function, and select the best candidate pose for the next scan. (4) Repeat steps (1)-(3) above to perform iterative optimization and complete multiple scans until the termination condition is met. In a specific implementation of this invention, the termination condition is set to the scan integrity evaluation index of the target mesh under the best candidate pose being greater than a preset evaluation threshold of 0.8.

[0076] In other implementations of this invention, the termination condition can also be set to the scan observation value of the target mesh under the best candidate pose being less than a preset revenue threshold of 0.1.

[0077] In the above scheme, a local optimum decision algorithm is used to obtain the optimal pose for the next scan, enabling the completion of the blind spot in the current scan. The core of the local optimum decision algorithm is "only considering the local optimum, ignoring the global optimum," while the path planning in this scheme aims to complete the scanned point cloud data. The goal is to effectively eliminate the scanning blind spot in the current scan and complex areas where laser occlusion may occur during the completion of the completion scan. Through several iterations of optimization, the completion scan is performed to ensure that the point cloud data obtained at the final stop is as complete and of higher quality as possible.

[0078] Preferably, in this embodiment of the invention, the method for obtaining the candidate pose set is as follows: First, based on the scanning difficulty evaluation index and coordinate information of the point cloud mesh, a clustering algorithm is used to divide all point cloud meshes into several clusters.

[0079] Since each cluster can represent a point cloud mesh with the same point cloud distribution, clustering can summarize scattered and difficult-to-scan meshes into several specific local regions, such as deep cavity regions and rib plate regions. This facilitates the subsequent determination of different candidate regions for each point cloud distribution.

[0080] Specifically, a parameter sequence for each point cloud grid is constructed using the scanning difficulty evaluation index and coordinate information of each grid. The Euclidean distance between the parameter sequences of two grids is used as the distance metric for clustering. Based on the parameter sequences of all point cloud grids, a clustering algorithm is used to divide all grids into several clusters.

[0081] In one specific implementation of this invention, AP (Affinity Propagation) clustering is used to divide all point cloud meshes into several clusters. The AP clustering algorithm is a commonly used technique in the field of data processing, and the specific process will not be described in detail.

[0082] In other implementations of this invention, the clustering algorithm can also be k-means clustering, decision tree clustering, or hierarchical clustering. This invention does not impose any special restrictions on this. Sequence clustering is a commonly used technique in the field of data processing, and the specific process will not be described in detail.

[0083] Secondly, for each cluster, the geometric center of the cluster is obtained. A spherical region is determined with the geometric center as the center and a preset scanning distance as the radius. Several points are sampled on the surface of the spherical region as candidate observation positions.

[0084] Specifically, based on the coordinate information of the point cloud grid corresponding to all parameter sequences within the cluster, the geometric center of the cluster is determined. A spherical region is then defined with the geometric center as the center and a preset scanning distance as the radius. Several points are generated on the surface of this spherical region as candidate observation locations through uniform or random sampling. The calculation of the geometric center's coordinates is a well-known technique, and the specific process will not be elaborated further.

[0085] In one specific implementation of this invention, the preset scanning distance is set to 0.7 times the upper limit of the effective working distance of the sensor in the laser scanning device.

[0086] Then, for each candidate observation position, the orientation angle that makes the optical axis of the laser sensor point to the geometric center is obtained.

[0087] The orientation angle refers to the angle parameter required by the controller of the multi-axis robotic arm to ensure that the conical field of view of the 3D laser scanning device covers all point cloud grids corresponding to the parameter sequence in the cluster.

[0088] Specifically, for each candidate observation location, the difference vector between the geometric center of the cluster corresponding to the sphere where the observation location is located and the coordinate information of the candidate observation location is calculated. The direction of this difference vector is the target direction of the optical axis of the laser sensor.

[0089] Furthermore, the difference vector is normalized to obtain the corresponding unit vector. Based on unit vectors and preset auxiliary vectors A rotation matrix R is constructed, with its three axes representing forward, right, and upward directions, respectively. The rotation matrix R is then converted into Euler angles to obtain the roll angle, pitch angle, and yaw angle, which serve as the orientation angles required by the controller of the multi-axis robotic arm. The conversion of Euler angles is a well-known technique, and the specific process will not be elaborated further.

[0090] In one specific implementation of this invention, the rotation matrix is: ,in, , , The preset auxiliary vector is: .

[0091] Subsequently, each candidate pose includes a candidate observation position and the orientation angle at that candidate observation position.

[0092] Specifically, for each candidate pose, the coordinates of the candidate observation position determine the position of the 3D laser scanning device, and the orientation angle at the candidate observation position represents the direction in which the sensor of the 3D laser scanning device is looking. Therefore, a candidate pose consists of two parts: the coordinates of the candidate observation position and the orientation angle at each candidate observation position.

[0093] Finally, the set of all candidate poses is called the candidate pose set.

[0094] Specifically, for each cluster, several candidate poses are generated using the above method, and the set of candidate poses from all clusters is taken as the candidate pose set.

[0095] In the above scheme, a candidate pose set is generated by clustering the point cloud mesh. This means that when planning the scanning path based on the candidate pose set, the candidate poses are no longer randomly and blindly generated for optimization. Instead, the optimization range is restricted from an infinite space to a finite set of candidate poses, thus solving the problem that the simple greedy algorithm has too much computation and is prone to getting trapped in local optima.

[0096] In some possible implementations of this invention, the method for obtaining the scanning difficulty evaluation index of point cloud mesh is as follows: The point cloud density of each point cloud grid is obtained separately, and the point cloud density of each point cloud grid is normalized based on the point cloud density of all point cloud grids. In a specific implementation of this invention, Min-Max normalization is used.

[0097] Based on the normalization results, the specific formula for calculating the scanning difficulty evaluation index of the point cloud mesh is as follows:

[0098] In the formula, This represents the evaluation index of the scanning difficulty of the point cloud mesh, where 'a' represents the normalized result of the point cloud density of the mesh. b and c represent the rate of change of curvature of the point cloud mesh and the discreteness evaluation index of the normal vector, respectively. For the sigmoid function, , as well as Let the first, second, and third weights be respectively, satisfying the following conditions: In one specific implementation of this invention, the first, second, and third weights are each set to one-third.

[0099] It should be noted that the reason for using the sigmoid function is that adding 1 to 'a' in the denominator is to prevent the denominator from being zero. In this case, the first weighted object... The value range of b is 0.5-1. In order to ensure that the value range of the three weighted objects is consistent, the sigmoid function is used to map the function values ​​of b and c within the range of 0.5-1.

[0100] In some possible implementations of this invention, the method for obtaining the scan integrity evaluation index of the point cloud mesh is as follows: after obtaining the scan difficulty evaluation index V and the coverage score H of the point cloud mesh, the scan integrity evaluation index of the point cloud mesh is obtained through weighted fusion. The specific calculation formula is:

[0101] In the formula, B represents the scanning integrity evaluation index of the point cloud mesh. To adjust the parameters and prevent the denominator from being 0, the value range is set to... In one specific implementation of this invention, an empirical value is taken. .

[0102] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0103] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A mobile three-dimensional laser scanning inspection system for large castings, characterized in that, The system includes: The data acquisition module is used to acquire the pose data of the 3D laser scanning device and the point cloud data of the large casting scanned by the 3D laser scanning device in each pose; the point cloud data is processed into a grid to obtain several point cloud grids; the pose data includes the line of sight angle and device position of the 3D laser scanning device in each pose. The scanning difficulty assessment module is used to obtain the scanning difficulty evaluation index of the point cloud mesh at each pose, based on the information content of the point cloud in the point cloud mesh, the local geometric features of the point cloud on the fitting plane, and the distribution differences of the orientation of the point cloud on the fitting plane. The scan integrity assessment module is used to obtain the coverage score of the point cloud mesh based on the point cloud data within the point cloud mesh and the pose data of the 3D laser scanning device at each pose; and to obtain the scan integrity assessment index of the point cloud mesh based on the scan difficulty evaluation index and the coverage score. The scanning path planning module is used to obtain the optimal scanning pose for the next scan based on the current pose data, the coverage score, and the scan integrity evaluation index during the moving scan process, using a local optimal decision algorithm. The method for obtaining the coverage score of the point cloud mesh includes: For each pose, based on the pose data of the 3D laser scanning device and the scanned point cloud data, the point cloud mesh located within the field of view of the 3D laser scanning device is selected. Use any selected point cloud mesh as a target mesh; For each target grid, calculate the spatial distance between the target grid and the 3D laser scanning device; Calculate the angle between the surface normal vector of the target mesh and the sensor's line-of-sight angle during 3D laser scanning; The coverage score is negatively correlated with the spatial distance and positively correlated with the cosine of the included angle. Set the coverage score of any non-target point cloud mesh to the preset coverage score.

2. The mobile three-dimensional laser scanning inspection system for large castings according to claim 1, characterized in that, For each point cloud grid, the point cloud density is obtained based on the number of points and the area within the point cloud grid, and is used as the information quantity. The rate of change of curvature is obtained based on the distribution of curvature of the point cloud within the point cloud mesh on the fitting plane, and is used as the distribution difference of the local geometric features. The discreteness evaluation index of the normal vector is obtained based on the distribution of the normal vector of the point cloud within the point cloud mesh on the fitting plane, which serves as the distribution difference of the orientation.

3. The mobile three-dimensional laser scanning inspection system for large castings according to claim 1, characterized in that, The method for obtaining the optimal scanning pose for the next scan includes: Before starting the scan, the pose of the 3D laser scanning device is initialized; During the moving scan, a set of candidate poses for the current pose is obtained, and the objective function value is obtained based on the scan integrity evaluation index and coverage score of all target meshes under each candidate pose in the candidate pose set. Based on the point cloud data scanned in the current pose and the objective function value, the optimal pose for the next scan is obtained using a local optimum decision algorithm.

4. The mobile three-dimensional laser scanning inspection system for large castings according to claim 3, characterized in that, The method for obtaining the objective function value is as follows: For each candidate pose, the scan observation value of each target grid is obtained based on the scan integrity evaluation index of each target grid under each candidate pose and the coverage score of each target grid. The product of the scan observations of all target grids under each candidate pose and the number of target grids is used as the objective function value for each candidate pose.

5. The mobile three-dimensional laser scanning inspection system for large castings according to claim 3, characterized in that, The method for obtaining the candidate pose set is as follows: Based on the scanning difficulty evaluation index and coordinate information of point cloud mesh, a clustering algorithm is used to divide all point cloud mesh into several clusters; For each cluster, the geometric center of the cluster is obtained. A spherical region is determined with the geometric center as the center and a preset scanning distance as the radius. Several points are sampled on the surface of the spherical region as candidate observation positions. For each candidate observation location, obtain the orientation angle that makes the optical axis of the laser sensor point towards the geometric center; Each candidate pose includes a candidate observation position and the orientation angle at that candidate observation position; The set of all candidate poses is called the candidate pose set.

6. The mobile three-dimensional laser scanning inspection system for large castings according to claim 4, characterized in that, The method for obtaining the optimal pose for the next scan is as follows: the candidate pose with the largest objective function value in the candidate pose set is taken as the optimal pose for the next scan.

7. The mobile three-dimensional laser scanning inspection system for large castings according to claim 5, characterized in that, The preset scanning distance is less than the upper limit of the effective scanning distance of the three-dimensional laser scanning device.

8. The mobile three-dimensional laser scanning inspection system for large castings according to claim 5, characterized in that, The sampling generation method is either random sampling or uniform sampling.

Images