An interactive scan viewpoint online generation planning method

Abstract

The application discloses an interactive scanning viewpoint online generation planning method, and belongs to the technical field of assembly state detection, and comprises the following steps: a, pasting a circle of coding blocks at the boundary of a scanning area; b, completing global three-dimensional reconstruction of all coding blocks; c, setting pose adjustment parameters of a coordinate system, and calculating an adjusted coordinate system homogeneous matrix; d, determining the connection order between reference sampling points according to a continuous scanning type, generating other sampling points between the reference sampling points and corresponding sampling directions; e, generating scanning viewpoints and corresponding scanning directions by using sampling point positions and sampling directions according to an effective scanning distance; and f, completing online generation planning of the scanning viewpoints. Through online generation of the scanning viewpoints, the application can realize self-positioning of a scanning device, and can also realize fast generation and planning of a large-range scanning viewpoint in a non-structure environment without a model, and has good applicability.

Description

Technical Field

[0001] This invention relates to the field of assembly condition detection technology, and in particular to an interactive online scanning viewpoint generation planning method. Background Technology

[0002] The aircraft's surface shape affects aerodynamic efficiency and aerodynamic noise levels, significantly influencing aircraft performance. However, after the assembly of heavy components such as the vertical tail, wings, and engines, the fuselage's frame structure inevitably undergoes structural deformation, impacting the aircraft's shape and performance. Therefore, during final aircraft assembly, a 3D scan of the aircraft's shape is necessary to verify whether its manufactured shape meets design requirements.

[0003] The aircraft assembly site is a non-structural environment, making it difficult to perform 3D scanning of the aircraft using fixed measuring mechanisms. Mobile scanning equipment is required to adapt to the aircraft's position and attitude. However, the large size and three-dimensional shape of aircraft make manual scanning inefficient and labor-intensive. Fully automated scanning methods struggle to achieve self-positioning on the smooth, textureless surface of the aircraft, making it impossible to plan the scanning viewpoint.

[0004] Existing technologies require the use of a digital model of the target object before performing automatic viewpoint calculation and planning, which presents the following problems: It cannot be applied to situations without a model, i.e., it cannot be used for temporary needs; The calculation process is complex and faces the problem of long computation time. The scanning mode cannot be adjusted according to the actual scanning needs, especially when a mixture of fine and coarse scanning is required.

[0005] Chinese patent document with publication number CN116901079A and publication date of October 20, 2023 discloses a robot path planning system based on scanner vision guidance, characterized in that it includes: a 3D scanner, a path planning module, a vision guidance module, and a robot; The 3D scanner is fixedly connected to the end of the robot and is used to create 3D data of the surface of the workpiece to be inspected. The path planning module is communicatively connected to the 3D scanner and is used to determine the optimal motion trajectory of the robot based on a preset scanning viewpoint. The vision guidance module is used to determine the target scanning viewpoint in the optimal motion trajectory, visualize the pose deviation between the 3D scanner and the target scanning viewpoint in real time, and guide the 3D scanner to a designated position. The robot has multiple degrees of freedom and is used to adjust the pose of the 3D scanner according to the pose deviation until it coincides with the target viewpoint.

[0006] The robot path planning system and method disclosed in this patent document, based on scanner vision guidance, improves the efficiency of robot path planning teaching. However, it is difficult to achieve rapid generation and planning of large-scale scanning viewpoints without a model in unstructured environments, resulting in poor applicability. Summary of the Invention

[0007] To overcome the shortcomings of the prior art, this invention provides an interactive online scanning viewpoint generation and planning method. By generating scanning viewpoints online, this invention can achieve both self-positioning of the scanning device and rapid generation and planning of large-scale scanning viewpoints without models in unstructured environments, thus exhibiting good applicability.

[0008] This invention is achieved through the following technical solution: An interactive online planning method for scanning viewpoint generation, characterized by the following steps: a. Determine the area to be scanned on the aircraft's shape, and attach a ring of coded blocks around the boundary of the area to be scanned; b. Manually use a photogrammetry instrument to photograph all coded blocks, complete the global 3D reconstruction of all coded blocks, extract the number of each coded block, and establish the corresponding coordinate system at the center point of each coded block; c. Set the pose adjustment parameters of the coordinate system so that the center point of the coordinate system is located at the target scanning boundary, the Z-axis of the coordinate system is the target scanning viewpoint direction, and calculate the homogeneous matrix of the adjusted coordinate system. d. Using the center point of the coordinate system as the reference sampling point, the corresponding code block number as the reference sampling point number, and the Z-axis of the coordinate system as the reference sampling direction, the connection order between the reference sampling points is determined according to the continuous scanning type, and the interpolation method is used to generate other sampling points and corresponding sampling directions between the reference sampling points. e. Based on the effective scanning distance, generate the scanning viewpoint and corresponding scanning direction using the sampling point position and sampling direction; f. Using the connection order between the reference sampling points, number the generated scanning viewpoints according to the scanning order to complete the online generation planning of the scanning viewpoints.

[0009] In step a, attaching a ring of coding blocks at the boundary of the area to be scanned means making the X-axis direction of the coding blocks parallel to the tangent of the adjacent boundary, and making the Y-axis direction of the coding blocks away from the boundary of the area to be scanned.

[0010] In step c, the adjusted coordinate system homogeneous matrix is ​​calculated using Equation 1; Formula 1; in, The adjusted homogeneous matrix of the coordinate system. The coordinate system homogeneous matrix of the coded block before adjustment. To calculate the homogeneous matrix for translation along the X-axis, To calculate the homogeneous matrix for translation along the Y-axis, To calculate the homogeneous matrix for rotation about the Z-axis, To calculate the homogeneous matrix for rotation about the X-axis, and These are all position adjustment parameters. and These are all attitude adjustment parameters.

[0011] The homogeneous matrix calculated by translation along the X-axis is calculated using Equation 2; Formula 2.

[0012] The homogeneous matrix calculated by translation along the Y-axis is calculated using Equation 3; Formula 3.

[0013] The homogeneous matrix for rotation around the Z-axis is calculated using Equation 4; Formula 4.

[0014] The homogeneous matrix for rotation around the X-axis is calculated using Equation 5; Formula 5.

[0015] In step d, the continuous scan types include region scan and border scan.

[0016] During the region scan, the attitude adjustment parameters and All are 0; during bounding box scanning, the pose adjustment parameters are... =0, It is 45 or -45.

[0017] In step e, the scanning viewpoint and the corresponding scanning direction are determined by Equation 6; Formula 6; in, For scanning viewpoints, For sampling points, For effective scanning distance, Sampling direction, This indicates the scanning direction.

[0018] The beneficial effects of this invention are mainly reflected in the following aspects: 1. This invention comprises: a) determining the area to be scanned on the aircraft's shape and attaching a ring of coded blocks at the boundary of the area; b) manually photographing all coded blocks using a photogrammetry instrument and completing global 3D reconstruction of all coded blocks, extracting the numbers of each coded block, and establishing a corresponding coordinate system at the center point of each coded block; c) setting the pose adjustment parameters of the coordinate system so that the center point of the coordinate system is located at the target scanning boundary, the Z-axis of the coordinate system is the target scanning viewpoint direction, and calculating the homogeneous matrix of the adjusted coordinate system; d) using the center point of the coordinate system as the reference sampling point, the corresponding coded block number as the reference sampling point number, and the Z-axis of the coordinate system as the reference sampling direction, based on... Based on the continuous scanning type, the connection order between reference sampling points is determined, and interpolation is used to generate other sampling points and corresponding sampling directions between the reference sampling points; e. Based on the effective scanning distance, the scanning viewpoints and corresponding scanning directions are generated using the sampling point positions and sampling directions; f. Using the connection order between the reference sampling points, the generated scanning viewpoints are numbered according to the scanning order to complete the online generation planning of scanning viewpoints. Compared with the existing technology, the online generation of scanning viewpoints can realize the self-positioning of scanning equipment and the rapid generation and planning of large-scale scanning viewpoints without models in unstructured environments, which has good applicability.

[0019] 2. This invention solves the problem that mobile scanning devices have difficulty in achieving self-positioning and planning scanning viewpoints when automatically scanning the shape of aircraft.

[0020] 3. This invention, by introducing the manual application of coded blocks at the scanning boundaries on the aircraft surface and interactively determining the scanning viewpoint and scanning sequence, achieves rapid online generation and planning of scanning viewpoints in the absence of a model.

[0021] 4. This invention, through topological distribution of coding blocks and global 3D reconstruction, improves the efficiency of benchmark construction and the accuracy of coding recognition compared to the traditional marker localization method.

[0022] 5. This invention, based on coordinate system adaptive calibration using a homogeneous matrix, can achieve real-time matching between the scanning direction and the surface normal, effectively eliminating the point cloud loss problem caused by viewpoint deviation in traditional methods.

[0023] 6. This invention achieves precision, efficiency, and full-process controllability in 3D scanning of aerospace components by integrating spatial benchmark construction, dynamic pose adjustment, and path planning. Attached Figure Description

[0024] The present invention will now be further described in detail with reference to the accompanying drawings and specific embodiments: Figure 1 This is a schematic diagram of the coded block pasting of the present invention; Figure 2 This is a schematic diagram of the reference sampling points and their connection order in this invention; Figure 3 This is a schematic diagram showing the connection order of the encoding blocks and reference sampling points during border scanning in this invention. Figure 4 This is a schematic diagram showing the sampling points and corresponding sampling directions generated by the interpolation method of this invention; Figure 5 This is a schematic diagram of the scanning viewpoint generation result of the present invention; Marked in the image: 1. First sampling point 2. Second sampling point 3. The third sampling point, 4. The fourth sampling point, 1. First sampling direction, 2. Second sampling direction, 3. Third sampling direction, 4. Fourth sampling direction, 1. First scanning direction, 2. Second scanning direction, 3. Third scanning direction, 1. First scanning viewpoint 2. Second scanning viewpoint 3. Third scanning viewpoint. Detailed Implementation

[0025] Example 1 See Figures 1-5 An interactive online planning generation method for scanning viewpoints includes the following steps: a. Determine the area to be scanned on the aircraft's shape, and attach a ring of coded blocks around the boundary of the area to be scanned; b. Manually use a photogrammetry instrument to photograph all coded blocks, complete the global 3D reconstruction of all coded blocks, extract the number of each coded block, and establish the corresponding coordinate system at the center point of each coded block; c. Set the pose adjustment parameters of the coordinate system so that the center point of the coordinate system is located at the target scanning boundary, the Z-axis of the coordinate system is the target scanning viewpoint direction, and calculate the homogeneous matrix of the adjusted coordinate system. d. Using the center point of the coordinate system as the reference sampling point, the corresponding code block number as the reference sampling point number, and the Z-axis of the coordinate system as the reference sampling direction, the connection order between the reference sampling points is determined according to the continuous scanning type, and the interpolation method is used to generate other sampling points and corresponding sampling directions between the reference sampling points. e. Based on the effective scanning distance, generate the scanning viewpoint and corresponding scanning direction using the sampling point position and sampling direction; f. Using the connection order between the reference sampling points, number the generated scanning viewpoints according to the scanning order to complete the online generation planning of the scanning viewpoints.

[0026] This embodiment is the most basic implementation method. a) Determine the area to be scanned on the aircraft's shape and attach a ring of coded blocks along the boundary of the area; b) Manually use a photogrammetry instrument to photograph all coded blocks and complete the global 3D reconstruction of all coded blocks, extract the number of each coded block, and establish a corresponding coordinate system at the center point of each coded block; c) Set the pose adjustment parameters of the coordinate system so that the center point of the coordinate system is located at the target scanning boundary, the Z-axis of the coordinate system is the target scanning viewpoint direction, and calculate the homogeneous matrix of the adjusted coordinate system; d) Use the center point of the coordinate system as the reference sampling point, the corresponding coded block number as the reference sampling point number, and the Z-axis of the coordinate system as the reference sampling point. The process involves several steps: a) determining the connection order between reference sampling points based on the continuous scanning type, and using interpolation to generate other sampling points and corresponding sampling directions between the reference sampling points; b) generating scanning viewpoints and corresponding scanning directions based on the effective scanning distance and the sampling point positions and directions; c) numbering the generated scanning viewpoints according to the scanning sequence using the connection order between the reference sampling points, thus completing the online generation planning of scanning viewpoints. Compared with existing technologies, online generation of scanning viewpoints enables both self-positioning of the scanning device and rapid generation and planning of large-scale scanning viewpoints without models in unstructured environments, demonstrating good applicability.

[0027] Example 2 See Figures 1-5 An interactive online planning generation method for scanning viewpoints includes the following steps: a. Determine the area to be scanned on the aircraft's shape, and attach a ring of coded blocks around the boundary of the area to be scanned; b. Manually use a photogrammetry instrument to photograph all coded blocks, complete the global 3D reconstruction of all coded blocks, extract the number of each coded block, and establish the corresponding coordinate system at the center point of each coded block; c. Set the pose adjustment parameters of the coordinate system so that the center point of the coordinate system is located at the target scanning boundary, the Z-axis of the coordinate system is the target scanning viewpoint direction, and calculate the homogeneous matrix of the adjusted coordinate system. d. Using the center point of the coordinate system as the reference sampling point, the corresponding code block number as the reference sampling point number, and the Z-axis of the coordinate system as the reference sampling direction, the connection order between the reference sampling points is determined according to the continuous scanning type, and the interpolation method is used to generate other sampling points and corresponding sampling directions between the reference sampling points. e. Based on the effective scanning distance, generate the scanning viewpoint and corresponding scanning direction using the sampling point position and sampling direction; f. Using the connection order between the reference sampling points, number the generated scanning viewpoints according to the scanning order to complete the online generation planning of the scanning viewpoints.

[0028] Preferably, in step a, attaching a ring of coding blocks at the boundary of the area to be scanned means making the X-axis direction of the coding blocks parallel to the tangent of the adjacent boundary, and making the Y-axis direction of the coding blocks away from the boundary of the area to be scanned.

[0029] In step c, the adjusted coordinate system homogeneous matrix is ​​calculated using Equation 1; Formula 1; in, The adjusted homogeneous matrix of the coordinate system. The coordinate system homogeneous matrix of the coded block before adjustment. To calculate the homogeneous matrix for translation along the X-axis, To calculate the homogeneous matrix for translation along the Y-axis, To calculate the homogeneous matrix for rotation about the Z-axis, To calculate the homogeneous matrix for rotation about the X-axis, and These are all position adjustment parameters. and These are all attitude adjustment parameters.

[0030] This embodiment is a preferred implementation method, which solves the problem that mobile scanning devices have difficulty in self-positioning and planning scanning viewpoints when performing automatic scanning of aircraft shapes.

[0031] Example 3 See Figures 1-5 An interactive online planning generation method for scanning viewpoints includes the following steps: a. Determine the area to be scanned on the aircraft's shape, and attach a ring of coded blocks around the boundary of the area to be scanned; b. Manually use a photogrammetry instrument to photograph all coded blocks, complete the global 3D reconstruction of all coded blocks, extract the number of each coded block, and establish the corresponding coordinate system at the center point of each coded block; c. Set the pose adjustment parameters of the coordinate system so that the center point of the coordinate system is located at the target scanning boundary, the Z-axis of the coordinate system is the target scanning viewpoint direction, and calculate the homogeneous matrix of the adjusted coordinate system. d. Using the center point of the coordinate system as the reference sampling point, the corresponding code block number as the reference sampling point number, and the Z-axis of the coordinate system as the reference sampling direction, the connection order between the reference sampling points is determined according to the continuous scanning type, and the interpolation method is used to generate other sampling points and corresponding sampling directions between the reference sampling points. e. Based on the effective scanning distance, generate the scanning viewpoint and corresponding scanning direction using the sampling point position and sampling direction; f. Using the connection order between the reference sampling points, number the generated scanning viewpoints according to the scanning order to complete the online generation planning of the scanning viewpoints.

[0032] In step a, attaching a ring of coding blocks at the boundary of the area to be scanned means making the X-axis direction of the coding blocks parallel to the tangent of the adjacent boundary, and making the Y-axis direction of the coding blocks away from the boundary of the area to be scanned.

[0033] In step c, the adjusted coordinate system homogeneous matrix is ​​calculated using Equation 1; Formula 1; in, The adjusted homogeneous matrix of the coordinate system. The coordinate system homogeneous matrix of the coded block before adjustment. To calculate the homogeneous matrix for translation along the X-axis, To calculate the homogeneous matrix for translation along the Y-axis, To calculate the homogeneous matrix for rotation about the Z-axis, To calculate the homogeneous matrix for rotation about the X-axis, and These are all position adjustment parameters. and These are all attitude adjustment parameters.

[0034] The homogeneous matrix calculated by translation along the X-axis is calculated using Equation 2; Formula 2.

[0035] The homogeneous matrix calculated by translation along the Y-axis is calculated using Equation 3; Formula 3.

[0036] The homogeneous matrix for rotation around the Z-axis is calculated using Equation 4; Formula 4.

[0037] The homogeneous matrix for rotation around the X-axis is calculated using Equation 5; Formula 5.

[0038] This embodiment is another preferred implementation method. By introducing manual application of coded blocks at the scanning boundaries on the aircraft surface and interactively determining the scanning viewpoint and scanning order, rapid online generation and planning of scanning viewpoints in the absence of a model is achieved.

[0039] Example 4 See Figures 1-5 An interactive online planning generation method for scanning viewpoints includes the following steps: a. Determine the area to be scanned on the aircraft's shape, and attach a ring of coded blocks around the boundary of the area to be scanned; b. Manually use a photogrammetry instrument to photograph all coded blocks, complete the global 3D reconstruction of all coded blocks, extract the number of each coded block, and establish the corresponding coordinate system at the center point of each coded block; c. Set the pose adjustment parameters of the coordinate system so that the center point of the coordinate system is located at the target scanning boundary, the Z-axis of the coordinate system is the target scanning viewpoint direction, and calculate the homogeneous matrix of the adjusted coordinate system. d. Using the center point of the coordinate system as the reference sampling point, the corresponding code block number as the reference sampling point number, and the Z-axis of the coordinate system as the reference sampling direction, the connection order between the reference sampling points is determined according to the continuous scanning type, and the interpolation method is used to generate other sampling points and corresponding sampling directions between the reference sampling points. e. Based on the effective scanning distance, generate the scanning viewpoint and corresponding scanning direction using the sampling point position and sampling direction; f. Using the connection order between the reference sampling points, number the generated scanning viewpoints according to the scanning order to complete the online generation planning of the scanning viewpoints.

[0040] In step a, attaching a ring of coding blocks at the boundary of the area to be scanned means making the X-axis direction of the coding blocks parallel to the tangent of the adjacent boundary, and making the Y-axis direction of the coding blocks away from the boundary of the area to be scanned.

[0041] In step c, the adjusted coordinate system homogeneous matrix is ​​calculated using Equation 1; Formula 1; in, The adjusted homogeneous matrix of the coordinate system. The coordinate system homogeneous matrix of the coded block before adjustment. To calculate the homogeneous matrix for translation along the X-axis, To calculate the homogeneous matrix for translation along the Y-axis, To calculate the homogeneous matrix for rotation about the Z-axis, To calculate the homogeneous matrix for rotation about the X-axis, and These are all position adjustment parameters. and These are all attitude adjustment parameters.

[0042] The homogeneous matrix calculated by translation along the X-axis is calculated using Equation 2; Formula 2.

[0043] The homogeneous matrix calculated by translation along the Y-axis is calculated using Equation 3; Formula 3.

[0044] The homogeneous matrix for rotation around the Z-axis is calculated using Equation 4; Formula 4.

[0045] The homogeneous matrix for rotation around the X-axis is calculated using Equation 5; Formula 5.

[0046] In step d, the continuous scan types include region scan and border scan.

[0047] More preferably, during the region scanning, the attitude adjustment parameters and All are 0; during bounding box scanning, the pose adjustment parameters are... =0, It is 45 or -45.

[0048] In step e, the scanning viewpoint and the corresponding scanning direction are determined by Equation 6; Formula 6; in, For scanning viewpoints, For sampling points, For effective scanning distance, Sampling direction, This indicates the scanning direction.

[0049] This embodiment is the best implementation method. By using topological distribution of coding blocks and global 3D reconstruction, it improves the efficiency of benchmark construction and the accuracy of coding recognition compared to the traditional marker localization method.

[0050] Coordinate system adaptive calibration based on homogeneous matrix can achieve real-time matching between scanning direction and surface normal, effectively eliminating the point cloud missing problem caused by viewpoint deviation in traditional methods.

[0051] By integrating spatial benchmark construction, dynamic pose adjustment, and path planning, the accuracy, efficiency, and full-process controllability of 3D scanning of aerospace components have been achieved.

[0052] The basic principle of this invention is as follows: A collaborative control mechanism based on coded block spatial reference network and path planning is used to achieve high-precision scanning: First, coded block markers are deployed on the boundary of the area to be scanned. A global coordinate system is constructed through offline algorithms of multi-view photogrammetry and 3D reconstruction. Then, the scanning direction is accurately matched with the surface normal through dynamic calibration of the pose matrix. Combined with interpolation algorithm, a continuous scanning path with curvature adaptation is generated. Topological coding is used to achieve full-path traceability control. Based on the effective scanning distance, the scanning viewpoint and the corresponding scanning direction are generated using the sampling point position and sampling direction. Finally, the generated scanning viewpoints are numbered according to the scanning order using the connection order between reference sampling points, realizing online generation and planning of scanning viewpoints.

[0053] When performing automatic scanning of aircraft shapes, the online generation of scanning viewpoints is achieved through interactive methods. This enables both the self-positioning of the scanning equipment and the rapid generation and planning of large-scale scanning viewpoints in unstructured environments without models. This breaks through the original limitations of the scanning equipment and expands the scope of application of automatic scanning equipment.

Claims

1. An interactive online planning method for scanning viewpoint generation, characterized in that, Includes the following steps: a. Determine the area to be scanned on the aircraft's shape, and attach a ring of coded blocks around the boundary of the area to be scanned; b. Manually use a photogrammetry instrument to photograph all coded blocks, complete the global 3D reconstruction of all coded blocks, extract the number of each coded block, and establish the corresponding coordinate system at the center point of each coded block; c. Set the pose adjustment parameters of the coordinate system so that the center point of the coordinate system is located at the target scanning boundary, the Z-axis of the coordinate system is the target scanning viewpoint direction, and calculate the homogeneous matrix of the adjusted coordinate system. d. Using the center point of the coordinate system as the reference sampling point, the corresponding code block number as the reference sampling point number, and the Z-axis of the coordinate system as the reference sampling direction, the connection order between the reference sampling points is determined according to the continuous scanning type, and the interpolation method is used to generate other sampling points and corresponding sampling directions between the reference sampling points. e. Based on the effective scanning distance, generate the scanning viewpoint and corresponding scanning direction using the sampling point position and sampling direction; f. Using the connection order between the reference sampling points, number the generated scanning viewpoints according to the scanning order to complete the online generation planning of the scanning viewpoints; In step c, the adjusted coordinate system homogeneous matrix is ​​calculated using Equation 1; Formula 1; in, The adjusted homogeneous matrix of the coordinate system. This is the homogeneous matrix of the coding block coordinate system before adjustment. To calculate the homogeneous matrix for translation along the X-axis, To calculate the homogeneous matrix for translation along the Y-axis, To calculate the homogeneous matrix for rotation about the Z-axis, To calculate the homogeneous matrix for rotation about the X-axis, and These are all position adjustment parameters. and All of these are attitude adjustment parameters; The homogeneous matrix calculated by translation along the X-axis is calculated using Equation 2; Formula 2; The homogeneous matrix calculated by translation along the Y-axis is calculated using Equation 3; Formula 3; The homogeneous matrix for rotation around the Z-axis is calculated using Equation 4; Equation 4; The homogeneous matrix for rotation around the X-axis is calculated using Equation 5; Formula 5.

2. The interactive scanning viewpoint online generation planning method according to claim 1, characterized in that: In step a, attaching a ring of coding blocks at the boundary of the area to be scanned means making the X-axis direction of the coding blocks parallel to the tangent of the adjacent boundary, and making the Y-axis direction of the coding blocks away from the boundary of the area to be scanned.

3. The interactive online scanning viewpoint generation planning method according to claim 1, characterized in that: In step d, the continuous scan types include region scan and border scan.

4. The interactive online scanning viewpoint generation planning method according to claim 3, characterized in that: During the region scan, the attitude adjustment parameters and All are 0; during bounding box scanning, the pose adjustment parameters are... =0, It is 45 or -45.

5. The interactive online scanning viewpoint generation planning method according to claim 1, characterized in that: In step e, the scanning viewpoint and the corresponding scanning direction are determined by Equation 6; Formula 6; in, For scanning viewpoints, For sampling points, For effective scanning distance, Sampling direction, This indicates the scanning direction.

Images