A multi-aircraft cooperative pose detection method and device for aircraft panel parts

Abstract

This invention relates to a multi-aircraft collaborative pose detection method and apparatus for aircraft panel parts. The method includes acquiring scene point cloud data of the target part using multiple cameras; segmenting the background plane of the workbench where the target part is located using a random sampling consensus algorithm to extract target point clouds containing only the surface features of the target part; fusing the target point clouds from multiple cameras using a multi-view alignment strategy, obtaining an accurate transformation matrix through coarse and fine matching, mapping all target point clouds to the world coordinate system of the reference camera and superimposing and fusing them to form a complete global point cloud; and employing a multi-stage ICP pose matching strategy to register the global point cloud with a preset template point cloud of the target part, solving and outputting the pose parameters of the target part in the world coordinate system. This effectively avoids the local optimum dilemma that is easily trapped in single-camera detection, significantly improving the accuracy and stability of pose detection, and can be used in multiple fields such as robotic arms and total mass production control machines.

Description

Technical Field

[0001] This invention relates to the field of high-end equipment technology, specifically to a multi-aircraft collaborative pose detection method and device for aircraft panel parts. Background Technology

[0002] As the aerospace manufacturing industry moves towards higher precision and automation, the assembly of large and complex components (such as aircraft panels) places extremely high demands on the accuracy and robustness of target pose detection. Pose detection, as a core technology for robotic collaborative assembly, requires the accurate acquisition of the three-dimensional position and orientation information of parts to provide crucial data support for assembly operations.

[0003] However, existing pose detection technologies still have many limitations. Traditional single-machine pose detection relies on point cloud data from a single viewpoint, which is easily affected by object occlusion and missing surface textures, and cannot completely cover the target's 3D structure, resulting in insufficient pose calculation accuracy. At the same time, single-machine point clouds are susceptible to interference from environmental noise and sensor errors, and the detection results are prone to jumps, and it is difficult to escape the local optimum dilemma of the ICP algorithm.

[0004] In aerospace equipment manufacturing scenarios, the large size of components, complex assembly environments, and numerous signal interferences further exacerbate the difficulty of inspection. Existing single-machine inspection solutions lack efficient point cloud alignment and fusion mechanisms, making it difficult to effectively integrate multi-view data and fully leverage the advantages of multi-device collaboration.

[0005] In summary, there is an urgent need for a multi-machine collaborative pose detection method and system that can overcome the limitations of single-machine perspective, achieve efficient fusion of multi-view data, and improve the accuracy and stability of pose detection in complex scenarios, so as to meet the high-precision assembly requirements of aerospace equipment manufacturing. Summary of the Invention

[0006] This invention aims to solve the problems of single-machine perspective limitation, lack of efficient point cloud alignment and fusion mechanism, and insufficient detection accuracy in existing pose detection technologies, and provides a multi-machine collaborative pose detection method and device for aircraft panel parts.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A multi-aircraft cooperative pose detection method for aircraft panel components includes the following steps:

[0009] Multiple cameras are deployed at different angles around the target part to collect scene point cloud data of the target part. One camera is used as the reference camera, and the coordinate systems of the other cameras are fixedly related to the world coordinate system of the reference camera.

[0010] The random sampling consistency algorithm is used to segment the background plane of the workbench where the target part is located on the scene point cloud data. After removing the points in the plane, the target point cloud containing only the surface features of the target part is extracted.

[0011] A multi-view alignment strategy is adopted to fuse target point clouds from multiple cameras. The precise transformation matrix is ​​obtained by coarse matching and fine matching in sequence. All target point clouds are then mapped to the world coordinate system of the reference camera through the precise transformation matrix and superimposed and fused to form a complete global point cloud. This fills the occlusion blind spot of a single view and fully restores the three-dimensional structure of the target part.

[0012] A multi-stage ICP pose matching strategy is adopted to register the fused global point cloud with the preset template point cloud of the target part, solve and output the pose parameters of the target part in the world coordinate system, including the position coordinates in the x, y and z directions, as well as the rotation angles in the roll, pitch and yaw directions, providing accurate data support for robot collaborative assembly.

[0013] Furthermore, the cameras are depth cameras, with no fewer than three, and are evenly distributed around the target part; the scene point cloud data is preprocessed by depth scaling and truncating the scene point cloud image to filter out invalid point clouds that are farther from the camera than a preset value.

[0014] Furthermore, the color images captured by the camera are verified as three-channel RGB format, while the depth images are converted to single-channel uint16 type. If the sizes of the color and depth images do not match, the depth image is adjusted proportionally to the size of the color image to ensure that the RGBD image sizes are consistent.

[0015] Furthermore, the random sampling consensus algorithm uses a planar model to fit the workbench plane where the target part is located, randomly selects multiple non-collinear points to calculate candidate plane equations, iteratively calculates and updates the optimal planar model through the candidate plane equations, removes the optimal planar model from the scene point cloud, and extracts the target point cloud containing only the surface features of the target part.

[0016] Furthermore, during the coarse matching process, voxel downsampling is performed on the target point clouds extracted by each camera to reduce computational complexity while preserving core geometric features. A fast point feature histogram (FPFH) is calculated for the downsampled point cloud, and a statistical histogram is constructed using the angular features of neighboring points to accurately describe local geometric relationships. Using a distance threshold of 1.5 times the voxel size, the Random Sample Consensus Algorithm (RANSAC) is employed to match the point clouds of each camera's coordinate system with the reference camera's world coordinate system. Three random sampling iterations are set, and a dual verification is performed using edge length consistency (0.9 times the edge length) and distance consistency based on the distance threshold. The matching process gradually converges to obtain candidate transformation matrices, iterating 100,000 times until a confidence level of 0.999 is reached. Only transformation matrices with a matching degree (fitness value) greater than 0.3 are retained as initial transformation matrices to ensure the robustness of coarse matching, and these initial transformation matrices serve as the starting point for fine matching.

[0017] Furthermore, during the fine matching process, the initial transformation matrix is ​​used as the initial value, and the search radius is set to twice the size of the voxel to estimate the target point cloud normal to improve the registration accuracy. Then, the point-to-surface transformation estimation method is used to perform iterative nearest point registration, with the distance threshold set to 0.4 times the size of the voxel. The spatial correspondence between the target point cloud of each camera and the target point cloud of the reference camera is iteratively optimized, and the accurate transformation matrix from the coordinate system of each camera to the world coordinate system of the reference camera is output.

[0018] Furthermore, the initial transformation matrix and the precise transformation matrix are stored using a caching mechanism. The cached objects are valid camera pairs that have completed coarse and fine matching and whose matching degree meets the requirements. In subsequent detection, if the point cloud acquisition scene of the camera pair does not change significantly, the cached precise transformation matrix is ​​directly reused. If the point cloud feature matching degree is detected to drop to a preset value or the scene is significantly disturbed, the coarse and fine matching process is re-executed to update the precise transformation matrix.

[0019] Furthermore, the multi-stage ICP pose matching strategy is preferably a three-stage ICP matching strategy, which gradually tightens the threshold to achieve high-precision pose calculation:

[0020] (1) First stage coarse registration: The previous pose is used as the initial value. The initial value is a manually set unit symmetric matrix. The distance threshold is set to 0.8 meters and the number of iterations is 50. This allows for matching of points with larger distances, quickly corrects the initial pose deviation, and avoids the lack of effective matching points due to excessively strict thresholds.

[0021] (2) Precision registration in the second stage: Using the matching result of the first stage as the initial value, the distance threshold is adjusted to 0.07 meters and the number of iterations is 100. Closer point pairs are selected and the pose parameters are further optimized to lay the foundation for precise registration.

[0022] (3) Third stage fine registration: Using the registration result of the second stage as the initial value, set the distance threshold to 0.03 meters and the number of iterations to 150, retain only the closest point pairs, minimize the error of the sum of squared distances of the corresponding point pairs, and finally solve the rigid transformation parameters (rotation matrix R, translation vector T) of the target part in the camera coordinate system.

[0023] An apparatus for a multi-machine collaborative pose detection method includes:

[0024] The data acquisition module includes multiple cameras deployed at different angles around the target part. One camera serves as a reference camera, and the coordinate systems of the other cameras are fixedly related to the world coordinate system of the reference camera. This module is used to acquire scene point cloud data of the target part from different angles.

[0025] The target point cloud extraction module is used to segment the background plane of the workbench where the target part is located by the random sampling consistency algorithm on the scene point cloud data, and extract the target point cloud containing only the surface features of the target part after removing the points in the plane.

[0026] The fusion module is used to fuse target point clouds from multiple cameras using a multi-view alignment strategy. It obtains the accurate transformation matrix by coarse matching and fine matching, and then maps all target point clouds to the world coordinate system of the reference camera through the accurate transformation matrix and superimposes and fuses them to form a complete global point cloud.

[0027] The pose calculation module is used to register the global point cloud with the preset template point cloud of the target part using a multi-stage ICP pose matching strategy, and solve and output the pose parameters of the target part in the world coordinate system.

[0028] An electronic device includes a processor and a memory, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the above-described multi-machine cooperative pose detection method.

[0029] In summary, the present invention has the following advantages:

[0030] This invention overcomes the challenges of single-camera, multi-view collaborative acquisition and point cloud fusion strategies, combining coarse and fine matching with a multi-stage ICP registration algorithm. This successfully addresses the problems of single-camera limited viewpoints, incomplete point cloud information, susceptibility to occlusion interference, and low pose calculation accuracy. Furthermore, by employing the RANSAC algorithm to remove background noise and combining it with a three-stage ICP registration strategy that progressively tightens the distance threshold, the invention effectively avoids the local optima trap often encountered in single-camera detection, significantly improving the accuracy and stability of pose detection.

[0031] Furthermore, the introduction of a multi-machine collaborative mechanism enhances the pose detection system's anti-interference capabilities and environmental adaptability. Combined with verification using a high-precision motion capture system, multi-machine collaborative detection achieved a 45.357% improvement in accuracy in the x-direction, a 7.050% improvement in accuracy in the y-direction, and a 14.502% improvement in accuracy in the yaw direction. Steady optimization was also achieved in the z-direction, roll direction, and pitch direction. This addresses the issues of single-camera limitations due to sensor errors and missing surface textures, improving the system's robustness in complex aerospace assembly scenarios. It provides precise pose data support for the assembly of large components, significantly improving the automation accuracy and operational efficiency of aircraft panel assembly. Attached Figure Description

[0032] Figure 1 This is a flowchart of the single-machine pose detection process.

[0033] Figure 2 Define a fixed coordinate system for FPFH features

[0034] Figure 3 This is a flowchart of multi-machine pose detection.

[0035] Figure 4 The interface for motion capture software.

[0036] Figure 5 This is the interface of the MeshLab software.

[0037] Figure 6 This is the original scene point cloud directly captured by the depth camera.

[0038] Figure 7 The background areas, such as the workbench, are separated after processing by background segmentation algorithms such as RANSAC plane segmentation.

[0039] Figure 8 It is a target point cloud that contains only the surface features of the target part itself.

[0040] Figure 9 The point cloud of the target part template is obtained by modeling the target part in advance using MeshLab software.

[0041] Figure 10 The target point cloud is acquired and processed by camera 1.

[0042] Figure 11 The target point cloud is acquired and processed by camera 2.

[0043] Figure 12 The target point cloud is obtained by camera 3 and processed.

[0044] Figure 13 This is the result of multi-view point cloud fusion.

[0045] Figure 14 This is a block diagram of the device structure for a multi-machine collaborative pose detection method. Detailed Implementation

[0046] Existing single-machine pose detection solutions based on 3D point cloud technology, such as Figure 1 As shown, a depth camera acquires an observation point cloud in the camera coordinate system, while simultaneously loading a 3D model point cloud of the target part. The RANSAC algorithm is used to separate the target part point cloud from the background point cloud of the worktable, while removing environmental noise to reduce its impact on pose detection accuracy. The preprocessed target part point cloud and the loaded model point cloud are then compared using a multi-stage ICP matching algorithm to solve the rigid transformation from the object coordinate system to the camera coordinate system, yielding the pose of the target part in the camera coordinate system. This scheme uses only a single camera to acquire the target part's point cloud, which cannot cover the complete 3D structure of the target part and is easily affected by environmental noise and sensor errors, limiting the accuracy of pose calculation.

[0047] Therefore, based on the single-camera detection scheme, this invention introduces a multi-camera point cloud fusion strategy to collect target point cloud data from different perspectives, supplement the blind spots of a single camera's view, and reduce noise interference through downsampling and outlier removal preprocessing.

[0048] The present invention will now be described in further detail.

[0049] A multi-aircraft cooperative pose detection method for aircraft panel components includes the following steps:

[0050] Multiple cameras deployed at different angles around the target part collect scene point cloud data of the target part. One camera serves as the reference camera, and the coordinate systems of the other cameras are fixedly related to the world coordinate system of the reference camera using the Zhang calibration method.

[0051] Specifically, multi-camera point cloud acquisition includes using a 3D camera to acquire depth information from the scene, obtaining color and depth images. The color image undergoes format verification and conversion to ensure it is a 3-channel RGB format. Simultaneously, the depth image is processed to ensure it is a single-channel image and converted to uint16 type. If the color and depth image sizes do not match, the depth image size is adjusted. The processed color and depth images are then converted to Open3D format images for creating RGBD images. Depth scaling and truncation techniques are used to create the RGBD images: depth scaling parameters convert the stored values ​​of the depth image into physical distances, ensuring the depth values ​​correspond to the real 3D spatial scale and providing an accurate depth reference for point cloud coordinate calculations; depth truncation filters out invalid depth data exceeding a set effective depth range, avoiding point cloud distortion or excessive invalid points due to erroneous depth information in excessively deep areas, while also reducing computational load. Downsampling is used to simplify the point cloud data, and statistical methods are employed to remove outliers, ultimately resulting in an optimized scene point cloud.

[0052] The random sampling consistency algorithm is used to segment the background plane of the workbench where the target part is located on the scene point cloud data. After removing the points in the plane, the target point cloud containing only the surface features of the target part is extracted.

[0053] Specifically, extracting the target point cloud involves using the RANSAC algorithm to remove background regions irrelevant to the target part. The RANSAC algorithm fits a planar model to the background plane of the workbench where the target part is located. In each iteration, five non-collinear points are randomly selected, and the equations of candidate planes are calculated from these selected points. All points are traversed, and the distance from each point to the candidate plane is calculated. If the distance is less than a threshold, the point is considered an inliner of the candidate plane; otherwise, it is considered an outliner. If the number of inliners exceeds the optimal model, the optimal model is updated. The plane fitted by the RANSAC algorithm is removed from the scene point cloud to obtain the target point cloud of the target part in each camera coordinate system.

[0054] A multi-view alignment strategy is adopted to fuse target point clouds from multiple cameras. The precise transformation matrix is ​​obtained by coarse matching and fine matching in sequence. All target point clouds are then mapped to the world coordinate system of the reference camera through the precise transformation matrix and superimposed to form a complete global point cloud.

[0055] A multi-stage ICP pose matching strategy is adopted to register the fused global point cloud with the preset template point cloud of the target part, and solve and output the pose parameters of the target part in the world coordinate system.

[0056] Preferably, the construction method of the preset template point cloud for the target part is as follows: 3D models of the hoop, bulkhead, and corner pieces in the aircraft panel assembly are designed using SolidWorks, and assembly is completed to ensure the correctness of the shape and hole design. Then, these parts are printed using 3D printing technology, and orange metallic powder is sprayed onto the surface to restore a realistic texture. The part design modeling file generated by SolidWorks is imported into MeshLab software, where the coordinate system of the target part is precisely defined, including the calibration of the coordinate origin and the setting of the coordinate axis directions. After the coordinate system definition is completed, the model is exported and saved in PLY point cloud format, allowing the pose detection program to directly call the template point cloud file to perform pose matching calculations. This ensures complete consistency between the target coordinate system and the custom coordinate system in MeshLab in the pose detection results, guaranteeing the consistency of the detection data's benchmark.

[0057] Specifically, multi-view alignment includes two steps: coarse matching and fine matching.

[0058] (1) Coarse matching: Since the optimization effect of the ICP algorithm depends on the initial transformation matrix, coarse matching is performed by feature extraction and RANSAC algorithm to provide the initial pose for the transformation from each camera coordinate system to the reference camera world coordinate system, so as to ensure that the initial deviation is within the convergence range of fine matching. In point cloud processing, Fast Point Feature Histogram (FPFH) is used as the local feature descriptor for the downsampled target point cloud. For neighboring points in the fixed point neighborhood of the point cloud, a fixed coordinate system is defined as follows: Figure 2 As shown. For query point The normal vector, For query point Point to neighboring points Given a vector, calculate the distance between any two points in the neighborhood. and normal and The relative positional deviation represents the positional relationship between points in the neighborhood. This is expressed using angular features within the defined uvw coordinate system. Indicates two normals and The differences are as follows:

[0059] ;

[0060] ;

[0061] .

[0062] According to the query point Calculate the angular features of all points in its neighborhood and record the angular features as a statistical histogram. The same operation is performed on other points within this domain. .calculate The characteristic statistics are as follows:

[0063] .

[0064] The points with the highest FPFH feature similarity are selected as the initial point pairs to form a set. ,in This represents the image point of the target part in the reference camera. This represents the image points of the target part on cameras other than the reference camera. The RANSAC algorithm is used to filter inlier points and calculate candidate transformation matrices. In each iteration, three sets of non-collinear point pairs are randomly selected from S to solve for the transformation matrix. , This represents the rotation matrix from each camera to the reference camera. Let represent the translation vector from each camera to the reference camera, and define the distance threshold as . ,in For a custom downsampling voxel, the number of interior points is counted, and at the end of the iteration, the candidate transformation matrix with the most interior points is selected as the coarse matching result. .

[0065] (2) Fine matching: using the results of coarse matching As the initial transformation matrix, the ICP algorithm is used to calculate a more accurate multi-camera transformation matrix, which integrates multi-angle point clouds into a more complete global point cloud, while effectively removing noise and improving pose detection accuracy.

[0066] The flowchart of the multi-machine collaborative pose detection solution is as follows: Figure 3 As shown, each camera acquires scene point clouds and extracts target part point clouds. Coarse and fine matching algorithms are used to align multiple target point clouds to the reference camera's world coordinate system for superposition and fusion, forming a complete global point cloud. The fused global point cloud is then subjected to multi-stage ICP matching with a preset template point cloud of the target part to obtain the pose of the target part in the camera coordinate system.

[0067] An experimental scheme was designed to compare the detection performance of this invention with existing pose detection schemes. The experimental hardware system was constructed collaboratively using an NVIDIA Orin NX onboard computer, an Orbbec Gemini Pro depth camera, and a high-precision motion capture system. The NVIDIA Orin NX, with its powerful edge computing capabilities, can efficiently process multi-camera data and complex iterative algorithms, providing computational support for real-time pose detection. The Orbbec Gemini Pro depth camera possesses high-precision depth perception capabilities, enabling rapid capture of scene 3D information and target features. The high-precision motion capture system can accurately track the pose of target parts, and, in conjunction with the host computer software, can output high-fidelity pose ground truth data. The relevant hardware selections and software environment are shown in Tables 1 and 2. The motion capture system is a high-precision 3D motion measurement instrument based on computer graphics, sensor technology, and data processing algorithms. It scientifically arranges multiple video capture devices or inertial sensors within the target measurement space to track the key feature points of moving objects in real time, accurately recording and calculating their coordinates, pose, and trajectory data in 3D space. To ensure the accuracy and comprehensiveness of the pose measurement of the target part, this experiment employed a measurement system built with six MARS2H motion capture lenses. The six lenses were evenly distributed and fixedly arranged on supports around the experimental workbench, facing the central measurement area of ​​the workbench. This model of motion capture lens boasts a resolution of 2048*1988 and a latency as low as 2.4ms, effectively capturing minute displacements and attitude changes of the part, ensuring real-time and reliable data acquisition. During the experiment, the motion capture system continuously acquired high-precision pose information of the target part through multi-lens collaborative operation, including key parameters such as 3D coordinates and rotation angles. The acquired raw data was transmitted in real-time to XINGYING dedicated software, where the pose state was visualized in the form of a 3D model, intuitively reflecting the motion state and position information of the target part. Figure 4 As shown.

[0068] Table 1. Relevant Hardware Selection

[0069] Onboard computer NVIDIA Jetson Orin NX 16GB Depth camera Orbbec Gemini Pro motion capture system Measurement of motion capture MARS2H

[0070] Table 2 Relevant Software Environment

[0071] Matlab R2023b XINGYING 4.4.0.6556 Ubuntu operating system 20.04 Python compilation environment 3.8.20 Meshlab 2023.12

[0072] The target part to be tested is fixed in the center of the worktable, and three cameras (camera 1, camera 2, and camera 3) are arranged at different angles around it to collect point clouds of the target part's surface. The fixed relationship between the camera coordinate system and the world coordinate system of the camera base is calibrated using Zhang's calibration method to determine the pose of the object in the world coordinate system. At the same time, the motion capture system calibration coordinate system is set to coincide with the camera base to ensure that the world coordinate system remains consistent in both systems.

[0073] Import the 3D model file of the target part into the MeshLab software environment. Using MeshLab's coordinate system editing function, manually set the origin and positive directions of the X / Y / Z axes according to the object's structural features. This ensures the coordinate system accurately reflects the object's spatial pose and matches the requirements of subsequent pose detection. Figure 5 As shown. After defining the coordinate system, MeshLab's file export function is used to save the point cloud data with custom coordinate system information in PLY format, providing standardized data support for the algorithm to achieve accurate object pose recognition through template matching.

[0074] The effect of single-machine pose detection is as follows Figures 6-9 As shown. Figure 6 It is the raw scene point cloud directly captured by the depth camera, including the target part to be tested, the workbench background and environmental noise, presenting a dense three-dimensional spatial structure; Figure 7 After being processed by background segmentation algorithms such as RANSAC plane segmentation, the background areas such as the workbench are separated, and the remaining part is the relevant point cloud range of the target part to be tested. Figure 8 The target point cloud contains only the surface features of the target part itself. Its geometry is clear and it is the target point cloud for subsequent matching. Figure 9 It is a point cloud template of the target part to be tested, pre-modeled using MeshLab software, that accurately represents the geometric features of the target part design, and... Figure 8 The target point cloud is matched by a multi-stage ICP algorithm, and the pose of the target part in the camera coordinate system can be finally solved.

[0075] Multi-camera, multi-angle image fusion effect, such as Figures 10-13 As shown. Figure 10 It is the target point cloud acquired and processed by camera 1, which presents the local features of the target part under test from this perspective; Figure 11 The point cloud of the target part for camera 2, due to the different viewing angle, shows the geometric information of another area of ​​the target part; Figure 12 It is the target point cloud of camera 3, which further supplements other viewpoint features of the target parts; Figure 13 This is the result of multi-view point cloud fusion, which integrates point cloud information from three cameras to completely and accurately restore the three-dimensional structure of the target part under test.

[0076] The results of single-machine and multi-machine testing are shown in Table 3. The testing error is defined as follows: , , These are single-machine measurement values ​​and multi-machine measurement values, respectively. This represents the true value of the motion capture system. The percentage improvement in accuracy between multi-machine collaborative pose measurement and single-machine pose detection is defined as: The test results show that the multi-machine strategy significantly improves the overall accuracy of pose detection. In the x, y, and yaw directions, the multi-machine errors are 1.224 mm, 0.501 mm, and 0.902 degrees, respectively, which are significantly lower than the single-machine errors of 2.240 mm, 0.539 mm, and 1.055 degrees, with accuracy improvements of 45.357%, 7.050%, and 14.502%, respectively. While the accuracy improvements in the z direction and roll and pitch angles are relatively smaller (0.702%, 2.019%, and 0.793%, respectively), they still demonstrate some optimization. Overall, the experimental results show that the multi-machine collaborative pose detection scheme can effectively improve the pose detection accuracy of aircraft panel assembly parts compared to existing single-machine detection schemes.

[0077] Table 3 Comparison of Single-Machine and Multi-Machine Detection Results

[0078] x(mm) -177.263 -175.023 -178.487 2.240 1.224 45.357% y(mm) 51.787 51.248 51.286 0.539 0.501 7.050% z(mm) 37.490 40.054 40.036 2.564 2.546 0.702% roll(deg) -6.704 -4.227 -4.277 2.477 2.427 2.019% pitch(deg) 36.282 37.543 37.533 1.261 1.251 0.793% yaw(deg) 53.725 54.780 54.627 1.055 0.902 14.502%

[0079] This invention establishes a deep coupling between the data layer, feature layer, and solution layer. Multi-view data complementarity provides a rich geometric feature description foundation for FPFH (Factor-Based Physics), while FPFH coarse matching provides initial values ​​for ICP (Inductively Coupled Phenomena) that are far removed from local optima. The three-stage progressive ICP strategy further mines the data potential based on this. The overall pose detection accuracy produced by this combination far exceeds the sum of the individual improvements of each technical feature, achieving unexpected technical results and effectively solving the accuracy and robustness challenges of pose detection for large parts in aerospace manufacturing.

[0080] Example 2:

[0081] like Figure 14 As shown, this embodiment provides an apparatus for a multi-machine cooperative pose detection method, comprising:

[0082] The data acquisition module 501 includes multiple cameras deployed at different angles around the target part. One camera is selected as the reference camera, and the coordinate systems of the other cameras are fixedly related to the world coordinate system of the reference camera. This module is used to acquire scene point cloud data of the target part from different angles.

[0083] The target point cloud extraction module 502 is used to segment the background plane of the workbench where the target part is located by the random sampling consistency algorithm on the scene point cloud data, and extract the target point cloud containing only the surface features of the target part after removing the points in the plane.

[0084] The fusion module 503 is used to fuse target point clouds from multiple cameras using a multi-view alignment strategy. It obtains an accurate transformation matrix by coarse matching and fine matching in sequence, and then maps all target point clouds to the reference camera's world coordinate system through the accurate transformation matrix and superimposes and fuses them to form a complete global point cloud.

[0085] The pose calculation module 504 is used to register the global point cloud with the preset template point cloud of the target part using a multi-stage ICP pose matching strategy, and solve and output the pose parameters of the target part in the world coordinate system.

[0086] The specific implementation of each module in this embodiment can be found in Embodiment 1 above, and will not be repeated here. It should be noted that the device provided in this embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above.

[0087] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A multi-aircraft cooperative pose detection method for aircraft panel parts, characterized in that: include Multiple cameras are deployed at different angles around the target part to collect scene point cloud data of the target part. One camera is used as the reference camera, and the coordinate systems of the other cameras are fixedly related to the world coordinate system of the reference camera. The random sampling consistency algorithm is used to segment the background plane of the workbench where the target part is located on the scene point cloud data. After removing the points in the plane, the target point cloud containing only the surface features of the target part is extracted. A multi-view alignment strategy is adopted to fuse target point clouds from multiple cameras. The precise transformation matrix is ​​obtained by coarse matching and fine matching in sequence. All target point clouds are then mapped to the world coordinate system of the reference camera through the precise transformation matrix and superimposed to form a complete global point cloud. A multi-stage ICP pose matching strategy is adopted to register the global point cloud with the preset template point cloud of the target part, and solve and output the pose parameters of the target part in the world coordinate system. In the coarse matching process, the voxel size is set for each target point cloud and voxel downsampling is performed. The FPFH feature of the downsampled point cloud is calculated. Based on the FPFH feature, the target point cloud in each camera coordinate system is matched with the target point cloud in the reference camera world coordinate system through the random sampling consensus algorithm. In the matching process, the point-to-point transformation estimation method is adopted, and multiple random sampling times are set. The edge length consistency check and the distance consistency check based on the distance threshold are combined for double verification. The matching is gradually converged to obtain the candidate transformation matrix. Only the candidate transformation matrix that meets the matching degree requirement is retained as the initial transformation matrix, and the initial transformation matrix is ​​used as the starting point for fine matching. During the fine matching process, the initial transformation matrix is ​​used as the initial value. The search radius is set by several times the size of the voxel to estimate the normal of the target point cloud. Then, the point-to-surface transformation estimation method is used to perform iterative nearest point registration. The distance threshold is set by several times the size of the voxel. The spatial correspondence between the target point cloud of each camera and the target point cloud of the reference camera is iteratively optimized, and the accurate transformation matrix from the coordinate system of each camera to the world coordinate system of the reference camera is output. The multi-stage ICP pose matching strategy includes at least a first-stage matching and a second-stage matching. The initial value of the first-stage matching is set to a unit symmetric matrix when it is first executed. The second-stage matching uses the matching result of the first-stage matching as the initial value, and so on. By gradually tightening the distance threshold, only the closest point pairs are retained in the end, minimizing the sum of squared distance errors of the corresponding point pairs. The rigid transformation parameters are solved in the camera coordinate system and converted into pose parameters in the world coordinate system based on Zhang's calibration method.

2. The multi-machine cooperative pose detection method according to claim 1, characterized in that: The scene point cloud data is preprocessed by depth scaling and truncation of the scene point cloud image to filter out invalid point clouds that are farther from the camera than a preset value.

3. The multi-machine cooperative pose detection method according to claim 1, characterized in that: The color images captured by the camera are verified as three-channel RGB format, while the depth images are converted to single-channel uint16 type. If the sizes of the color and depth images do not match, the depth image is adjusted proportionally to the size of the color image to ensure that the RGBD image sizes are consistent.

4. The multi-machine cooperative pose detection method according to claim 1, characterized in that: The random sampling consensus algorithm uses a planar model to fit the workbench plane where the target part is located, randomly selects multiple non-collinear points to calculate candidate plane equations, iteratively calculates and updates the optimal planar model through the candidate plane equations, removes the optimal planar model from the scene point cloud, and extracts the target point cloud containing only the surface features of the target part.

5. The multi-machine cooperative pose detection method according to claim 4, characterized in that: The initial transformation matrix and the precise transformation matrix are stored using a caching mechanism. The cached objects are valid camera pairs that have completed coarse and fine matching and whose matching degree meets the requirements. If the point cloud acquisition scene of the camera pair does not change significantly during subsequent detection, the cached precise transformation matrix is ​​directly reused. If the point cloud feature matching degree is detected to drop to a preset value or the scene is significantly disturbed, the coarse and fine matching process is re-executed to update the precise transformation matrix.

6. The apparatus for the multi-machine cooperative pose detection method according to any one of claims 1-5, characterized in that, include: The data acquisition module includes multiple cameras deployed at different angles around the target part. One camera is selected as the reference camera, and the coordinate systems of the other cameras are fixedly related to the world coordinate system of the reference camera. This module is used to acquire scene point cloud data of the target part from different angles. The target point cloud extraction module is used to segment the background plane of the workbench where the target part is located by the random sampling consistency algorithm on the scene point cloud data, and extract the target point cloud containing only the surface features of the target part after removing the points in the plane. The fusion module is used to fuse target point clouds from multiple cameras using a multi-view alignment strategy. It obtains the accurate transformation matrix by coarse matching and fine matching, and then maps all target point clouds to the world coordinate system of the reference camera through the accurate transformation matrix and superimposes and fuses them to form a complete global point cloud. The pose calculation module is used to register the global point cloud with the preset template point cloud of the target part using a multi-stage ICP pose matching strategy, and solve and output the pose parameters of the target part in the world coordinate system.

7. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the multi-machine cooperative pose detection method as described in any one of claims 1-5.

Images