An aviation component CAD model and point cloud cross-modal registration method based on visual key feature constraint
By extracting geometric features of point clouds of aerospace components using Hilbert curves and Mamba state-space models, and combining key visual features of multi-view RGB images with a generative point flow field matching model, stable and efficient cross-modal registration of CAD models of aerospace components with point clouds was achieved. This solved the problem of unstable pose calculation under occlusion, reflection, and symmetrical structures, and provided high-precision and real-time deviation measurement.
Patent Information
- Application Number
- CN202611126555.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-28
- Publication Date
- 2026-11-06
AI Technical Summary
In the context of aerospace assembly, existing technologies struggle to achieve stable and efficient pose registration between CAD models and point clouds of aerospace components under conditions of occlusion, reflection, or partial defects. This is especially true for the pose multi-solution and attitude flipping problems of symmetrical structures, which also incur significant computational overhead and cannot meet real-time requirements.
We use Hilbert curves and the Mamba state-space model to extract geometric features from point clouds. We combine the key visual features of multi-view RGB images and the confidence of two-dimensional images, and use a generative point flow field matching model for weighted constraints to achieve cross-modal registration.
It improves the accuracy and efficiency of large-scale point cloud registration, reduces computational overhead, solves the problem of unstable pose calculation under occlusion, reflection and symmetrical structures, provides high robustness and real-time performance, and outputs highly reliable deviation measurement results.
Smart Images
Figure CN123052008A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of aerospace manufacturing and computer vision measurement technology, and in particular relates to a cross-modal registration method for CAD models of aerospace components and point clouds based on visual key feature constraints. Background Technology
[0002] Large aerospace components, such as wing skins, fuselage panels, cabin structural components, reinforcing frames, and connecting brackets, are typically characterized by their large size, complex curved surfaces, and high assembly precision requirements. In the digital assembly process for aerospace components, it is necessary to spatially register the measured point cloud obtained from on-site 3D scanning with the theoretical CAD model to determine the actual position and orientation of the aerospace component relative to the design model. The accuracy of the registration results directly affects the reliability of subsequent assembly adjustments, deviation analysis, and quality inspection.
[0003] Existing methods for registering CAD models and point clouds of aerospace components mainly include registration methods based on manual targets, point cloud registration methods based on geometric features, and cross-modal registration methods based on image or point cloud features. Among these, registration methods based on manual targets typically require setting up targets and manually calibrating them on-site, resulting in complex procedures, low automation, and difficulty in adapting to the flexible and rapid demands of aerospace assembly. Point cloud registration methods based on geometric features usually rely on geometric information such as edges, holes, surface normals, or local descriptors in the point cloud. When aerospace components have occlusions, reflections, local missing parts, low texture, or repetitive structures, problems such as unstable candidate matching relationships, increased mismatches, and strong dependence on initial pose can easily arise.
[0004] Furthermore, aerospace assembly sites typically contain heterogeneous data from multiple sources, including multi-view RGB images, 3D scanned point clouds, and theoretical CAD models. RGB images can reflect key visual features such as hole positions, edges, and seams; 3D point clouds can reflect the actual spatial geometry of components; and theoretical CAD models describe the design geometric datum of components. Due to differences in data structure, feature representation, and spatial coordinate system among image data, point cloud data, and CAD models, directly establishing a stable correspondence among them is difficult. If highly reliable key visual features in the images are not effectively utilized to constrain the candidate matching relationship between the point cloud and the CAD model, registration results can easily shift under occlusion, reflection, or partial defects.
[0005] Meanwhile, the 3D scanned point clouds of aerospace components are usually large in scale. Traditional registration methods based on global feature matching or complex attention mechanisms have high computational costs when processing large-scale point clouds, making it difficult to meet the real-time and stability requirements of the assembly site. For aerospace components with symmetrical or approximately repetitive structures, registration based solely on geometric neighborhood features may also lead to problems such as multiple pose solutions, local optima, or attitude reversal, affecting the accuracy of the final pose solution.
[0006] Therefore, how to fully utilize the key visual features in multi-view RGB images in aviation assembly scenarios, reliably constrain the cross-modal candidate matching relationship between CAD models and measured point clouds, and achieve stable and efficient pose registration under conditions of large-scale point clouds, occlusion and reflection, local defects and symmetrical structures, is a technical problem that urgently needs to be solved in the digital assembly of aviation components. Summary of the Invention
[0007] To address the problems existing in the prior art, this invention proposes a cross-modal registration method for CAD models and point clouds of aerospace components based on visual key feature constraints. This method utilizes core technologies such as efficient feature extraction from Hilbert curves and Mamba state-space models, cross-modal weighted registration guided by 2D image confidence, and generative point flow field attitude fine regression. It aims to improve the registration accuracy and efficiency of incomplete, symmetrical, and massive point cloud data in large-scale aerospace scenarios, resulting in deviation measurements with high robustness, good real-time performance, and low uncertainty. Therefore, it plays a crucial role in the digital precision assembly of large aerospace components.
[0008] To achieve the above-mentioned technical objectives, the present invention provides the following technical solution:
[0009] A cross-modal registration method for CAD models and point clouds of aerospace components based on visual key feature constraints, specifically including:
[0010] S1. Acquire the theoretical CAD model, multi-view RGB images, and original measured point cloud of the aerospace component to be tested; perform surface discretization sampling on the theoretical CAD model to generate a CAD reference point set; and establish the point map association relationship between the pixels of the multi-view RGB image and the original measured point cloud based on the calibration parameters.
[0011] S2. The original measured point cloud is rearranged using Hilbert curves, and the geometric features of the measured point cloud of the aircraft component to be tested are extracted from the rearranged original measured point cloud using the Mamba state space model.
[0012] S3. Identify the key visual features of aerospace components from multi-view RGB images and calculate the two-dimensional image confidence of each key visual feature; summarize the two-dimensional image confidence of each view to obtain a multi-view two-dimensional image confidence mapping table.
[0013] S4. Based on the multi-view 2D image confidence mapping table, and combining the geometric features of the measured point cloud with the geometric neighborhood features of the CAD reference point set, establish candidate matching relationships between the CAD reference point set and the measured point cloud; calculate the registration weight based on the geometric feature similarity and 2D image confidence of the candidate matching point pairs, and use the registration weight to constrain the confidence of the candidate matching point pairs, eliminating candidate matching point pairs with low registration confidence; perform weighted pose solving based on the retained candidate matching point pairs and their registration weights to determine the initial pose state.
[0014] S5. Based on the initial pose state, construct a generative point flow field matching model that continuously evolves the measured point cloud to the standard pose of the theoretical point cloud. In the generative point flow field matching model, the overlapping sensing geometric features of the key contact area of the aerospace component, the confidence level of the two-dimensional image, and the registration weight of the candidate matching point pairs are introduced to impose weighted constraints on the residual displacement direction of the point flow field and the rigid pose increment solution process, so that the measured point cloud gradually evolves from the initial pose state to the target pose state along the continuous spatial vector field, and the final pose parameters are obtained through continuous pose optimization. The overlapping sensing geometric features include the distance between points, normal consistency, and local neighborhood overlap. The key contact area of the aerospace component is the area that participates in assembly contact or positioning.
[0015] S6. Based on the final pose parameters, unify the measured point cloud and the theoretical CAD model into the same coordinate system, calculate the spatial pose deviation parameters of the measured point cloud relative to the theoretical CAD model, and output the cross-modal registration results.
[0016] Furthermore, step S1 specifically includes:
[0017] 3D scanning of aerospace components using a 3D scanner to obtain the original measured point cloud. ;
[0018] Multi-view RGB images of aerospace components were acquired by deploying industrial cameras at different angles. Simultaneously retrieve the theoretical CAD digital model of the aerospace components. Based on theoretical CAD models Discretize the surface geometry, assembly boundaries, assembly holes, or assembly contact areas to generate a set of CAD reference points for registration. ;
[0019] Based on the intrinsic and extrinsic parameters of the industrial camera and the calibration parameters between the 3D scanner and the industrial camera, the original measured point cloud was... The three-dimensional points in the image are projected onto the RGB image plane of the corresponding viewpoint to obtain the projection position of each three-dimensional point in the image; based on the projection positions, a multi-view RGB image pixel and the original measured point cloud are established. The point graph relationships between three-dimensional points.
[0020] Furthermore, step S2 specifically includes:
[0021] Voxelization is performed on the original measured point cloud: each original measured point is mapped to voxel coordinates according to the preset voxel resolution;
[0022] Based on the Hilbert curve mapping function, voxel coordinates are mapped to Hilbert indices;
[0023] The original measured points are sorted in ascending order according to Hilbert indices to form a point cloud sequence that maintains the continuity of spatial neighborhood;
[0024] A point cloud geometric feature encoding model based on the Mamba state space architecture is constructed. The point cloud geometric feature encoding model includes an input embedding layer, a selective state space recursive unit, and an output mapping layer. The embedding layer encodes each point in the point cloud sequence into a point feature vector. The selective state space recursively updates the hidden state along the point cloud sequence and obtains the output feature based on the hidden state. The output mapping layer maps the output feature to the measured point cloud geometric feature. The measured point cloud geometric feature is a point-by-point feature vector that corresponds one-to-one with each point in the measured point cloud, including the geometric information of each point and its local neighborhood.
[0025] Furthermore, step S3 specifically includes:
[0026] A visual key feature recognition model for aviation assembly is constructed, comprising an image feature encoding layer, a feature decoding layer, and a key feature prediction layer; RGB images from the site are used for this purpose. Input the visual key feature recognition model to identify the visual key features in the aerospace components, and output the pixel-level feature map and semantic response score corresponding to each viewpoint of the visual key features.
[0027] For the l-th visual key feature pixel in the pixel-level feature map under the v-th viewpoint Based on the point map correlation between pixels in multi-view RGB images and the original measured point cloud, the relationship between the two is determined. Corresponding original measured points ,according to Projection position in the RGB image plane at the v-th viewpoint and Calculate the pixel distance between them. and Projection consistency score ;
[0028] Based on key visual feature pixels semantic response score Projection consistency score ,calculate Confidence of two-dimensional images;
[0029] The confidence scores of each visual key feature pixel from each viewpoint are summarized in two-dimensional images, and then written into the corresponding two-dimensional image confidence score mapping table according to the pixel position, finally obtaining the multi-view two-dimensional image confidence score mapping table of visual key features. .
[0030] Furthermore, step S4 specifically includes:
[0031] For any original measured point Obtain the geometric features of its measured point cloud. ;
[0032] Extract each CAD reference point from the CAD reference point set Q. Geometric neighborhood features ;calculate and Geometric feature similarity between ;
[0033] CAD reference points with feature similarity greater than a preset threshold or ranking in the top K of feature similarity are selected as candidate reference points, and a set of candidate matching point pairs is obtained. ;
[0034] Based on the original measured points Projection position in the RGB image plane at the corresponding viewpoint From the confidence mapping table of multi-view two-dimensional images The extracted 2D image confidence score, combined with candidate matching point pairs Calculate the registration weights of candidate matching point pairs based on the geometric feature similarity between them. The formula is expressed as:
[0035] ;
[0036] in, A mapping function representing a confidence mapping table for multi-view two-dimensional images;
[0037] Based on candidate matching point pairs and registration weights, a weighted pose calculation objective function is constructed; the objective function is then solved using weighted singular value decomposition to obtain the coarse registration rotation matrix. and coarse registration translation vector ;
[0038] based on , Calculate the registration residual between each pair of candidate matching points;
[0039] Candidate matching point pairs with registration weights lower than a preset weight threshold and registration residuals greater than a preset residual threshold are considered low-confidence mismatches and are discarded. Based on the retained candidate matching point pairs and their registration weights, the weighted pose solving objective function is reconstructed, and the initial rotation matrix is obtained. and initial translation vector This serves as the initial pose state.
[0040] Furthermore, step S5 specifically includes:
[0041] Original measured point cloud Perform pose transformation according to the initial pose state to obtain the initial aligned point cloud. ;
[0042] against Each initial alignment point in In the CAD reference point set Based on overlapping sensing geometric features, it provides Search for the corresponding nearby reference points The overlapping sensing geometric weights are calculated based on the distance between points, normal consistency, and the degree of local neighborhood overlap. The formula is expressed as:
[0043] ;
[0044] in, for and Overlap score between local neighborhoods for The normal vector, For nearby reference points The normal vector; For distance scale parameters; for Overlapping perceived geometric weights;
[0045] A continuously evolving generative point flow field matching model is constructed, which includes a point cloud feature encoding module, a cross-modal feature fusion module, a point flow field prediction module, and a rigid pose update module.
[0046] The initial aligned point cloud, theoretical point cloud, measured point cloud geometric features, 2D image confidence scores of each point, and registration weights of retained candidate matching point pairs are input into the generative point flow field matching model; the initial aligned point cloud is first extracted through the point cloud feature encoding module. With CAD reference point set Geometric neighborhood features; then, in the cross-modal feature fusion module, the initial aligned point cloud is processed. Geometric neighborhood features, CAD reference point set The geometric neighborhood features, measured point cloud geometric features, two-dimensional image confidence of each original measured point, and registration weights of candidate matching point pairs are concatenated to obtain fused features;
[0047] Subsequently, in the point flow field prediction module, the orientation of each initial alignment point toward the CAD reference point set is predicted based on the fusion features. The residual displacement direction at the corresponding position; the residual displacement direction is weighted and constrained using overlapping sensing geometric weights to obtain the constrained residual displacement vector of each initial alignment point. ;
[0048] Finally, in the rigid pose update module, based on the constrained residual displacement vector... A weighted optimization objective function is constructed to solve for the global rigid pose increment; the rotation increment is solved by weighted least squares or weighted singular value decomposition. Translational increment The current pose parameters are iteratively updated; when the pose change between two adjacent iterations is less than a preset threshold, or when the preset number of iterations is reached, the optimization stops, and the final rotation matrix is obtained. and the final translation vector , as the final pose parameter.
[0049] Furthermore, step S6 specifically includes:
[0050] Based on the final pose parameters, the original measured point cloud... A rigid transformation is performed to obtain the final registration point cloud unified to the theoretical CAD model design coordinate system. ;
[0051] For each final registration point Search for the corresponding reference point in the CAD reference point set Q or the surface of the theoretical CAD model. ,calculate and Inter-registration residuals ; and then according to Local normal vector at point ,calculate Deviation of normal surface relative to the theoretical CAD model surface ;
[0052] The final registration error evaluation results of all points are statistically analyzed, and the cross-modal registration results are output, including the final pose parameters, the final registered point cloud, the registration residuals of each point, and the distribution of normal surface deviation.
[0053] Furthermore, this application also discloses a cross-modal registration system for CAD models and point clouds of aerospace components based on visual key feature constraints. This system is used to perform the method described above, specifically including:
[0054] The data acquisition and preprocessing module is used to acquire the theoretical CAD model, multi-view RGB images, and original measured point cloud of the aerospace component to be tested; to perform surface discretization sampling on the theoretical CAD model to generate a CAD reference point set; and to establish the point map association relationship between the pixels of the multi-view RGB image and the original measured point cloud based on the calibration parameters.
[0055] The Hilbert-Mamba encoding module is used to rearrange the original measured point cloud using Hilbert curves and extract the geometric features of the measured point cloud of the aircraft component under test by combining the Mamba state space model.
[0056] The visual key feature recognition module is used to identify key visual features of aerospace assembly from multi-view RGB images, calculate the corresponding two-dimensional image confidence scores, and generate a multi-view two-dimensional image confidence score mapping table.
[0057] The weighted initial registration module is used to establish candidate correspondences based on the confidence mapping table of two-dimensional images, combined with the geometric features of the measured point cloud and the geometric neighborhood features of the CAD reference point set. It calculates the registration weights based on the geometric feature similarity of the candidate matching point pairs and the confidence of the two-dimensional images, uses the registration weights to eliminate low-confidence candidate matching point pairs, and uses the retained candidate matching point pairs and registration weights to solve the weighted pose and obtain the initial pose state.
[0058] The point flow field fine registration module is used to construct a generative point flow field matching model that continuously evolves from the measured point cloud to the standard pose of the theoretical point cloud based on the initial pose state. It introduces the overlapping sensing geometric features of the key contact area of the aerospace component, the confidence of the two-dimensional image, and the registration weight of the candidate matching point pairs. It applies weighted constraints to the residual displacement direction of the point flow field and the rigid pose increment solution process, and obtains the final pose parameters through continuous pose optimization.
[0059] The registration result output module is used to unify the measured point cloud and the theoretical CAD model into the same coordinate system based on the final pose parameters, calculate the registration residual and normal surface deviation, and output the cross-modal registration result.
[0060] An electronic device is also disclosed, comprising a memory and a processor, wherein:
[0061] Memory is used to store computer programs that can run on a processor;
[0062] The processor is configured to, while running the computer program, execute a cross-modal registration method for CAD models of aerospace components and point clouds based on visual key feature constraints, as described above.
[0063] A computer-readable storage medium is also disclosed, storing computer instructions for instructing a processor to implement, when executed, a cross-modal registration method for aerospace component CAD models and point clouds based on visual key feature constraints, as described above. Based on the above technical solution, the present invention has at least the following beneficial effects:
[0064] (1) Significantly improves the robustness and accuracy of registration in complex environments. This invention establishes a two-dimensional image confidence mapping table to implement dynamic weighted constraints on matching point pairs in the objective function of pose calculation. Compared with traditional methods that suffer from rotation errors during cross-modal registration, this invention effectively suppresses noise interference caused by strong reflections and incomplete areas in aerospace assembly sites, significantly improves the stability of pose calculation in heavily occluded environments, and solves the problem that traditional methods are prone to getting trapped in local optima.
[0065] (2) Significantly reduced computational overhead for processing massive amounts of data. This invention innovatively introduces a Hilbert curve space rearrangement mechanism and combines it with the Mamba state space model, which has linear complexity characteristics. This enables the system to reduce computational complexity from the traditional Transformer architecture to linear level when processing large-scale point clouds with tens of millions of sampling points at the aerospace level. It can respond in real time to the dynamic measurement needs of large component assembly and overcome the shortcomings of existing technologies in running slowly in large-scale data scenarios.
[0066] (3) Effectively eliminates ambiguities in pose calculation for symmetrical components. For symmetrical structures such as pipes and supports widely found in aerospace components, this invention utilizes a generative point flow field matching model to drive the point cloud to "flow" towards the target pose by learning a continuous spatial velocity field. This mechanism does not rely on any manually preset symmetry labels and can implicitly learn the geometric prior knowledge of the component, completely solving the 180° ambiguity that often occurs in the alignment of symmetrical components in traditional feature matching schemes. ° A complex problem involving flipping and orientation.
[0067] (4) Provides highly reliable and engineering-oriented deviation measurement reports. This invention not only achieves physical alignment of cross-modal data, but also combines measurement task weights to output cross-modal registration evaluation results including final pose parameters, registration residuals, normal surface deviations, and deviation distributions. By synchronously outputting system error compensation and uncertainty indices, the measurement results show extremely high agreement with manual precision measurements, providing adjustment suggestions with decision support value for aerospace digital flexible assembly. Attached Figure Description
[0068] Figure 1 This is the overall flowchart of a cross-modal registration method for CAD models of aerospace components and point clouds based on visual key feature constraints proposed in this invention.
[0069] Figure 2 This is a schematic diagram of the overall model structure and specific process of the method proposed in this invention in the embodiments. Detailed Implementation
[0070] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0071] Although the steps in this invention are arranged by reference numerals, this is not intended to limit the order of the steps. Unless the order of the steps is explicitly stated or the execution of a step requires other steps as a basis, the relative order of the steps can be adjusted. It is understood that the term "and / or" as used herein refers to and covers any and all possible combinations of one or more of the associated listed items.
[0072] like Figure 1 As shown, this invention proposes a cross-modal registration method for CAD models and point clouds of aerospace components based on visual key feature constraints, which specifically includes:
[0073] S1. Acquire the theoretical CAD model, multi-view RGB images, and original measured point cloud of the aerospace component to be tested; perform surface discretization sampling on the theoretical CAD model to generate a CAD reference point set; and establish the point map association relationship between the pixels of the multi-view RGB image and the original measured point cloud based on the calibration parameters.
[0074] As a preferred embodiment, in this embodiment, such as Figure 2 As shown, step S1 specifically includes:
[0075] 3D scanning of aerospace components (such as wing skin and fuselage panels) is performed using a 3D scanner to obtain the original measured point cloud. , For point cloud scale; each original measured point Includes spatial coordinate information, This is the index of the original measured points. Represents a three-dimensional real vector space;
[0076] Multi-view RGB images of aerospace components were acquired by deploying industrial cameras at different angles. , The total number of viewpoints; and simultaneously retrieve the theoretical CAD digital models of the aircraft components. Based on theoretical CAD models Discretize the surface geometry, assembly boundaries, assembly holes, or assembly contact areas to generate a set of CAD reference points for registration. , The total number of CAD reference points, and the set of CAD reference points Maintain the coordinate system specified in the CAD design.
[0077] Based on the intrinsic and extrinsic parameters of the industrial camera and the calibration parameters between the 3D scanner and the industrial camera, the original measured point cloud was... The three-dimensional points in the image are projected onto the RGB image plane of the corresponding viewpoint to obtain the projection position of each three-dimensional point in the image; based on the projection positions, a multi-view RGB image pixel and the original measured point cloud are established. The point map association between the three-dimensional points in the image enables key visual features such as holes, edges, and seams to be linked to the original measured three-dimensional point cloud. The corresponding three-dimensional point in the diagram.
[0078] S2. The original measured point cloud is rearranged using Hilbert curves, and the geometric features of the measured point cloud of the aircraft component to be tested are extracted from the rearranged original measured point cloud using the Mamba state space model.
[0079] As a preferred embodiment, in this embodiment, such as Figure 2 As shown, step S2 specifically includes:
[0080] Voxelization of the original measured point cloud: Each original measured point is processed according to a preset voxel resolution. Mapping to voxel coordinates The formula is expressed as:
[0081] ;
[0082] in, This represents the minimum coordinate boundary of the original measured 3D point cloud P in 3D space. For voxel resolution, This indicates a round-down operation;
[0083] Based on Hilbert curve mapping function Mapping voxel coordinates to Hilbert indices ;
[0084] The original measured points were arranged according to Hilbert's sequence number. Sort the points in ascending order to form a point cloud sequence that maintains the continuity of spatial neighborhood. .
[0085] A point cloud geometric feature encoding model based on the Mamba state-space architecture is constructed. This model includes an input embedding layer, a selective state-space recursive unit, and an output mapping layer, wherein:
[0086] The embedding layer encodes each point in the point cloud sequence into a point feature vector, as expressed by the formula:
[0087] ;
[0088] in, for The first in One point, Indicates the first Local geometric properties of a point This represents the input feature embedding function; for eigenvectors of points;
[0089] The selective state-space recursive unit recursively updates the hidden state along the point cloud sequence, and obtains the output feature based on the hidden state, as expressed by the formula:
[0090] ;
[0091] ;
[0092] in, for The hidden state, for The hidden state, for The output characteristics; , , It is a learnable parameter matrix;
[0093] The output mapping layer maps the output features to measured point cloud geometric features. , ,in for Geometric eigenvectors, express 3D real vector space, The feature dimension is defined as follows: the geometric features of the measured point cloud are point-by-point feature vectors that correspond one-to-one with each point in the measured point cloud, including the geometric information of each point and its local neighborhood. These features serve as the geometric feature inputs for establishing candidate matching relationships between the subsequent CAD reference point set Q and the original measured 3D point cloud P, solving weighted poses, and optimizing the continuous point flow field.
[0094] S3. Identify the key visual features of aerospace components from multi-view RGB images and calculate the two-dimensional image confidence of each key visual feature; summarize the two-dimensional image confidence of each view to obtain a multi-view two-dimensional image confidence mapping table.
[0095] As a preferred embodiment, in this embodiment, such as Figure 2 As shown, step S3 specifically includes:
[0096] A visual key feature recognition model for aviation assembly is constructed, comprising an image feature encoding layer, a feature decoding layer, and a key feature prediction layer; RGB images from the site are used for this purpose. Input the visual key feature recognition model to identify at least one visual key feature among hole positions, component edges, and joint seams in aerospace components, and output pixel-level feature maps and semantic response scores for each viewpoint corresponding to the visual key feature.
[0097] For the l-th visual key feature pixel in the pixel-level feature map under the v-th viewpoint Based on the point map correlation between pixels in multi-view RGB images and the original measured point cloud, the relationship between the two is determined. Corresponding original measured points ,according to Projection position in the RGB image plane at the v-th viewpoint and Calculate the pixel distance between them. and Projection consistency score The formula is expressed as:
[0098] ;
[0099] in, Indicates the first The projection function of a 3D point onto a 2D image plane from a given viewpoint. This is the projection error scale parameter.
[0100] It should be noted that in this application, the RGB image and the pixel-level feature map use the same two-dimensional pixel coordinate system, and the position coordinates of the visual key feature pixels in the pixel-level feature map correspond one-to-one with their position coordinates in the original RGB image.
[0101] Based on key visual feature pixels semantic response score Projection consistency score ,calculate Confidence of two-dimensional images The formula is expressed as:
[0102] ;
[0103] Where α is the weight balance coefficient. ;
[0104] The confidence scores of each visual key feature pixel from each viewpoint are summarized in two-dimensional images, and then written into the corresponding two-dimensional image confidence score mapping table according to the pixel position, finally obtaining the multi-view two-dimensional image confidence score mapping table of visual key features. A two-dimensional image confidence mapping table can be understood as a table that records the corresponding confidence values using the image pixel positions as indexes.
[0105] The two-dimensional image confidence mapping table is used to characterize the confidence level of key visual regions such as hole positions, component edges, and joint seams in images from various perspectives, and serves as the weight basis for subsequent weighted pose solving of candidate matching relationships between theoretical CAD models and measured point clouds.
[0106] S4. Based on the multi-view 2D image confidence mapping table, and combining the geometric features of the measured point cloud with the geometric neighborhood features of the CAD reference point set, establish candidate matching relationships between the CAD reference point set and the measured point cloud; calculate the registration weight based on the geometric feature similarity and 2D image confidence of the candidate matching point pairs, and use the registration weight to constrain the confidence of the candidate matching point pairs, eliminating candidate matching point pairs with low registration confidence; perform weighted pose solving based on the retained candidate matching point pairs and their registration weights to determine the initial pose state.
[0107] As a preferred embodiment, in this embodiment, such as Figure 2 As shown, step S4 specifically includes:
[0108] For any original measured point Obtain the geometric features of its measured point cloud. ;
[0109] Extract each CAD reference point from the CAD reference point set Q. Geometric neighborhood features ;calculate and Geometric feature similarity between The formula is expressed as:
[0110] ;
[0111] in, For feature similarity scale parameters, For the index of the original measured points, Index for CAD reference points;
[0112] CAD reference points with feature similarity greater than a preset threshold or ranking in the top K of feature similarity are selected as candidate reference points, and a set of candidate matching point pairs between the entire measured point cloud and the entire CAD reference point set is obtained. .
[0113] Based on the camera's intrinsic and extrinsic parameters and the 3D scanner's calibration parameters, the original measured points were obtained. Projection position in the RGB image plane at the corresponding viewpoint From the confidence mapping table of multi-view two-dimensional images The extracted 2D image confidence score, combined with candidate matching point pairs Calculate the registration weights of candidate matching point pairs based on the geometric feature similarity between them. The formula is expressed as:
[0114] ;
[0115] in, A mapping function representing a confidence mapping table for multi-view two-dimensional images;
[0116] Based on candidate matching point pairs and registration weights, a weighted pose calculation objective function is constructed, expressed by the following formula:
[0117] ;
[0118] in, Let be a rotation matrix. It is a translation vector; Represents a special orthogonal group in three dimensions. Represent a three-dimensional real vector space; solve the objective function through weighted singular value decomposition to obtain the coarse registration rotation matrix. and coarse registration translation vector ;
[0119] based on , Calculate the registration residual between each pair of candidate matching points;
[0120] Candidate matching point pairs with registration weights below a preset weight threshold and registration residuals above a preset residual threshold are considered low-confidence mismatches and are discarded to exclude extremely obvious mismatches, rather than excluding all possible valid matches. Based on the retained candidate corresponding point pairs and their registration weights, the weighted pose solving objective function is reconstructed to obtain the initial rotation matrix. and initial translation vector This serves as the initial pose state.
[0121] S5. Based on the initial pose state, construct a generative point flow field matching model that continuously evolves the measured point cloud to the standard pose of the theoretical point cloud. In the generative point flow field matching model, the overlapping sensing geometric features of the key contact area of the aerospace component, the confidence level of the two-dimensional image, and the registration weight of the candidate matching point pairs are introduced to impose weighted constraints on the residual displacement direction of the point flow field and the rigid pose increment solution process, so that the measured point cloud gradually evolves from the initial pose state to the target pose state along the continuous spatial vector field, and the final pose parameters are obtained through continuous pose optimization. The overlapping sensing geometric features include the distance between points, normal consistency, and local neighborhood overlap. The key contact area of the aerospace component is the area that participates in assembly contact or positioning.
[0122] As a preferred embodiment, in this embodiment, such as Figure 2 As shown, step S5 specifically includes:
[0123] Original measured point cloud Perform pose transformation according to the initial pose state to obtain the initial aligned point cloud. ;
[0124] against Each initial alignment point in In the CAD reference point set Based on overlapping sensing geometric features, it provides Search for the corresponding nearby reference points This is used for subsequent residual displacement prediction and fine registration optimization, and the overlap-aware geometric weights are calculated based on the distance between points, normal consistency, and the degree of local neighborhood overlap. The formula is expressed as:
[0125] ;
[0126] in, for and The overlap score between the local neighborhoods (i.e., the degree of overlap between local neighborhoods). for The normal vector, for The local normal vector; For distance scale parameters; for Overlapping perceived geometric weights;
[0127] A continuously evolving generative point flow field matching model is constructed, which includes a point cloud feature encoding module, a cross-modal feature fusion module, a point flow field prediction module, and a rigid pose update module.
[0128] In this application, the modules within the generative point flow field matching model are specifically as follows:
[0129] The point cloud feature encoding module includes a neighborhood construction unit, a geometric mapping unit, and a neighborhood feature stitching unit. The neighborhood construction unit is used to construct the local neighborhood of the initial aligned point cloud and the CAD reference point set. The geometric mapping unit and the neighborhood feature stitching unit are used to extract the geometric neighborhood features of the two.
[0130] The cross-modal feature fusion module includes a feature dimension alignment unit, a feature stitching unit, and a fusion feature mapping unit, which are used to perform dimension alignment, stitching, and feature mapping on the geometric neighborhood features of the initial aligned point cloud, the geometric neighborhood features of the CAD reference point set, the geometric features of the measured point cloud, the two-dimensional image confidence of each original measured point, and the registration weights of the candidate matching point pairs to obtain fused features.
[0131] The point flow field prediction module includes a residual feature mapping unit and a three-dimensional displacement output unit. The residual feature mapping unit is used to predict the residual displacement vector of each initial alignment point based on the fused features. The three-dimensional displacement output unit then utilizes the aforementioned overlapping sensing geometric weights. The residual displacement vector is weighted and constrained to obtain the constrained residual displacement vector;
[0132] The rigid pose update module includes a target position prediction unit, a weighted target optimization unit, a rigid pose increment unit, a pose iteration unit, and a convergence judgment unit. It is used to construct the target position prediction based on the initial alignment point and its constrained residual displacement vector, solve the rotation increment and translation increment through weighted least squares or weighted singular value decomposition, and iteratively update the current pose parameters until the preset convergence condition is met.
[0133] The specific process of obtaining the final pose parameters using a generative point flow field matching model is as follows:
[0134] The initial aligned point cloud, theoretical point cloud, measured point cloud geometric features, 2D image confidence scores of each point, and registration weights of candidate matching point pairs are input into the generative point flow field matching model; the initial aligned point cloud is first extracted through the point cloud feature encoding module. With CAD reference point set Geometric neighborhood features; then, in the cross-modal feature fusion module, the initial aligned point cloud is processed. Geometric neighborhood features, CAD reference point set The geometric neighborhood features, measured point cloud geometric features, two-dimensional image confidence of each original measured point, and registration weights of candidate matching point pairs are concatenated to obtain fused features;
[0135] Subsequently, in the point flow field prediction module, each initial alignment point is predicted based on the fusion features. Relative to its nearest CAD reference point Residual displacement vector The formula is expressed as:
[0136] ;
[0137] in, Indicates the initial alignment point Corresponding fusion features, This represents the mapping function of the point flow field prediction module. Represents the residual displacement vector without overlapping perceived geometric weight constraints;
[0138] Using the aforementioned overlapping sensing geometric weights For the residual displacement vector Apply weighted constraints to obtain the initial alignment point. constrained residual displacement vector The formula is expressed as:
[0139] ;
[0140] The constrained residual displacement vector Used to characterize the initial alignment point The displacement trend towards the corresponding position of the CAD reference point set is used to guide the solution of the overall rigid pose increment.
[0141] Finally, the rigid pose update module, based on the initial alignment point... and its constrained residual displacement vector The corresponding predicted target location is determined by the formula:
[0142]
[0143] Based on the initial alignment point, the predicted target position, and their corresponding weights, a weighted optimization objective function is constructed to solve for the overall rigid pose increment:
[0144] ;
[0145] in, The comprehensive weights are determined based on the overlap-aware geometric weights, the 2D image confidence, and the registration weights that retain candidate matching point pairs. and These represent the rotation increment and translation increment, respectively.
[0146] The rotation and translation increments are solved using weighted least squares or weighted singular value decomposition, and the current pose parameters are iteratively updated.
[0147] ;
[0148] in, They represent the first Rotation matrix and translation vector in each iteration; , They represent the first Rotation matrix and translation vector in each iteration; , They represent the first Rotation increment and translation increment in each iteration;
[0149] When the pose change between two consecutive iterations is less than a preset threshold, or when the preset number of iterations is reached, the optimization stops, and the final rotation matrix and final translation vector are obtained as the final pose parameters.
[0150] S6. Based on the final pose parameters, unify the measured point cloud and the theoretical CAD model into the same coordinate system, calculate the spatial pose deviation parameters of the measured point cloud relative to the theoretical CAD model, and output the cross-modal registration results.
[0151] As a preferred embodiment, in this embodiment, such as Figure 2 As shown, step S6 specifically includes:
[0152] Based on the final pose parameters, the original measured point cloud... A rigid transformation is performed to obtain the final registration point cloud unified to the theoretical CAD model design coordinate system. ;
[0153] For each final registration point Search for the corresponding reference point in the CAD reference point set Q or the surface of the theoretical CAD model. ,calculate and Inter-registration residuals ; and then according to Local normal vector at point ,calculate Deviation of normal surface relative to the theoretical CAD model surface ,when When indicates that the measured point is convex relative to the theoretical surface, when The time indicates that the measured point is concave relative to the theoretical surface;
[0154] The final registration error evaluation results of all points are statistically analyzed, and the cross-modal registration results are output, including the final pose parameters, the final registered point cloud, the registration residuals of each point, and the distribution of normal surface deviation.
[0155] In addition, such as Figure 2 As shown, this application also discloses a cross-modal registration system for CAD models and point clouds of aerospace components based on visual key feature constraints. This system is used to execute the method described above, specifically including:
[0156] The data acquisition and preprocessing module is used to acquire the theoretical CAD model, multi-view RGB images, and original measured point cloud of the aerospace component to be tested; to perform surface discretization sampling on the theoretical CAD model to generate a CAD reference point set; and to establish the point map association relationship between the pixels of the multi-view RGB image and the original measured point cloud based on the calibration parameters.
[0157] The Hilbert-Mamba encoding module is used to rearrange the original measured point cloud using Hilbert curves and extract the geometric features of the measured point cloud of the aircraft component under test by combining the Mamba state space model.
[0158] The visual key feature recognition module is used to identify key visual features of aerospace assembly from multi-view RGB images, calculate the corresponding two-dimensional image confidence scores, and generate a multi-view two-dimensional image confidence score mapping table.
[0159] The weighted initial registration module is used to establish candidate correspondences based on the confidence mapping table of two-dimensional images, combined with the geometric features of the measured point cloud and the geometric neighborhood features of the CAD reference point set. It calculates the registration weights based on the geometric feature similarity of the candidate matching point pairs and the confidence of the two-dimensional images, uses the registration weights to eliminate low-confidence candidate matching point pairs, and uses the retained candidate matching point pairs and registration weights to solve the weighted pose and obtain the initial pose state.
[0160] The point flow field fine registration module is used to construct a generative point flow field matching model that continuously evolves from the measured point cloud to the standard pose of the theoretical point cloud based on the initial pose state. It introduces the overlapping sensing geometric features of the key contact area of the aerospace component, the confidence of the two-dimensional image, and the registration weight of the candidate matching point pairs. It applies weighted constraints to the residual displacement direction of the point flow field and the rigid pose increment solution process, and obtains the final pose parameters through continuous pose optimization.
[0161] The registration result output module is used to unify the measured point cloud and the theoretical CAD model into the same coordinate system based on the final pose parameters, calculate the registration residual and normal surface deviation, and output the cross-modal registration result.
[0162] An electronic device is also disclosed, comprising a memory and a processor, wherein:
[0163] Memory is used to store computer programs that can run on a processor;
[0164] The processor is configured to, while running the computer program, execute a cross-modal registration method for CAD models of aerospace components and point clouds based on visual key feature constraints, as described above.
[0165] A computer-readable storage medium is also disclosed, storing computer instructions for instructing a processor to implement, as described above, a cross-modal registration method for aerospace component CAD models and point clouds based on visual key feature constraints. In summary, this invention provides a cross-modal registration method for aerospace component CAD models and point clouds based on visual key feature constraints, significantly improving the registration accuracy and efficiency of incomplete, symmetrical, and massive point cloud data in large-scale aerospace scenarios. This results in deviation measurements with high robustness, good real-time performance, and low uncertainty, playing a crucial role in the digital precision assembly of large aerospace components.
[0166] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.
[0167] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).
[0168] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. An aviation component CAD model and point cloud cross-modal registration method based on visual key feature constraint, characterized in that, Specifically, the following steps are included: S1. Acquire the theoretical CAD model, multi-view RGB images, and original measured point cloud of the aerospace component to be tested; perform surface discretization sampling on the theoretical CAD model to generate a CAD reference point set; and establish the point map association relationship between the pixels of the multi-view RGB image and the original measured point cloud based on the calibration parameters. S2. The original measured point cloud is rearranged using Hilbert curves, and the geometric features of the measured point cloud of the aircraft component to be tested are extracted from the rearranged original measured point cloud using the Mamba state space model. S3. Identify the key visual features of aerospace components from multi-view RGB images and calculate the two-dimensional image confidence of each key visual feature; summarize the two-dimensional image confidence of each view to obtain a multi-view two-dimensional image confidence mapping table. S4. Based on the multi-view 2D image confidence mapping table, and combining the geometric features of the measured point cloud with the geometric neighborhood features of the CAD reference point set, establish candidate matching relationships between the CAD reference point set and the measured point cloud; calculate the registration weight based on the geometric feature similarity and 2D image confidence of the candidate matching point pairs, and use the registration weight to constrain the confidence of the candidate matching point pairs, eliminating candidate matching point pairs with low registration confidence; perform weighted pose solving based on the retained candidate matching point pairs and their registration weights to determine the initial pose state. S5. Based on the initial pose state, construct a generative point flow field matching model that continuously evolves the measured point cloud to the standard pose of the theoretical point cloud. In the generative point flow field matching model, the overlapping sensing geometric features of the key contact area of the aerospace component, the confidence level of the two-dimensional image, and the registration weight of the candidate matching point pairs are introduced to impose weighted constraints on the residual displacement direction of the point flow field and the rigid pose increment solution process, so that the measured point cloud gradually evolves from the initial pose state to the target pose state along the continuous spatial vector field, and the final pose parameters are obtained through continuous pose optimization. The overlapping sensing geometric features include the distance between points, normal consistency, and local neighborhood overlap. The key contact area of the aerospace component is the area that participates in assembly contact or positioning. S6. Based on the final pose parameters, unify the measured point cloud and the theoretical CAD model into the same coordinate system, calculate the spatial pose deviation parameters of the measured point cloud relative to the theoretical CAD model, and output the cross-modal registration results.
2. The cross-modal registration method for CAD models and point clouds of aerospace components based on visual key feature constraints according to claim 1, characterized in that, Step S1 specifically includes: 3D scanning of aerospace components using a 3D scanner to obtain the original measured point cloud. ; Multi-view RGB images of aerospace components were acquired by deploying industrial cameras at different angles. Simultaneously retrieve the theoretical CAD digital model of the aerospace components. Based on theoretical CAD models Discretize the surface geometry, assembly boundaries, assembly holes, or assembly contact areas to generate a set of CAD reference points for registration. ; Based on the intrinsic and extrinsic parameters of the industrial camera and the calibration parameters between the 3D scanner and the industrial camera, the original measured point cloud was... The three-dimensional points in the image are projected onto the RGB image plane of the corresponding viewpoint to obtain the projection position of each three-dimensional point in the image; based on the projection positions, a multi-view RGB image pixel and the original measured point cloud are established. The point graph relationships between three-dimensional points.
3. The cross-modal registration method for CAD models and point clouds of aerospace components based on visual key feature constraints according to claim 1, characterized in that, Step S2 specifically includes: Voxelization is performed on the original measured point cloud: each original measured point is mapped to voxel coordinates according to the preset voxel resolution; Based on the Hilbert curve mapping function, voxel coordinates are mapped to Hilbert indices; The original measured points are sorted in ascending order according to Hilbert indices to form a point cloud sequence that maintains the continuity of spatial neighborhood; A point cloud geometric feature encoding model based on the Mamba state space architecture is constructed. The point cloud geometric feature encoding model includes an input embedding layer, a selective state space recursive unit, and an output mapping layer. The embedding layer encodes each point in the point cloud sequence into a point feature vector. The selective state space recursively updates the hidden state along the point cloud sequence and obtains the output feature based on the hidden state. The output mapping layer maps the output feature to the measured point cloud geometric feature. The measured point cloud geometric feature is a point-by-point feature vector that corresponds one-to-one with each point in the measured point cloud, including the geometric information of each point and its local neighborhood.
4. The cross-modal registration method for CAD models and point clouds of aerospace components based on visual key feature constraints according to claim 1, characterized in that, Step S3 specifically includes: A visual key feature recognition model for aviation assembly is constructed, comprising an image feature encoding layer, a feature decoding layer, and a key feature prediction layer; RGB images from the site are used for this purpose. Input the visual key feature recognition model to identify the visual key features in the aerospace components, and output the pixel-level feature map and semantic response score of each viewpoint corresponding to the visual key features. For the l-th visual key feature pixel in the pixel-level feature map under the v-th viewpoint Based on the point map correlation between pixels in multi-view RGB images and the original measured point cloud, the relationship between the two is determined. Corresponding original measured points ,according to Projection position in the RGB image plane at the v-th viewpoint and Calculate the pixel distance between them. and Projection consistency score ; Based on key visual feature pixels semantic response score Projection consistency score ,calculate Confidence of two-dimensional images; The confidence scores of each visual key feature pixel from each viewpoint are summarized in two-dimensional images, and then written into the corresponding two-dimensional image confidence score mapping table according to the pixel position, finally obtaining the multi-view two-dimensional image confidence score mapping table of visual key features. .
5. The cross-modal registration method for CAD models and point clouds of aerospace components based on visual key feature constraints according to claim 1, characterized in that, Step S4 specifically includes: For any original measured point Obtain the geometric features of its measured point cloud. ; Extract each CAD reference point from the CAD reference point set Q. Geometric neighborhood features ;calculate and Geometric feature similarity between ; CAD reference points with feature similarity greater than a preset threshold or ranking in the top K of feature similarity are selected as candidate reference points, and a set of candidate matching point pairs is obtained. ; Based on the original measured points Projection position in the RGB image plane at the corresponding viewpoint From the confidence mapping table of multi-view two-dimensional images The extracted 2D image confidence score, combined with candidate matching point pairs Calculate the registration weights of candidate matching point pairs based on the geometric feature similarity between them. The formula is expressed as: ； in, A mapping function representing a confidence mapping table for multi-view two-dimensional images; Based on candidate matching point pairs and registration weights, a weighted pose calculation objective function is constructed; the objective function is then solved through weighted singular value decomposition to obtain the coarse registration rotation matrix. and coarse registration translation vector ; based on , Calculate the registration residual between each pair of candidate matching points; Candidate matching point pairs with registration weights lower than a preset weight threshold and registration residuals greater than a preset residual threshold are considered low-confidence mismatches and are discarded. Based on the retained candidate matching point pairs and their registration weights, the weighted pose solving objective function is reconstructed, and the initial rotation matrix is obtained. and the initial translation vector This serves as the initial pose state.
6. The cross-modal registration method for CAD models and point clouds of aerospace components based on visual key feature constraints according to claim 1, characterized in that, Step S5 specifically includes: Original measured point cloud Perform pose transformation according to the initial pose state to obtain the initial aligned point cloud. ; against Each initial alignment point in In the CAD reference point set Based on overlapping sensing geometric features, it provides Search for the corresponding nearby reference points The overlapping sensing geometric weights are calculated based on the distance between points, normal consistency, and the degree of local neighborhood overlap. The formula is expressed as: ； in, for and Overlap score between local neighborhoods for The normal vector, For nearby reference points The normal vector; For distance scale parameters; for Overlapping perceived geometric weights; A continuously evolving generative point flow field matching model is constructed, which includes a point cloud feature encoding module, a cross-modal feature fusion module, a point flow field prediction module, and a rigid pose update module. The initial aligned point cloud, theoretical point cloud, measured point cloud geometric features, 2D image confidence scores of each point, and registration weights of retained candidate matching point pairs are input into the generative point flow field matching model; the initial aligned point cloud is first extracted through the point cloud feature encoding module. With CAD reference point set Geometric neighborhood features; then, in the cross-modal feature fusion module, the initial aligned point cloud is processed. Geometric neighborhood features, CAD reference point set The geometric neighborhood features, measured point cloud geometric features, two-dimensional image confidence of each original measured point, and registration weights of candidate matching point pairs are concatenated to obtain fused features; Subsequently, in the point flow field prediction module, the orientation of each initial alignment point toward the CAD reference point set is predicted based on the fusion features. The residual displacement direction at the corresponding position; the residual displacement direction is weighted and constrained using overlapping sensing geometric weights to obtain the constrained residual displacement vector of each initial alignment point. ; Finally, in the rigid pose update module, based on the constrained residual displacement vector... A weighted optimization objective function is constructed to solve for the global rigid pose increment; the rotation increment is solved by weighted least squares or weighted singular value decomposition. Translational increment The current pose parameters are iteratively updated; when the pose change between two adjacent iterations is less than a preset threshold, or when the preset number of iterations is reached, the optimization stops, and the final rotation matrix is obtained. and the final translation vector , as the final pose parameter.
7. The cross-modal registration method for CAD models and point clouds of aerospace components based on visual key feature constraints according to claim 1, characterized in that, Step S6 specifically includes: Based on the final pose parameters, the original measured point cloud... A rigid transformation is performed to obtain the final registration point cloud unified to the theoretical CAD model design coordinate system. ; For each final registration point Search for the corresponding reference point in the CAD reference point set Q or the surface of the theoretical CAD model. ,calculate and Inter-registration residuals ; and then according to Local normal vector at point ,calculate Deviation of normal surface relative to the theoretical CAD model surface ; The final registration error evaluation results of all points are statistically analyzed, and the cross-modal registration results are output, including the final pose parameters, the final registered point cloud, the registration residuals of each point, and the distribution of normal surface deviation.
8. A cross-modal registration system for CAD models and point clouds of aerospace components based on visual key feature constraints, the system being used to execute the method as described in any one of claims 1-7, characterized in that, Specifically, it includes: The data acquisition and preprocessing module is used to acquire the theoretical CAD model, multi-view RGB images and original measured point cloud of the aerospace component to be tested; The theoretical CAD model is discretized and sampled to generate a CAD reference point set; and the point map association relationship between the pixels of the multi-view RGB image and the original measured point cloud is established according to the calibration parameters. The Hilbert-Mamba encoding module is used to rearrange the original measured point cloud using Hilbert curves and extract the geometric features of the measured point cloud of the aircraft component under test by combining the Mamba state space model. The visual key feature recognition module is used to identify key visual features of aerospace assembly from multi-view RGB images, calculate the corresponding two-dimensional image confidence scores, and generate a multi-view two-dimensional image confidence score mapping table. The weighted initial registration module is used to establish candidate correspondences based on the confidence mapping table of two-dimensional images, combined with the geometric features of the measured point cloud and the geometric neighborhood features of the CAD reference point set. It calculates the registration weights based on the geometric feature similarity of the candidate matching point pairs and the confidence of the two-dimensional images, uses the registration weights to eliminate low-confidence candidate matching point pairs, and uses the retained candidate matching point pairs and registration weights to solve the weighted pose and obtain the initial pose state. The point flow field fine registration module is used to construct a generative point flow field matching model that continuously evolves from the measured point cloud to the standard pose of the theoretical point cloud based on the initial pose state. It introduces the overlapping sensing geometric features of the key contact area of the aerospace component, the confidence of the two-dimensional image, and the registration weight of the candidate matching point pairs. It applies weighted constraints to the residual displacement direction of the point flow field and the rigid pose increment solution process, and obtains the final pose parameters through continuous pose optimization. The registration result output module is used to unify the measured point cloud and the theoretical CAD model into the same coordinate system based on the final pose parameters, calculate the registration residual and normal surface deviation, and output the cross-modal registration result.
9. An electronic device, characterized in that, The electronic device includes a memory and a processor, wherein: Memory is used to store computer programs that can run on a processor; A processor, configured to, while running the computer program, execute a cross-modal registration method for CAD models of aerospace components and point clouds based on visual key feature constraints as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute a cross-modal registration method for CAD models of aerospace components and point clouds based on visual key feature constraints as described in any one of claims 1-7.