A component measurement path planning method, system, device and medium based on sparse images
By reconstructing sparse images and decomposing features, a surface normal vector field is generated, and the camera measurement pose is planned. This solves the problem of acquiring information on large-scale measurement surfaces, realizes efficient 3D surface measurement, and improves measurement efficiency and intelligence.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUNAN UNIV
- Filing Date
- 2026-04-13
- Publication Date
- 2026-06-26
AI Technical Summary
Existing high-precision 3D surface acquisition equipment is unable to acquire large-scale measurement surface information at once, and the measurement of unknown workpiece models requires human teaching, resulting in low measurement efficiency and low level of intelligence.
A coarse 3D data model is reconstructed using sparse images. A fine mask is generated by expanding the boundary and filling the mask matrix. A visual mask 3D point is constructed to generate an initial sparse point cloud. Clustering and redundant region removal are performed. A covariance matrix is constructed for eigenvalue decomposition to obtain the surface normal vector field. The camera's spatial position and pose model are calculated, the field of view coverage is analyzed, and the optimal path is planned.
It enables intelligent and autonomous measurement of unknown workpieces, enhances the effectiveness and reliability of the measurement coverage area, improves measurement efficiency, and solves the problem of balancing coverage integrity and path optimization in traditional planning.
Smart Images
Figure CN122289556A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of intelligent measurement, and in particular to a method, system, device and medium for component measurement path planning based on sparse images. Background Technology
[0002] With the innovation of digital and information technologies, modern digital intelligent manufacturing is continuously empowering industrial production fields, such as the manufacturing, processing, and assembly of aerospace equipment. For the production engineering of large aircraft, ensuring the quality of manufacturing and assembly is a key factor determining the overall performance and safe operation of the aircraft. Large aircraft components are characterized by complex curved shapes and large dimensions, such as large skin panels, and have high precision requirements for features such as the contour of fluid surfaces, rivet flatness, and seam gaps between connecting surfaces. Therefore, it is necessary to perform three-dimensional information measurement on components such as skin panels to provide quality assurance for subsequent manufacturing and assembly.
[0003] Existing high-precision 3D surface acquisition equipment has limited measurement coverage, making it difficult to acquire comprehensive, large-scale measurement surface information in a single operation. Therefore, comprehensive 3D information measurement requires intelligent, autonomous robot route planning to avoid scanning gaps and complete the full surface measurement. Currently, for measurement route planning of unknown workpiece models, since the workpiece is completely unknown, the measurement system often requires human teaching to continuously correct the measurement results. This approach often results in redundant perspectives, hindering intelligent, autonomous, and unmanned measurement, reducing the overall intelligence of the system, and leading to low measurement efficiency. These problems need to be addressed. Summary of the Invention
[0004] To quickly acquire 3D measurement information, provide high-quality data support for subsequent measurement operations, and improve the efficiency of component measurement, this application provides a component measurement path planning method, system, device, and medium based on sparse images, employing the following technical solution: In a first aspect, this application provides a component measurement path planning method based on sparse images, including: Obtain a sparse image, perform boundary expansion and fill the mask matrix on the sparse image to obtain a fine mask; Based on the fine mask, obtain the visual mask 3D points, generate an initial sparse point cloud based on the visual mask 3D points, and construct a rough 3D data model based on the initial sparse point cloud. Clustering and redundant regions are removed from the rough 3D data model to obtain the effective measurement unit working area. The covariance matrix is constructed based on the working area of the effective measurement unit, and the surface normal vector field is obtained by eigenvalue decomposition of the covariance matrix. The camera's spatial position is calculated based on the surface normal vector field and the preset vertical measurement height. The camera's measurement posture is then modeled using the camera's spatial position to obtain the camera measurement posture model. The effective scanning coverage area is obtained by analyzing the field of view coverage of the working area of the effective measurement unit and the normal vector field of the surface through the camera measurement attitude model. The optimal path objective function is obtained by analyzing the scan spatial position based on the effective scan coverage area and camera spatial position.
[0005] Preferably, the specific steps for expanding the boundaries of the sparse image and filling the mask matrix to obtain a fine mask are as follows: The geometric information of the sparse image is used to expand the boundary to obtain the expanded boundary parameters; A coarse mask is obtained by filling the mask matrix with the extended boundary parameters. Foreground and background cue points are extracted from the coarse mask, and the image features, foreground and background cue points are fused to obtain the candidate mask; The candidate masks are evaluated for confidence and filtered to obtain a refined mask.
[0006] Preferably, the specific steps for obtaining the visual mask 3D points based on the fine mask and generating the initial sparse point cloud based on the visual mask 3D points are as follows: The view frustum region is obtained by performing a three-dimensional back projection on the fine mask. Calculate the intersection of several visual cone regions to obtain visual shell information; The point cloud boundary is defined based on the visual shell information. The point cloud boundary is uniformly sampled in three dimensions. The sampled points are then projected to obtain the three-dimensional points of the visual mask. An initial sparse point cloud is generated from 3D points based on a visual mask.
[0007] Preferably, the specific steps for constructing the covariance matrix based on the effective measurement unit working area are as follows: The effective measurement unit's working area is downsampled, and a neighborhood search is performed on the downsampled points to obtain the neighborhood point set corresponding to the three-dimensional point; The covariance matrix is constructed based on the neighborhood point set corresponding to the three-dimensional point.
[0008] Preferably, the specific steps for performing eigenvalue decomposition on the covariance matrix to obtain the surface normal vector field are as follows: The covariance matrix is decomposed into eigenvalues to obtain the initial normal vector; The initial normal vector is smoothed and anomalies are removed to obtain the surface normal vector field.
[0009] Preferably, the specific steps for performing field-of-view coverage analysis on the effective measurement unit working area and the surface normal vector field using the camera measurement attitude model to obtain the effective scan coverage area are as follows: A measurement feature information database is constructed based on the working area of the effective measurement unit and the surface normal vector field; By using a camera-measured attitude model to perform field-of-view coverage analysis on the established measurement feature information database, the effective scan coverage area can be obtained. The global pose is calculated based on the effective scan coverage area, resulting in a global pose set; Based on the global pose set and camera spatial position analysis, the scan spatial position is obtained to obtain the global scan planning pose set; The optimal path objective function is calculated based on the global scan planning pose set.
[0010] Preferably, the specific steps for boundary expansion and mask matrix filling of the sparse image include: Preprocessing of sparse images includes distortion correction, color equalization, and setting of geometric information parameters.
[0011] Secondly, this application provides a component measurement path planning system based on sparse images, comprising: The sparse image acquisition module is used to acquire sparse images, perform boundary expansion and fill the mask matrix on the sparse images to obtain a fine mask; The coarse model construction module is used to obtain visual mask 3D points based on fine mask, generate an initial sparse point cloud based on the visual mask 3D points, and construct a coarse 3D data model based on the initial sparse point cloud. The coarse model processing module is used to cluster and remove redundant regions from the coarse 3D data model to obtain the effective measurement unit working area. The eigenvalue decomposition module is used to construct the covariance matrix based on the working area of the effective measurement unit, and to perform eigenvalue decomposition on the covariance matrix to obtain the surface normal vector field. The attitude model construction module is used to calculate the camera spatial position based on the surface normal vector field and the preset vertical measurement height, and to perform camera measurement attitude modeling based on the camera spatial position to obtain the camera measurement attitude model; The attitude model processing module is used to perform field-of-view coverage analysis on the working area of the effective measurement unit and the surface normal vector field through the camera measurement attitude model to obtain the effective scan coverage area. The path planning module is used to analyze the scan spatial position by effectively scanning the coverage area and the camera spatial position, and calculate the optimal path objective function.
[0012] Thirdly, this application provides a component measurement path planning device based on sparse images, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the component measurement path planning method based on sparse images as described above.
[0013] Fourthly, this application provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the component measurement path planning method based on sparse images as described above when running.
[0014] In summary, compared with the prior art, the beneficial effects of the technical solution provided in this application include at least the following: This application addresses the issues of excessive manual intervention and low intelligence in existing unknown workpiece measurement by reconstructing a coarse 3D data model from sparse images. Based on the coarse 3D data model, it obtains the effective measurement unit working area, surface normal vector field, and camera measurement posture model. Accurate matching is achieved through point cloud and camera posture, enhancing the effectiveness and reliability of the camera measurement coverage area. By analyzing the effective scanning coverage area and camera spatial position, the optimal path objective function is calculated, completing the optimal path planning. This solves the drawback of traditional planning where coverage integrity and path optimization are difficult to balance, providing high-quality data support for subsequent measurement operations and improving the efficiency of component measurement. Attached Figure Description
[0015] Figure 1 This is a flowchart illustrating a component measurement path planning method based on sparse images as described in an embodiment of this application.
[0016] Figure 2 This is a schematic diagram of a component measurement path planning system based on sparse images, as described in an embodiment of this application.
[0017] Explanation of reference numerals in the attached figures: 1. Sparse image acquisition module; 2. Coarse model construction module; 3. Coarse model processing module; 4. Feature decomposition module; 5. Pose model construction module; 6. Pose model processing module; 7. Path planning module. Detailed Implementation
[0018] The following combination Figures 1-2 The present application will be described in further detail below. The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting.
[0019] Reference Figure 1 The component measurement path planning method based on sparse images involved in this application specifically includes: Step S1: Obtain the sparse image, perform boundary expansion and fill the mask matrix on the sparse image to obtain the fine mask; Step S2: Obtain visual mask 3D points based on fine mask, generate initial sparse point cloud based on visual mask 3D points, and construct a rough 3D data model based on initial sparse point cloud; Step S3: Perform clustering and redundant region removal on the rough 3D data model to obtain the effective measurement unit working area; Step S4: Construct the covariance matrix based on the working area of the effective measurement unit, perform eigenvalue decomposition on the covariance matrix, and obtain the surface normal vector field; Step S5: Calculate the camera spatial position based on the surface normal vector field and the preset vertical measurement height, and perform camera measurement attitude modeling based on the camera spatial position to obtain the camera measurement attitude model; Step S6: Perform field-of-view coverage analysis on the working area of the effective measurement unit and the surface normal vector field using the camera measurement attitude model to obtain the effective scanning coverage area; Step S7: Analyze the scan spatial position by effectively scanning the coverage area and camera spatial position, and calculate the optimal path objective function.
[0020] Specifically, this embodiment of the application expands the boundaries of sparse images and fills them into a mask matrix to obtain a fine mask. This fine mask is then processed into an initial coefficient point cloud for reconstruction, resulting in a coarse 3D data model. Based on this coarse 3D data model, the effective measurement unit working area, surface normal vector field, and camera measurement pose model are obtained. Precise matching is achieved through point cloud and camera pose, enhancing the effectiveness and reliability of the camera measurement coverage area. By analyzing the effective scan coverage area and camera spatial position, the optimal path objective function is calculated, completing optimal path planning. This addresses the drawback of traditional planning methods that struggle to balance coverage integrity and path optimization, providing high-quality data support for subsequent measurement operations and improving the efficiency of component measurement.
[0021] As one implementation method, the specific steps for boundary expansion and mask matrix filling of the sparse image include: Preprocessing of sparse images includes distortion correction, color equalization, and setting of geometric information parameters.
[0022] Specifically, this embodiment utilizes multiple cameras to acquire high-resolution images of aircraft components from different orientations, ensuring that the images cover the main curved surface features and edge contours of the workpiece. After acquisition, preprocessing operations such as distortion correction, color equalization, and target bounding box parameter setting are performed on the n images to eliminate optical distortion and the influence of ambient lighting, obtaining a preprocessed image sequence and a corresponding bounding box geometric information parameter sequence. .
[0023] As one implementation method, the specific steps for expanding the boundaries of a sparse image and filling the mask matrix to obtain a fine mask are as follows: The geometric information of the sparse image is used to expand the boundary to obtain the expanded boundary parameters; A coarse mask is obtained by filling the mask matrix with the extended boundary parameters. Foreground and background cue points are extracted from the coarse mask, and the image features, foreground and background cue points are fused to obtain the candidate mask; The candidate masks are evaluated for confidence and filtered to obtain a refined mask.
[0024] Specifically, in this embodiment, the bounding box geometric information parameter T corresponding to the preprocessed image is used to expand the bounding box. The width and height of the target bounding box are calculated, and the original box is expanded outward by a certain proportion to obtain the expanded bounding box parameter sequence. The specific extended formula is as follows:
[0025] Among them W n H n represents the width and height of the nth image in the image sequence; r is the scaling parameter.
[0026] This application embodiment utilizes extended bounding box parameters to lock target candidate regions based on pixel-level binary segmentation, generating a geometrically approximate coarse mask. First, a background pixel matrix with the exact same size as the input image is constructed by initializing the mask matrix. Then, the target region is filled by foreground pixels within the extended bounding box. The pixel values of the mask are defined as follows: ; in For mask pixels The pixel value; 0 represents the background pixel value; 255 represents the foreground pixel value.
[0027] This application's embodiments are based on a coarse mask. Foreground and background cue points are extracted, and the image features and geometric cue information are fused using the SAM model to generate multiple candidate masks. The optimal result is selected based on confidence scores to obtain the final refined mask. The specific expression for extracting cue points in this embodiment is as follows:
[0028] in, For a rough mask outline, For the area inside the mask, For the outer area of the mask, and These represent the number of samples taken from the foreground and background points, respectively.
[0029] The formula for mask confidence evaluation and optimization is as follows: .
[0030] in The set of candidate mask images generated for SAM. For the corresponding confidence level, The optimal mask image.
[0031] If the best confidence level , If the confidence threshold is used, then the system is re-optimized based on bounding box hints, and the new confidence level is determined. satisfy .
[0032] As one implementation method, the specific steps for obtaining visual mask 3D points based on fine masking and generating an initial sparse point cloud based on the visual mask 3D points are as follows: The view frustum region is obtained by performing a three-dimensional back projection on the fine mask. Calculate the intersection of several visual cone regions to obtain visual shell information; The point cloud boundary is defined based on the visual shell information. The point cloud boundary is uniformly sampled in three dimensions. The sampled points are then projected to obtain the three-dimensional points of the visual mask. An initial sparse point cloud is generated from 3D points based on a visual mask.
[0033] Specifically, the embodiments of this application will use the corresponding viewpoint mask image. Back-projecting into 3D space, a view frustum region is constructed representing the potential location of the corresponding component. The intersection of multiple view frustums is calculated to obtain the possible spatial range of the aircraft component, i.e., the visual shell, which defines the initial point cloud generation boundary. Subsequently, 3D points are uniformly sampled within this defined region, and each sampled point is projected back into the mask region of each view image. 3D points whose projections fall within all view masks are retained to generate the initial point cloud. The specific formula is: ; ; ; Where n is the total number of viewpoints; This indicates that the three-dimensional point P i Project the image plane onto the k-th viewpoint; It is the mask region of the aircraft component from the kth viewpoint; The view frustum corresponding to the k-th viewpoint; For visual shell; This is the initial point cloud; This application embodiment utilizes the KNN algorithm to further optimize the initial point cloud, sets a neighborhood search radius, removes isolated points with fewer than a preset threshold in the neighborhood, further optimizes the point cloud quality, and obtains the final initial sparse point cloud.
[0034] The initial sparse point cloud obtained in this embodiment of the application The 3D points are transformed to the camera coordinate system of each viewpoint through camera extrinsic parameters, and then projected onto the 2D image plane using intrinsic parameters. The transformation formula is: ; ; ; in Let be the extrinsic rotation matrix of the k-th camera. Let be the extrinsic translation vector of the k-th camera; Let be the coordinates of a 3D point in the k-th viewpoint camera coordinate system; The intrinsic parameter matrix of the k-th camera; the pixel coordinates corresponding to the two-dimensional image plane. ; The coordinates of the main point.
[0035] This application's embodiments utilize bilinear interpolation to obtain the color information of corresponding pixels and employ spherical harmonic expansion to encode the colors, capturing the color distribution characteristics in different directions. Then, the average Euclidean distance between each point and its neighboring points is calculated as a spatial scale estimate for that point, controlling the spatial coverage and blurriness of its corresponding Gaussian particle. The scale estimation formula is: ; Where K is the number of preset domain points; This is a two-dimensional Euclidean distance calculation operator.
[0036] Simultaneously, the rotation parameters of each point in the point cloud are initialized to unit quaternions to ensure an initial state without rotation. Based on the above parameters, a complete initial 3D Gaussian model is constructed. The specific expression of the initial 3D Gaussian model is as follows:
[0037] Among them, G i Represents a 3D Gaussian model. Initial point cloud of aircraft components. The spherical harmonic color coefficient for each point in the spectrum is c. i The Gaussian scale is σ i Quaternion rotation to R i Transparency is α i .
[0038] In this embodiment, the initial Gaussian model optimization first involves rendering corresponding 2D images from multiple viewpoints. The rendered images are then compared with real-world multi-view images, and three main losses—color loss, masking loss, and depth loss—are calculated. These three losses are then weighted and combined to form a total loss function. The expression for the loss function is: = + + ; ; ; ; Where α, β, and γ are hyperparameters; L color Color loss measures the difference in pixel color between the rendered image and the real image; L mask The masking loss is used to enhance the accuracy of aircraft component outlines, calculating the difference between the masked regions predicted by the segmentation model and the actual mask labels; L depth To address the depth loss, a monocular depth estimation network, ZoeDepth, is introduced as an auxiliary supervisor to estimate the distance from surface points to the camera. The L1 norm is used to measure the difference between the rendered depth and the estimated depth. This represents the rendered color of the j-th pixel; The color of the corresponding pixel in the real image; M is the total number of pixels; y i For real mask labels; The predicted mask probability for the corresponding pixel in the rendered image; This represents the depth value of the rendered image obtained from the current 3D Gaussian model. This represents the depth estimated by the pre-trained depth estimation network.
[0039] In this embodiment of the application, the backpropagation algorithm is used during the training process to iteratively optimize each Gaussian particle in the point cloud. By considering parameters such as these, a continuous, coarse three-dimensional data model is ultimately generated.
[0040] As one implementation method, the specific steps for constructing the covariance matrix based on the effective measurement unit working area are as follows: The effective measurement unit's working area is downsampled, and a neighborhood search is performed on the downsampled points to obtain the neighborhood point set corresponding to the three-dimensional point; The covariance matrix is constructed based on the neighborhood point set corresponding to the three-dimensional point.
[0041] Specifically, in this embodiment, after the rough 3D data model is trained, the optimal pose constraints of the camera in the working measurement space need to be defined to ensure that the camera can clearly acquire workpiece features during the measurement process, thus completing the camera measurement pose modeling. Specifically, based on the trained rough 3D data model, Euclidean clustering algorithm is used to segment the model point cloud into multiple continuous sub-region measurement working units. Redundant regions exceeding the preset measurement space range are removed to obtain the effective working area, and the constraints are defined. The expression for the effective measurement unit working area is: ; ; ; Where S is the set of all point cloud sub-regions obtained after segmentation by the Euclidean clustering algorithm; Euclidean clustering operator; These are independent, continuous, and uninterrupted individual point cloud sub-regions; This is a rough three-dimensional data model; The Euclidean clustering distance threshold; This is the preset effective measurement space range; Effective work area; This is a redundant area.
[0042] In this embodiment, the effective working unit point cloud after segmentation is downsampled, and a K-nearest neighbor search is performed on the downsampled point cloud to obtain the neighborhood point set corresponding to each 3D point. A covariance matrix is constructed based on the neighborhood point set.
[0043] As one implementation method, the specific steps for performing eigenvalue decomposition on the covariance matrix to obtain the surface normal vector field are as follows: The covariance matrix is decomposed into eigenvalues to obtain the initial normal vector; The initial normal vector is smoothed and anomalies are removed to obtain the surface normal vector field.
[0044] Specifically, in this embodiment, the initial normal vector is extracted by eigenvalue decomposition of the covariance matrix, and then smoothing optimization and anomaly removal are performed to obtain an accurate surface normal vector field.
[0045] Specifically, for any measurement point's neighborhood set, a three-dimensional covariance matrix is constructed, and eigenvalue decomposition is performed, as shown in the formula: ; ; ; in The coordinates are the mean coordinates of the neighborhood point set; Let K be the coordinates of the j-th point in the neighborhood; K is the preset number of neighborhood points. It is an eigenvalue diagonal matrix; For the eigenvector matrix, find the smallest eigenvalue. The corresponding eigenvector is used as the initial normal vector of that point. .
[0046] This application's embodiment uses the Laplace smoothing algorithm to optimize the initial normal vector, specifically: ; ; in This is the normal vector of the i-th point after smoothing; Let i be the set of neighborhood points of the i-th point; This represents the neighborhood weight.
[0047] By statistically analyzing the distribution of normal vector directions and eliminating anomalies that deviate from the mainstream direction, a continuous and smooth normal vector field for each working unit is finally obtained. The direction of the normal vector is the optimal optical axis direction for camera measurement.
[0048] This application embodiment is based on the aforementioned continuous smooth normal vector field. By combining the set vertical measurement height of the measuring camera, the spatial position of the camera is calculated.
[0049] The formula for calculating the spatial position L of the camera is: .
[0050] Where l is the set vertical measurement height of the measuring camera; The normal vector field is a continuous and smooth normal vector field, i.e., the optimized normal vector; L is the three-dimensional spatial position of the camera corresponding to the measurement point.
[0051] To ensure the continuity between the camera's measurement field of view and the measurement surface, this embodiment of the application is based on... Construct a camera rotation matrix R to rotate and move the camera from its initial pose to... Direction. The formula for constructing the camera rotation matrix is: ; ; Where R is the camera rotation matrix; R(1), R(2), and R(3) are the corresponding constructed eigenvectors.
[0052] The formula for converting a rotation matrix into three-axis Euler attitude angles is: ; in For Euler's pitch angle; For Euler roll angle; This is the Euler yaw angle.
[0053] This embodiment further completes the modeling of the final camera measurement attitude by constructing a rotation matrix and converting it into three-axis Euler attitude angles. It integrates the camera's three-dimensional spatial position L with the three-axis Euler attitude angles. The final camera measurement pose corresponding to each measurement point is obtained. The expression is: ; in To measure the pose of the final camera; For Euler's pitch angle; For Euler roll angle; This is the Euler yaw angle.
[0054] As one implementation method, the specific steps for performing field-of-view coverage analysis on the effective measurement unit working area and the surface normal vector field using a camera measurement attitude model to obtain the effective scan coverage area are as follows: A measurement feature information database is constructed based on the working area of the effective measurement unit and the surface normal vector field; By using a camera-measured attitude model to perform field-of-view coverage analysis on the established measurement feature information database, the effective scan coverage area can be obtained. The global pose is calculated based on the effective scan coverage area, resulting in a global pose set; Based on the global pose set and camera spatial position analysis, the scan spatial position is obtained to obtain the global scan planning pose set; The optimal path objective function is calculated based on the global scan planning pose set.
[0055] Specifically, the embodiments of this application are based on an effective set of measurement working units. Continuous smooth normal vector field of each measurement unit A measurement feature information database is constructed. The formula for constructing the measurement feature information database is: ; Where X is the measurement feature information database; for Any three-dimensional geometric point in the, This is the optimized normal vector corresponding to this point.
[0056] Combining the camera-measured pose model Camp with the measurement feature information database as input, the field of view coverage is solved to obtain the set of nearest neighbor scan coverage areas corresponding to each sparse point, specifically: ; in For camera field of view coverage operator; This is the set of nearby scan coverage areas.
[0057] This application embodiment collects all fused information tuples. A viewpoint-greedy optimization algorithm is applied to this tuple set to perform global pose selection, resulting in a set of global workpiece scan poses that satisfy the global optimal constraints. The constraint formulas and selection formulas are as follows: ; ; Where M is the minimum number of global poses; To satisfy the global optimal constraint, the workpiece scanning global pose set; It represents a single globally optimal scan pose, containing both camera spatial position and three-axis Euler attitude angle information.
[0058] Finally, by combining the camera spatial position calculation formula L, the scanning spatial position is solved, and the final workpiece global scanning planning pose set is obtained, specifically: ; ; in Plan the pose set for global scanning of the workpiece; Let be the spatial position parameters of the k-th camera.
[0059] This application embodiment constructs a path planning target node set based on the spatial coordinate information of all planned poses. Using the global scan-planned pose as input, a genetic algorithm is used for iterative optimization to solve for the optimal path objective function. The objective function is: ; in The objective function is the optimal path. Adjust the weighting coefficients for the attitude; This represents the total three-axis attitude adjustment of the camera between adjacent poses; M is the length of the spatial translation path of the camera between adjacent planned poses; M is the minimum number of global poses.
[0060] This application significantly improves the efficiency and accuracy of modeling unknown workpieces and enhances the full surface coverage and system reliability of complex curved surface component measurements through sparse image modeling, geometric-normal vector feature fusion and coverage, and a two-layer path optimization mechanism. Specifically, it proposes a general and automatically deployable sparse image 3D reconstruction mechanism, which transforms a small number of sparse images into a continuous coarse 3D model through mask segmentation, view frustum intersection filtering, and Gaussian model optimization, reducing the influence of traditional unknown workpiece measurements relying on dense images or prior models. It proposes a feature-level point cloud information fusion strategy, which accurately fuses the initial sparse point cloud geometric information with optimized normal vectors to construct a complete information database. Through precise matching of point cloud features and camera pose, it systematically enhances the effectiveness and reliability of the camera measurement coverage area. It proposes a two-layer planning mechanism for coverage and path collaborative optimization, which achieves minimum pose full coverage through a viewpoint greedy algorithm and completes optimal path planning by combining a genetic algorithm, solving the drawback of the difficulty in balancing coverage integrity and path optimization in traditional planning.
[0061] Reference Figure 2 This application provides a component measurement path planning system based on sparse images, the system comprising: Sparse image acquisition module 1 is used to acquire sparse images, perform boundary expansion and fill the mask matrix on the sparse images to obtain a fine mask; The coarse model construction module 2 is used to obtain visual mask 3D points based on fine mask, generate an initial sparse point cloud based on visual mask 3D points, and construct a coarse 3D data model based on the initial sparse point cloud. The coarse model processing module 3 is used to perform clustering and redundant region removal on the coarse three-dimensional data model to obtain the effective measurement unit working area. Eigenvalue decomposition module 4 is used to construct the covariance matrix based on the working area of the effective measurement unit, and to perform eigenvalue decomposition on the covariance matrix to obtain the surface normal vector field. The attitude model construction module 5 is used to calculate the camera spatial position based on the surface normal vector field and the preset vertical measurement height, and to perform camera measurement attitude modeling based on the camera spatial position to obtain the camera measurement attitude model. The attitude model processing module 6 is used to perform field coverage analysis on the working area of the effective measurement unit and the surface normal vector field through the camera measurement attitude model to obtain the effective scanning coverage area. The path planning module 7 is used to analyze the scan spatial position by effectively scanning the coverage area and the camera spatial position, and calculate the optimal path objective function.
[0062] This application provides a component measurement path planning device based on sparse images, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the component measurement path planning method based on sparse images as described above.
[0063] This application provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the component measurement path planning method based on sparse images as described above when it runs.
[0064] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device and product described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0065] In the several embodiments provided in this application, it should be understood that the disclosed methods, systems, apparatus and program products can be implemented in other ways.
[0066] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0067] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A component measurement path planning method based on sparse images, characterized in that, include: Obtain a sparse image, perform boundary expansion and fill the mask matrix on the sparse image to obtain a fine mask; Based on the fine mask, obtain the visual mask 3D points, generate an initial sparse point cloud based on the visual mask 3D points, and construct a rough 3D data model based on the initial sparse point cloud. Clustering and redundant regions are removed from the rough 3D data model to obtain the effective measurement unit working area. The covariance matrix is constructed based on the working area of the effective measurement unit, and the surface normal vector field is obtained by eigenvalue decomposition of the covariance matrix. The camera's spatial position is calculated based on the surface normal vector field and the preset vertical measurement height. The camera's measurement posture is then modeled using the camera's spatial position to obtain the camera measurement posture model. The effective scanning coverage area is obtained by analyzing the field of view coverage of the working area of the effective measurement unit and the normal vector field of the surface through the camera measurement attitude model. The optimal path objective function is obtained by analyzing the scan spatial position based on the effective scan coverage area and camera spatial position.
2. The component measurement path planning method based on sparse images according to claim 1, characterized in that, The specific steps for expanding the boundaries of the sparse image and filling the mask matrix to obtain a fine mask are as follows: The geometric information of the sparse image is used to expand the boundary to obtain the expanded boundary parameters; A coarse mask is obtained by filling the mask matrix with the extended boundary parameters. Foreground and background cue points are extracted from the coarse mask, and the image features, foreground and background cue points are fused to obtain the candidate mask; The candidate masks are evaluated for confidence and filtered to obtain a refined mask.
3. The component measurement path planning method based on sparse images according to claim 1, characterized in that, The specific steps for obtaining the visual mask 3D points based on the fine mask, and generating the initial sparse point cloud based on the visual mask 3D points are as follows: The view frustum region is obtained by performing a three-dimensional back projection on the fine mask. Calculate the intersection of several visual cone regions to obtain visual shell information; The point cloud boundary is defined based on the visual shell information. The point cloud boundary is uniformly sampled in three dimensions. The sampled points are then projected to obtain the three-dimensional points of the visual mask. An initial sparse point cloud is generated from 3D points based on a visual mask.
4. The component measurement path planning method based on sparse images according to claim 1, characterized in that, The specific steps for constructing the covariance matrix based on the effective measurement unit working area are as follows: The effective measurement unit's working area is downsampled, and a neighborhood search is performed on the downsampled points to obtain the neighborhood point set corresponding to the three-dimensional point; The covariance matrix is constructed based on the neighborhood point set corresponding to the three-dimensional point.
5. The component measurement path planning method based on sparse images according to claim 4, characterized in that, The specific steps for performing eigenvalue decomposition on the covariance matrix to obtain the surface normal vector field are as follows: The covariance matrix is decomposed into eigenvalues to obtain the initial normal vector; The initial normal vector is smoothed and anomalies are removed to obtain the surface normal vector field.
6. The component measurement path planning method based on sparse images according to claim 5, characterized in that, The specific steps for performing field-of-view coverage analysis on the working area of the effective measurement unit and the surface normal vector field using the camera measurement attitude model to obtain the effective scanning coverage area are as follows: A measurement feature information database is constructed based on the working area of the effective measurement unit and the surface normal vector field; By using a camera-measured attitude model to perform field-of-view coverage analysis on the established measurement feature information database, the effective scan coverage area can be obtained. The global pose is calculated based on the effective scan coverage area, resulting in a global pose set; Based on the global pose set and camera spatial position analysis, the scan spatial position is obtained to obtain the global scan planning pose set; The optimal path objective function is calculated based on the global scan planning pose set.
7. The component measurement path planning method based on sparse images according to claim 1, characterized in that, The specific steps for boundary expansion and mask matrix filling of sparse images include: Preprocessing of sparse images includes distortion correction, color equalization, and setting of geometric information parameters.
8. A component measurement path planning system based on sparse images, characterized in that, include: The sparse image acquisition module is used to acquire sparse images, perform boundary expansion and fill the mask matrix on the sparse images to obtain a fine mask; The coarse model construction module is used to obtain visual mask 3D points based on fine mask, generate an initial sparse point cloud based on the visual mask 3D points, and construct a coarse 3D data model based on the initial sparse point cloud. The coarse model processing module is used to cluster and remove redundant regions from the coarse 3D data model to obtain the effective measurement unit working area. The eigenvalue decomposition module is used to construct the covariance matrix based on the working area of the effective measurement unit, and to perform eigenvalue decomposition on the covariance matrix to obtain the surface normal vector field. The attitude model construction module is used to calculate the camera spatial position based on the surface normal vector field and the preset vertical measurement height, and to perform camera measurement attitude modeling based on the camera spatial position to obtain the camera measurement attitude model; The attitude model processing module is used to perform field-of-view coverage analysis on the working area of the effective measurement unit and the surface normal vector field through the camera measurement attitude model to obtain the effective scan coverage area. The path planning module is used to analyze the scan spatial position by effectively scanning the coverage area and the camera spatial position, and calculate the optimal path objective function.
9. A component measurement path planning device based on sparse images, characterized in that, It includes a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the component measurement path planning method based on sparse images as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute, at runtime, the component measurement path planning method based on sparse images as described in any one of claims 1-7.