Visual automatic measurement method for precast size of skirt pile sleeve

Abstract

The application provides a visual automatic measurement method for precast size of skirt pile sleeve, which comprises the following steps: collecting multi-view images of the skirt pile sleeve, detecting feature points and extracting local descriptors and optical flow, and calculating structural confidence based on the minimum eigenvalue of structural tensor and the stability of optical flow; extracting Hough straight line segments and deep texture features, constructing a cross-modal attention network, generating a spatial prior distribution according to the structural confidence, and generating a fusion descriptor; obtaining an initial matching point pair based on the fusion descriptor, constructing a matching point graph model, identifying mis-matching points, and solving an essential matrix by combining two-stage weighted RANSAC; reconstructing a sparse point cloud by using camera internal participation and multi-view geometry, recovering an absolute scale based on a physical baseline, and registering with a digital twin model, so as to map key points as search initial values to determine the measured key point coordinates, calculate the sleeve size, positioning uncertainty and confidence interval.

Description

Technical Field

[0001] This application belongs to the field of automatic measurement, and in particular relates to a visual automatic measurement method for the prefabricated dimensions of skirt pile sleeves. Background Technology

[0002] Because the skirt pile sleeve is an ultra-large spatial steel structure with large component dimensions and complex spatial geometric topology, precise measurements of key dimensions such as diameter, height, flange interface position, and guide structure spacing are required during the land-based prefabrication stage. Existing measurement methods mainly rely on contact or semi-contact means such as total stations, theodolites, and manual string lines, which have problems such as cumbersome measurement procedures, time-consuming on-site setup, and high labor costs. At the same time, the measurement results are usually concentrated on a small number of discrete feature points, making it difficult to comprehensively reflect the overall three-dimensional morphology of the component and local structural deviations. Three-dimensional vision measurement technology based on image sequences has advantages such as non-contact operation, wide coverage, high acquisition efficiency, and high degree of digitization, and can provide a new technical approach for the quality inspection of large steel structure prefabrication.

[0003] However, in actual visual measurement scenarios for large precast steel structures such as skirted pile sleeves, the surface of the components often exhibits large areas of weak texture, similar paint colors, significant metallic reflection, and local occlusion. Furthermore, significant perspective variations exist between multi-view images, easily leading to instability in front-end feature extraction and cross-view feature mismatches. Simultaneously, multi-view geometric reconstruction results typically only provide relative scale, making it difficult to directly meet the requirements for absolute physical dimensions in engineering measurements. A reliable registration relationship also needs to be established between the measured point cloud and the standard digital model to further extract key dimensions and evaluate the reliability of the measurement results. Therefore, it is necessary to propose a comprehensive visual measurement method suitable for large skirted pile sleeves to achieve stable feature fusion, mismatch elimination, absolute scale recovery, digital model registration, and measurement confidence assessment, thereby improving the automation level and reliability of precast dimension detection. Summary of the Invention

[0004] To address the problems in existing technologies regarding the measurement of large skirt pile sleeve dimensions, such as reliance on manual point placement, low measurement efficiency, unstable extraction of weak textured surface features, numerous cross-viewpoint mismatches, lack of absolute scale in visual reconstruction, and difficulty in quantifying the reliability of measurement results, this invention proposes, firstly, an automatic visual measurement method for the prefabricated dimensions of skirt pile sleeves, comprising:

[0005] A multi-viewpoint 2D image sequence of the skirt pile sleeve is acquired, feature points are detected, and local descriptors and optical flow are extracted. Based on the minimum eigenvalue of the feature point neighborhood structure tensor matrix and the optical flow tracking stability, the structure confidence of the feature points is calculated. Hough transform line segments and deep texture feature maps of the 2D images are extracted, and the line segment features in the neighborhood of the feature points are encoded into query vectors. The texture feature maps are used as keys and values. A cross-modal attention network is constructed with feature points as bridges. A spatial prior distribution is generated according to the structure confidence, and the spatial prior distribution is injected into the attention matrix to generate a fusion descriptor for the feature points. Initial matching point pairs are obtained by feature matching based on fusion descriptors, a matching point graph model is constructed, the betweenness centrality of nodes is calculated, and mismatched points are identified and removed based on the betweenness centrality. The essence matrix is ​​solved by combining a two-stage weighted random sampling consensus algorithm with structural confidence weighted sampling and model optimization. A sparse point cloud is generated through 3D reconstruction using the essential matrix and camera intrinsic parameters. The absolute scale of the sparse point cloud is recovered based on the known physical baseline length. The sparse point cloud is registered with a sleeve digital twin model to obtain a rigid transformation matrix. The predefined key measurement points of the twin model are mapped to the point cloud coordinate system. The mapped key measurement points are used as initial search values ​​to determine the coordinates of the measured key measurement points in the neighborhood point cloud. The size of the sleeve is calculated based on the measured key measurement point coordinates. The positioning uncertainty and confidence interval are calculated based on the neighborhood point cloud.

[0006] On the other hand, the present invention also proposes a visual automatic measurement system for the prefabricated dimensions of skirt pile sleeves, comprising: The generation module is used to acquire multi-viewpoint two-dimensional image sequences of the skirt pile sleeve, detect feature points and extract local descriptors and optical flow, calculate the structural confidence of feature points based on the minimum eigenvalue of the feature point neighborhood structure tensor matrix and the optical flow tracking stability, extract Hough transform line segments and deep texture feature maps of the two-dimensional images, encode the line segment features in the neighborhood of the feature points into query vectors, use the texture feature maps as keys and values, construct a cross-modal attention network with feature points as bridges, generate spatial prior distributions based on the structural confidence, inject the spatial prior distributions into the attention matrix, and generate fusion descriptors for feature points. The solution module is used to obtain initial matching point pairs based on feature matching of fused descriptors, construct a matching point graph model, calculate the betweenness centrality of nodes, identify and remove mismatched points based on the betweenness centrality, and solve the essential matrix by combining a two-stage weighted random sampling consensus algorithm with structural confidence weighted sampling and model optimization. The calculation module is used to generate a sparse point cloud by performing 3D reconstruction using the essential matrix and camera intrinsic parameters, recover the absolute scale of the sparse point cloud based on the known physical baseline length, register the sparse point cloud with the sleeve digital twin model to obtain the rigid transformation matrix, map the predefined key measurement points of the twin model to the point cloud coordinate system, use the mapped key measurement points as initial search values, determine the coordinates of the measured key measurement points in the neighborhood point cloud, calculate the size of the sleeve based on the measured key measurement point coordinates, and calculate the positioning uncertainty and confidence interval based on the neighborhood point cloud.

[0007] This invention calculates the structural confidence of feature points by using optical flow stability and structural tensor saliency, and constructs a cross-modal attention network by combining the geometric features of straight line segments and deep texture features to generate a fused descriptor. Mismatched points are identified based on a matching point map model, and the essential matrix is ​​solved using a two-stage weighted random sampling consensus algorithm guided by structural confidence, improving the stability of feature matching and 3D reconstruction. Furthermore, a sparse point cloud is generated and its absolute scale is restored. This cloud is then registered with a digital twin model, and the coordinates of the measured key points are determined using the model's key points as initial search values. Key dimensions, positioning uncertainty, and confidence intervals are then calculated. The overall solution helps improve the automation level, dimensional extraction accuracy, and result reliability of visual measurement of large skirt pile sleeves. Attached Figure Description

[0008] Figure 1 A flowchart of the first embodiment; Figure 2 This is a schematic diagram of the iterative convergence curve; Figure 3 A schematic diagram illustrating the relationship between the stability coefficient and the deviation; Figure 4 This is a schematic diagram comparing reprojection and measurement error. Detailed Implementation

[0009] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0010] In the first embodiment, the present invention proposes a visual automatic measurement method for the prefabricated dimensions of skirt pile sleeves, such as... Figure 1 As shown, it includes: S1. Acquire a multi-viewpoint 2D image sequence of the skirt pile sleeve, detect feature points and extract local descriptors and optical flow. Based on the minimum eigenvalue of the feature point neighborhood structure tensor matrix and the optical flow tracking stability, calculate the structure confidence of the feature points, extract the Hough transform line segments and deep texture feature maps of the 2D images, and encode the line segment features in the neighborhood of the feature points into query vectors. Use the texture feature maps as keys and values, construct a cross-modal attention network with feature points as bridges, generate a spatial prior distribution based on the structure confidence, and inject the spatial prior distribution into the attention matrix to generate a fusion descriptor for the feature points.

[0011] A multi-viewpoint 2D image sequence was acquired around the skirt pile sleeve using an industrial camera. The FAST corner detection algorithm was used to extract corner points as feature points, and the SIFT algorithm was used to calculate the local descriptor for each feature point. The Lucas-Kanade optical flow algorithm was used to calculate the optical flow vector of the feature points between two consecutive image frames, and the tracking trajectory of the feature points in multiple image frames was recorded to calculate the optical flow tracking stability. The Sobel operator was used to calculate the image gradient of the fixed pixel neighborhood of the feature points, constructing a 2D structure tensor matrix. The minimum eigenvalue of this matrix was solved using the singular value decomposition algorithm. The minimum eigenvalue and the optical flow tracking stability score were normalized and linearly added to obtain the structure confidence of the feature points.

[0012] The probabilistic Hough transform algorithm is used to extract line segments from the two-dimensional image. At the same time, the two-dimensional image is input into the pre-trained ResNet50 model to extract the feature map output by the fourth convolution block as the deep texture feature map.

[0013] For each feature point, the length and orientation angle of the line segments contained in the neighborhood are calculated. The geometric information is then concatenated and mapped into a high-dimensional query vector using a multilayer perceptron. The extracted deep texture feature map is flattened and linearly transformed through a fully connected layer to obtain the key vector and value vector.

[0014] A cross-modal attention network is constructed. The inner product of the query vector and the key vector is calculated to obtain the original attention matrix. Based on the structural confidence of each feature point and its corresponding spatial position in the deep texture feature map, a spatial confidence distribution map corresponding to the spatial sequence of the deep texture feature map is generated. This spatial confidence distribution map is flattened along the spatial dimension to form a spatial prior weight vector, which is used as the spatial prior bias matrix. Each element in the spatial prior bias matrix corresponds to a key position, and its value is assigned by the spatial confidence of the corresponding spatial position. The spatial prior bias matrix is ​​added element-wise to the initial cross-attention score matrix and then subjected to Softmax normalization to obtain the modulated attention weight matrix.

[0015] As one possible implementation, the detection of feature points and extraction of local descriptors and optical flow includes: A corner detection algorithm is used to extract the pixel coordinates of feature points in a two-dimensional image sequence; Constructing a multi-resolution image pyramid for an image sequence; Calculate the horizontal and vertical gray-level gradients of the neighborhood of feature points in each layer of the pyramid to generate local structure descriptors; By calculating the gray-level smoothing constraints and basic optical flow equations of local image regions between adjacent frames, and using Taylor expansion and least squares iterative solution, the optical flow vectors of each feature point in the image sequence are obtained.

[0016] Extract pixel coordinates from the preprocessed 2D image using either the fast corner detection algorithm or the Harris corner detection algorithm. , The preferred range for setting the corner quality threshold parameter is 0.01 to 0.05, for example, 0.02, to filter out low-quality feature points, so that the number of feature points extracted from each frame of image is preferably controlled within the range of 1000 to 3000.

[0017] In each layer of the pyramid, a neighborhood centered on the feature point with a preferred size of 16×16 or 32×32 pixels is extracted, and the horizontal gray-level gradient of each pixel is calculated using the Sobel operator. grayscale gradient in the vertical direction and according to and Calculate the gradient magnitude and gradient direction angle; divide the neighborhood into multiple sub-regions, perform statistics in each sub-region according to the interval to which the gradient direction angle belongs, and generate a local orientation histogram using the gradient magnitude as the voting weight; splice the local orientation histograms of multiple sub-regions in spatial order to generate a spatial structure descriptor vector, with the dimension set to 128 or 256.

[0018] The optical flow vector of feature points between frames is calculated based on the optical flow method. Assuming that the grayscale of neighboring pixels remains constant during the time interval between two adjacent frames, the basic equation of optical flow is established. ,in This represents the grayscale gradient change of the corresponding pixel over time. A first-order Taylor expansion is used to linearize the objective function, establishing an overdetermined system of linear equations within a d×d neighborhood centered on the feature point with an optimal window size of 21×21 or 31×31. The optical flow velocity vector of the feature point is then solved. , The least squares method is applied to iteratively solve the pseudo-inverse matrix of the overdetermined equation. The maximum number of iterations is preferably set to 10 to 30, for example, 15. The displacement convergence accuracy threshold ε is set to 0.01 to 0.05 pixels. Iteration stops and the result is output when the magnitude of the velocity-displacement increment calculated in two consecutive iterations is less than 0.03 pixels, thus obtaining the optical flow vector of each feature point in the image sequence. The iterative convergence curve is shown below. Figure 2As shown.

[0019] As one possible implementation, the calculation of the structural confidence of a feature point based on the minimum eigenvalue of the feature point's neighborhood structure tensor matrix and optical flow tracking stability includes: A local neighborhood window is established with each feature point as the center. The horizontal and vertical gradients of the pixels within the window are calculated, and a second-order structure tensor matrix is ​​constructed. The structural tensor matrix is ​​subjected to eigenvalue decomposition, and the minimum eigenvalue is extracted as the structural saliency factor. Track the optical flow vectors of feature points in multiple consecutive frames of images, calculate the Euclidean distance deviation between the actual tracking trajectory and the theoretical smooth trajectory, and convert it into an optical flow tracking stability coefficient; The structural saliency factor is normalized, and the normalized structural saliency factor is linearly weighted and fused with the optical flow tracking stability coefficient to obtain the structural confidence of the feature point.

[0020] Centered on the coordinates of each detected feature point, a local neighborhood window with a preferred size of 9×9 to 15×15 pixels, such as 11×11 pixels, is constructed. The horizontal gradient of all pixels within the window is calculated using a difference operator. with vertical gradient By calculating the gradient outer product matrix and applying a two-dimensional Gaussian weighted kernel with a standard deviation of σ, preferably in the range of 1.0 to 2.0, such as 1.5, for spatial smoothing, a second-order structure tensor matrix M with a size of 2×2 is constructed.

[0021] Perform eigenvalue decomposition on the structure tensor matrix M to extract the minimum eigenvalue among the two eigenvalues. This serves as the structural saliency factor for the feature point. The value of this factor ranges from 0.01 to 100.0. When its value is greater than a set threshold, such as 0.05, it indicates that the pixel has texture variations in multiple directions within its local neighborhood, and serves as a matching reference point.

[0022] In an image sequence with a time span N preferably of 5 to 10 frames, such as 7 consecutive frames, the position of feature points is tracked through the aforementioned optical flow vector iteration. The temporal discrete feature point coordinate sequence output by the optical flow method is fitted into a cubic polynomial curve using the least squares method, and this curve is set as the theoretical smooth trajectory of the moving target.

[0023] Calculate the Euclidean distance deviation between the actual feature point tracking coordinates and the corresponding theoretical smooth trajectory points in each frame. Take the total average distance deviation of N frames and divide the total average distance deviation by a preset pixel scale factor. The dimensionless average deviation is obtained. ,in The preset pixel scale factor is preferably set to 1 to 10 pixels, for example, 5 pixels; and an exponential decay function is used. This is converted to an optical flow tracking stability coefficient S between 0 and 1, where α is a dimensionless attenuation hyperparameter, preferably ranging from 0.5 to 2.0, with an example value of 1.2. The relationship between the tracking stability coefficient and the deviation is as follows: Figure 3 As shown.

[0024] Structural significance factor Normalized to the interval of 0 to 1 to obtain Through linear combination formula The overall structural confidence C of the feature points is calculated, and the weighting parameter β is preferably set to a range of 0.4 to 0.6, for example, 0.5, to ensure that the structure and tracking attributes are balanced in the confidence assessment.

[0025] As one possible implementation, the step of extracting the Hough transform line segments and deep texture feature maps of the two-dimensional image, and encoding the line segment features in the neighborhood of the feature points into a query vector, with the texture feature map serving as the key and value, includes: The endpoint coordinates, length, and orientation angle of the Hough transform line segments in the two-dimensional image are extracted to construct geometric primitive feature vectors. The geometric primitive feature vectors in the neighborhood of each feature point are aggregated and mapped into a query vector sequence of a specified channel dimension through a multilayer perceptron. Multi-scale feature extraction of two-dimensional images is performed by pre-trained convolutional neural networks. The feature maps output by the deep layers of the network are flattened into two-dimensional spatial sequences and generated as key and value matrices through linear projection, providing global texture context information for cross-modal attention fusion networks.

[0026] Edge detection is performed on the input 2D full-size image. The ratio of high to low thresholds is preferably set to 1:3, for example, the low threshold is set to 50 and the high threshold is set to 150. A probabilistic Hough transform is applied to extract continuous edge line segments. The minimum effective line length is preferably set to 20 to 50 pixels, such as 30 pixels, and the maximum allowable break gap parameter is set to 5 to 10 pixels, such as 8 pixels.

[0027] For each extracted line segment, the image coordinates of its two endpoints are obtained through parsing. , and , The length L of the line segment is calculated using Euclidean distance, and the direction angle θ is calculated using the arctangent function based on the coordinate difference between the two endpoints. The above values ​​are concatenated and spliced ​​to construct a geometric primitive feature vector.

[0028] In the feature extraction stage, a circular convergence neighborhood with a search radius R of 30 to 60 pixels, such as 45 pixels, is preferably searched centered on each feature point. All geometric primitive feature vectors falling within this neighborhood are retrieved and combined.

[0029] Since the number of line segments in the neighborhood of different feature points may vary, to obtain a fixed-dimensional input, multiple geometric primitive feature vectors falling within the neighborhood of the same feature point are sorted in descending order of line segment length, and the first K geometric primitive feature vectors are selected. When the number is less than K, zero vectors are used to pad to K. The K geometric primitive feature vectors are concatenated sequentially to obtain a fixed-dimensional geometric convergence vector. A multilayer perceptron network is constructed, with the fixed-dimensional geometric convergence vector as its input. The structure includes three hidden layers: the first hidden layer has 64 neurons, the second hidden layer has 128 neurons, and the third hidden layer has 256 neurons. Each hidden layer is followed by a modified linear unit activation function. The output of the multilayer perceptron network is a query vector sequence with a fixed channel dimension D of 256.

[0030] The fixed-dimensional geometric convergence vector is input into the multilayer perceptron network to generate the query vector sequence. Simultaneously, a pre-trained convolutional neural network is used, with the same full-size 2D image as input. The network structure is a 50-layer residual network model pre-trained on the ImageNet dataset, containing an initial 7×7 convolutional layer and a max-pooling layer, followed by four concatenated residual block stages with bottleneck structures. Feature fusion is achieved within each residual block through skip connections. The 3D tensor output from the last residual convolutional block of the network, i.e., the deep texture feature map, is extracted. This full-size 2D image is then input into the convolutional neural network for forward inference to perform multi-scale global feature extraction, outputting a deep texture feature map. At this point, the spatial downsampling factor of the feature map is 1 / 32, and the channel dimension... The value is 2048. Flatten this three-dimensional tensor of size H'×W'×2048 along the spatial dimensions to convert it into length. =H'×W' two-dimensional spatial feature sequence.

[0031] By using two independent linear fully connected projection layers, ensuring that the weights of the two layers are not shared, the 2D sequence is reduced by projection. The target dimension of the projection is set to D=256, generating shapes as follows: The key and value matrices are given. This projection step converts the convolutional features into a modal format processed by the self-attention mechanism.

[0032] As one possible implementation, the step of generating a spatial prior distribution based on the structural confidence and injecting the spatial prior distribution into the attention matrix to generate a fusion descriptor for the feature points includes: Calculate the inner product of the query vector and the key matrix to obtain the initial cross-attention score matrix; Based on the structural confidence of each feature point and its corresponding receptive field position in the deep texture feature map, a spatial confidence distribution map corresponding to the spatial sequence of the deep texture feature map is generated. The spatial confidence distribution map is flattened into a spatial prior weight vector corresponding to the spatial sequence of the key matrix, and then expanded into a spatial prior bias matrix with the same dimension as the initial cross-attention score matrix through a tensor broadcasting mechanism. The spatial prior bias values ​​corresponding to different key positions are determined according to their spatial confidence. The spatial prior bias matrix is ​​added element-wise to the initial cross-attention score matrix, and a normalized exponential function is applied to obtain the modulated attention weights. The value matrix is ​​weighted and summed using the modulated attention weights, and then concatenated with the original local structure descriptors of the feature points through residual connections to generate a fused descriptor for the feature points.

[0033] Extract the query vector generated above with a size of 1×D, where D is preferably 256, and combine it with a size of... The query vector is multiplied by the transpose of the key matrix. The result of the multiplication is then divided by the scaling factor. In this embodiment, it is 16, thus obtaining a size of The initial cross-attention score matrix.

[0034] Based on the aforementioned spatial downsampling ratio from the image space to the deep feature map of the network, such as 1 / 32, the original image pixel coordinates of each feature point are proportionally converted to obtain the corresponding receptive field index of each feature point in the deep texture feature map. According to the structural confidence of each feature point and its corresponding receptive field index, a spatial confidence distribution map is generated in the deep texture feature map space. For positions that do not directly correspond to feature points, nearest neighbor interpolation is used to assign values, ensuring that the spatial confidence distribution map has the same spatial size as the deep texture feature map.

[0035] Flatten the spatial confidence distribution map along the spatial dimension to a length of [length missing]. The spatial prior weight vector is used as a 1× A spatial prior bias matrix is ​​generated; each element in the spatial prior bias matrix corresponds to a key position, and its value is assigned by the spatial confidence of the corresponding spatial position. This spatial prior bias matrix is ​​then element-wise added to the initial cross-attention score matrix and subjected to Softmax normalization to obtain an output size of 1× The modulated attention weight matrix.

[0036] The attention weight matrix and the size of The ×D value matrix performs matrix multiplication to achieve weighted summation of features across global pixel dimensions, outputting a 1×256-dimensional cross-modal attention fusion feature vector.

[0037] The fused feature vector is concatenated with the original local structure descriptor of the feature point using a skip residual connection and channel concatenation mechanism. When the local structure descriptor is 128-dimensional, the concatenation results in a 384-dimensional feature. When the local structure descriptor uses other preset dimensions, the concatenation is completed according to the corresponding number of channels, and the dimension is reduced to 256-dimensional through a single-layer fully connected layer with layer regularization to generate a feature fusion descriptor with stable representation capabilities in the texture, geometry, and spatial domains.

[0038] S2, based on the fusion descriptor, feature matching is performed to obtain initial matching point pairs, a matching point graph model is constructed, the intermediary centrality of the nodes is calculated, and mismatched points are identified and removed based on the intermediary centrality. Combining the two-stage weighted random sampling consensus algorithm, the essential matrix is ​​solved by weighted sampling based on structural confidence and model optimization.

[0039] The K-nearest neighbor matching algorithm is executed. For each feature point in the first image, a fused descriptor is created. Then, in the fused descriptor set of the second image, the nearest first candidate point and the second nearest second candidate point are searched, and their corresponding Euclidean distances are calculated. and When the distance ratio / Less than the preset ratio threshold When the first candidate point is matched with the current feature point, an initial matching point pair is formed; where, The preferred value is between 0.70 and 0.85, for example, 0.75.

[0040] Each initial matching point pair is abstracted as a node in the graph model, and its geometric correlation is judged based on the local spatial distance relationship and length ratio consistency of any two nodes in two images. When the preset spatial neighborhood threshold and preset geometric consistency conditions are met, an adjacent edge is added between the corresponding two nodes, thereby constructing a matching point graph model to represent the local spatial consistency of the initial matching point pair.

[0041] The Brandes algorithm is used to calculate the betweenness centrality of each node in the graph model, which is calculated by counting the proportion of times the shortest path between all pairs of nodes passes through that node. Anomaly centrality filtering criteria are set, and mismatched nodes are identified based on the relationship between node betweenness centrality and these criteria. These mismatched nodes and their associated edges are then removed from the matching vertex graph model, retaining only the set of matching vertices that meet the topological consistency requirements.

[0042] A two-stage weighted random sampling consensus algorithm is executed. In the first stage, hypothesis generation, the calculated structural confidence is converted into sampling probabilities. Point pairs with higher structural confidence have a greater probability of being selected. Based on this probability distribution, five matching points are randomly selected. The candidate essence matrix is ​​then calculated using the five-point method combined with the selected five matching points.

[0043] In the second-stage model optimization step, the symmetric epipolar distance from all retained matching point pairs to the epipolar line corresponding to the candidate essential matrix is ​​calculated, and the number of inliers whose distance is less than the epipolar distance threshold is counted. This weighted sampling and evaluation process is iterated until the maximum number of iterations is reached, and the candidate essential matrix containing the most inliers is selected as the optimal initial model. The Levenberg-Marquardt algorithm is then executed using all inliers to minimize the weighted symmetric epipolar distance or the weighted Sampson distance to solve for the essential matrix.

[0044] As one possible implementation, the construction of the matching point graph model, the calculation of the betweenness centrality of nodes, and the identification and removal of mismatched points based on the betweenness centrality, combined with a two-stage weighted random sampling consensus algorithm, and the solution of the essential matrix based on structural confidence weighted sampling and model optimization, includes: The initial matching point pairs are used as graph nodes, and adjacent edges are constructed based on the spatial geometric consistency between the point pairs to generate a matching point graph model. Calculate the betweenness centrality of each node in the matching point graph model, calculate the mean and standard deviation of the betweenness centrality of each node, and determine the abnormal centrality threshold range based on the mean and standard deviation. Nodes whose betweenness centrality is outside the abnormal centrality threshold range are identified as mismatched nodes, and the mismatched nodes and their associated edges are removed from the matching point graph model. In the first stage, the sampling probability is calculated based on the sum of the structural confidence of the remaining matching points to the feature points at both ends, and the roulette wheel algorithm is used to extract a subset of samples for estimating the essential matrix. In the second stage, after calculating the candidate essential matrix using a subset of samples, the mean of the structural confidence of the feature points at both ends of the matching pairs corresponding to all interior points is used as a weight to be assigned to the corresponding residuals. The candidate essential matrix is ​​then nonlinearly optimized by minimizing the weighted epipolar geometric error.

[0045] A set of initial matching feature point pairs is obtained between the source and target images using nearest neighbor search and distance ratio testing algorithms, and these pairs are set as the node set V of an undirected graph data structure. Any two sets of matching point pairs are randomly selected, and the Euclidean length ratio of the corresponding connecting lines between the two views and the change in the angle between the line segments are calculated. If the line length ratio is within the preferred tolerance threshold range of 0.8 to 1.25, and the absolute value of the change in the angle is less than the preferred angle threshold of 15 degrees (approximately 0.26 radians), it is determined that the local spatial geometric changes of the two pairs of matching points have topological consistency. An undirected adjacent edge is then constructed between the corresponding two nodes, generating the overall matching point graph model G(V,E).

[0046] Run Brandes' algorithm on the graph model to calculate the betweenness centrality of each node in the network via the shortest path. Calculate the mean betweenness centrality of all nodes globally. and standard deviation Set an abnormal threshold and lower abnormal threshold The preferred range for parameter k is 1.5 to 2.5, for example, 2.0. When the betweenness centrality of a node is higher than... or below When a node is identified as a topologically anomalous, it and its associated edges are removed from the graph model. This method can eliminate bridging anomalous matches or isolated anomalous matches, retaining a set of matching points with more consistent topological relationships.

[0047] In the first stage of the weighted random sampling consensus algorithm, for the candidate matching point set retained after cleaning by the matching point graph model, the original structural confidence of each matching point with the independent feature points at its left and right ends is extracted. and By calculating the sum of confidence levels The scores used to quantify and set the corresponding sampling probabilities are based on these scores.

[0048] Based on the overall confidence level of the matching point pairs As a sampling evaluation function, five pairs of matching points are randomly selected from the candidate matching point pair set as the minimum sample subset by roulette wheel selection, and the candidate essential matrix E is estimated by the five-point method.

[0049] During the model validation phase, the corresponding fundamental matrix F is calculated based on the candidate essential matrix E and the camera intrinsic parameter matrix, and the epipolar geometric error of the remaining candidate matching point pairs relative to the fundamental matrix F is calculated. The epipolar geometric error is calculated based on the epipolar constraint relationship between the matching point pairs and the fundamental matrix F, preferably using symmetric epipolar distance or Sampson distance. When the epipolar geometric error is less than a preset error tolerance threshold... When this happens, the corresponding matching point pair is added to the consensus internal point set; where, The preferred value is 1.5 to 3.0 pixels, for example, 2.0 pixels.

[0050] During the model optimization phase, for each pair of matching points in the consensus intrapoint set, the arithmetic mean of the structural confidence of the matching point with respect to the feature points at both ends is extracted. and will As the optimization weights for this matching point pair, construct a weighted objective cost function. ,in, The number of matching point pairs within the consensus point set. Let be the optimized weight of the i-th pair of matching points. Let E be the epipolar geometric error of the i-th pair of matching points, where the epipolar geometric error is either the symmetric epipolar distance or the Sampson distance. The objective cost function is iteratively optimized using the Levenberg-Marquardt nonlinear least squares algorithm. The maximum number of iterations is preferably set to 100, and the initial value of the damping factor is preferably 0.001. During the iteration process, the damping factor is adaptively adjusted based on the error reduction, thereby obtaining the optimized essential matrix E.

[0051] S3. Using the essential matrix and camera intrinsic parameters, 3D reconstruction is performed to generate a sparse point cloud. The absolute scale of the sparse point cloud is recovered based on the known physical baseline length. The sparse point cloud is registered with the sleeve digital twin model to obtain the rigid transformation matrix. The predefined key measurement points of the twin model are mapped to the point cloud coordinate system. The mapped key measurement points are used as the initial search values ​​to determine the coordinates of the measured key measurement points in the neighborhood point cloud. The size of the sleeve is calculated based on the coordinates of the measured key measurement points. The positioning uncertainty and confidence interval are calculated based on the neighborhood point cloud.

[0052] For candidate image pairs with overlapping fields of view in a multi-view 2D image sequence, the corresponding essential matrices are solved. Singular value decomposition is performed on the essential matrix of each candidate image pair to obtain the candidate relative rotation matrix and the translation direction with undetermined scale. The initial relative extrinsic values ​​of the corresponding candidate image pairs are obtained by chiral constraint screening. Then, based on the initial relative extrinsic values ​​between each candidate image pair and the effective matching point pairs, the relative poses are combined sequentially along the connection relationship between the candidate image pairs to unify the extrinsic parameters of each viewpoint camera to the same global coordinate system. The camera extrinsic parameters and 3D point coordinates are then globally optimized to obtain the global projection matrix of each viewpoint camera.

[0053] Using a 3D spatial point triangulation algorithm, combined with a pre-calibrated camera intrinsic parameter matrix and selected camera extrinsic parameters, the 3D coordinates of the set of interior points in 3D space are calculated to generate a sparse point cloud.

[0054] At the shooting location, the true Euclidean distance between two preset physical markers is measured using a laser rangefinder as the known physical baseline length. The 3D Euclidean distance between the corresponding two markers in the sparse point cloud is calculated. The true Euclidean distance is divided by the calculated Euclidean distance to obtain the absolute scale factor. The coordinates of all 3D points in the sparse point cloud are multiplied by this absolute scale factor to complete the restoration of the absolute scale of the sparse point cloud.

[0055] The 3D point cloud of a sleeve digital twin model constructed using computer-aided design software is loaded. The Fast Feature Histogram (FPFH) algorithm from the PCL point cloud library is used to extract point cloud features, and a sample consistency initial registration algorithm is executed to complete the coarse registration between the sparse point cloud and the sleeve digital twin model. The Iterative Closest Point (ICP) algorithm is then used for registration, minimizing the sum of squared Euclidean distances between corresponding points, and outputting the rigid transformation matrix after registration.

[0056] The rigid transformation matrix is ​​used to transform the coordinate system of the predefined key measurement points in the twin model. The transformation is then multiplied by the rotation matrix and a translation vector is added to map the points to the reconstructed 3D point cloud coordinate system.

[0057] A spherical search neighborhood is set around the mapped key measurement points. The mapped key measurement points are used as the initial values ​​for the neighborhood search. The actual reconstructed points in the neighborhood are retrieved, and the least squares method is used to perform local plane fitting on the actual reconstructed points. The mapped key measurement points are projected onto the fitting plane along the normal direction of the local plane, and the spatial coordinates after projection are determined as the coordinates of the corresponding measured key measurement points.

[0058] The standard deviation of the statistical deviation distribution is used as the positioning uncertainty of the key measurement point based on the vertical distance from the actual reconstructed point in the neighborhood to the local fitting plane; or, when calculating the positioning uncertainty using the eigenvalues ​​of the neighborhood point cloud covariance matrix, the square root of the sum of the three eigenvalues ​​of the covariance matrix is ​​used as the positioning uncertainty to ensure that its dimension is the length unit.

[0059] The diameter, height, or other key dimensions of the skirt pile sleeve are calculated based on the measured key measurement point coordinates. The calculated positioning uncertainty is multiplied by the quantile coefficient set based on the standard normal distribution to obtain the boundary tolerance. The boundary tolerance is added or subtracted with the measured key dimensions as the center to obtain the confidence interval of the key dimensions of the sleeve.

[0060] As one possible implementation, the method of generating sparse point clouds through 3D reconstruction using the essential matrix and camera intrinsic parameters includes: For candidate image pairs with overlapping fields of view in a multi-view 2D image sequence, solve for the corresponding essential matrix respectively; Singular value decomposition is performed on the essential matrix, and the relative rotation matrix and translation vector with undetermined scale between the corresponding views are solved by combining multi-view geometric constraints. Based on the relative rotation matrix obtained between each candidate image pair, the translation direction with undetermined scale, and the effective matching point pairs, the extrinsic parameters of each viewpoint camera are unified to the same global coordinate system, and the camera extrinsic parameters and 3D point coordinates are globally optimized to obtain the global projection matrix of each viewpoint camera. Based on the two-dimensional pixel coordinates of the matching feature points and the corresponding global projection matrix, the three-dimensional coordinates of the corresponding three-dimensional spatial points are solved by the triangulation method. Triangulation calculations are performed on all valid matching points, and spatial points with reprojection errors greater than a set threshold are filtered out. The points are then fused to generate an initial sparse point cloud. The absolute scale is then restored by combining the known physical baseline length to generate the sparse point cloud of the skirt pile sleeve.

[0061] For any two images in a multi-view 2D image sequence, perform coarse matching statistics. If the number of valid initial matching points between the two images is not less than a preset threshold for the number of shared matching points, then the matching is successful. When the distribution range of the effective initial matching points in the image plane meets the preset coverage condition, the two images are determined as a candidate image pair with overlapping fields of view; wherein, The preferred value is between 30 and 80, for example, 50. Subsequently, for each candidate image pair, the corresponding essence matrix E is solved based on the effective matching points in the candidate image pair.

[0062] Perform singular value decomposition on the obtained 3×3 essential matrix E, and the decomposition form is as follows: Where Σ=diag(σ,σ,0). Two sets of candidate rotation matrices are calculated using the orthogonal auxiliary matrix W. and ; the third column vector of matrix U As translation directions with undetermined scale, two sets of candidate translation directions are obtained. and .Depend on , , and The combinations form four sets of candidate relative poses.

[0063] The four sets of candidate relative poses are filtered using chiral constraints. Specifically, some consensus point pairs in the candidate image pairs are selected and triangulated under the four sets of candidate relative poses, and the depth values ​​of the 3D points in the two camera coordinate systems are counted. The set of candidate relative poses that makes most of the 3D points have positive depth in both camera coordinate systems is determined as the relative rotation matrix R and the translation direction vector t with undetermined scale for the candidate image pair.

[0064] The camera coordinate system for the first frame is set to the world coordinate system, and the calibrated camera intrinsic parameter matrix K is loaded. The camera intrinsic parameter matrix K includes focal length parameters. , and principal point coordinate parameters , For local reconstruction of two views, the camera projection matrices corresponding to the two images are represented as follows: and .

[0065] For multi-viewpoint global reconstruction, the camera coordinate system corresponding to the first frame image is selected as the global coordinate system, and the camera extrinsic parameters of the first frame are initialized as a unit rotation matrix and a zero translation vector. Then, based on the relative rotation matrix obtained between each candidate image pair, the translation direction with undetermined scale, and the effective matching point pairs, the relative poses are sequentially combined along the connection relationship of the candidate image pairs to obtain the initial camera extrinsic parameters of each viewpoint in the same global coordinate system. The global projection matrix corresponding to each viewpoint is obtained through global optimization.

[0066] For each pair of matching feature points in the input set after topological cleaning and random sampling consistency test, the image pixel coordinates are... , and , According to the projection matrix of the corresponding camera and Using the linear transformation method, a system of equations is established with respect to the three-dimensional homogeneous coordinates. The homogeneous linear equation system AX=0 is obtained. The optimal least squares solution under the condition of minimum modulus of AX is obtained by using singular value decomposition, thereby deriving and generating the three-dimensional geometric coordinates of each corresponding spatial object.

[0067] The calculated 3D points are then reprojected back to the image plane via an intrinsic parameter projection matrix to calculate the estimated pixel positions. The planar Euclidean error is then calculated between the original detected feature points and the reprojection operation. A maximum allowable reprojection error removal threshold is preferably set to a distance of 0.5 to 1.5 pixels, for example, 1.0 pixel. Singularities with reprojection errors exceeding this threshold are filtered out. The accumulated data are then stitched together to obtain a local 3D sparse point cloud with a dimensionless scale.

[0068] Read the physical baseline distance between at least one pair of spatially fixed points on site. For example, 5000.0 mm, to calculate the spatial distance between two specific points in the initial point cloud model. Find the normalized scale multiplier For example, if the scaling factor s = 10.5 is obtained, the x, y, and z coordinate parameters of all three-dimensional points are multiplied by the multiplier s to restore the three-dimensional reconstructed point cloud model to the real spatial scale.

[0069] As one possible implementation, the process of using the mapped key measurement points as initial search values, determining the coordinates of the measured key measurement points in the neighborhood point cloud, calculating the sleeve dimensions based on the measured key measurement point coordinates, and calculating the positioning uncertainty and confidence interval based on the neighborhood point cloud includes: For each key measurement point mapped to the point cloud coordinate system that has recovered absolute scale, the key measurement point is used as the initial search value, and the actual reconstructed point in the sparse point cloud is searched within a set radius using a Kd tree. Based on the actual reconstructed points obtained from the search, local plane fitting is performed to determine the coordinates of the measured key measurement points corresponding to the key measurement points; Calculate the deviation between the actual reconstructed point and the locally fitted geometric model, and statistically analyze the distribution parameters of the deviation as the positioning uncertainty of the measured key measurement point; Select key measurement points on a specific structure of the skirt pile sleeve and calculate the absolute spatial distance as the key dimension. The positioning uncertainties of the measured key measurement points are combined using the error propagation formula to calculate the upper and lower bounds of the confidence interval of the corresponding key dimensions at the preset probability level.

[0070] For the three-dimensional measurement points located at the center of the flange mounting hole or the apex of the outer wall of the steel cylinder, which are predefined a priori in the computer-aided design documents and digital twin model. Align its spatial reference and translate it into the absolute scale point cloud coordinate system. It is used only as the initial value for the neighborhood search of the key measurement point, and not directly as the final size calculation point. The Kd tree spatial retrieval algorithm is used to search for a group of points within a set expected radius r around this virtual theoretical point. The preferred search distance radius is 5.0 to 20.0 mm, for example, 10.0 mm.

[0071] Obtain all N actual reconstructed points falling within a spherical region. The lower bound of N is preferably set to 30 to ensure statistical consistency. Based on the mapping position of the key measurement point in the point cloud coordinate system, extract actual reconstructed points within its neighborhood. Use the least squares method to perform local plane fitting on the actual reconstructed points within the neighborhood. Project the mapping position of the key measurement point along the normal direction of the local plane onto the fitting plane, and determine the projected spatial coordinates as the corresponding measured key measurement point coordinates. For each of the N actual reconstructed points in the neighborhood, the vertical distance from it to the local fitting plane is calculated to obtain the deviation sequence. Calculate the standard deviation of the deviation sequence. and will The positioning uncertainty of the key measurement point in this experiment The typical magnitude range is 0.5 to 2.5 mm, for example, the positioning uncertainty is 1.2 mm.

[0072] When calculating critical dimensions, such as measuring the horizontal centerline distance between the two guide frame flange interfaces or the upper and lower thicknesses of the anti-sinking plate, a pair of specified critical measurement positioning point numbers or theoretical initial values ​​are read from the model configuration table. and . respectively and To search for initial values, determine the corresponding coordinates of key measured points within the absolute-scale point cloud coordinate system. and Based on the measured coordinates of the key measurement points, the absolute line segment distance between the two points is obtained using spatial analytical geometry calculations. Use this distance result as the measurement value of the critical dimension. For example, the vertical installation distance between the outer sides of the two flange faces is calculated to be 2500.5mm.

[0073] Extract the distribution uncertainty parameters of each independently acquired key measured point. and Parameter merging is performed using the first-order linear error propagation algorithm of random variables expanded by Taylor series. The measurement variance is obtained according to the variance synthesis identity. After square root conversion, the combined standard uncertainty coefficient of the critical dimension L of this component was calculated. .

[0074] Set a preset probability level, such as a 95% or 99% confidence level. Taking a 95% normality confidence level as an example, the corresponding extended coverage factor constant k is set to 1.96, deriving the upper bound of the quality boundary. and lower limit of safety margin boundary Based on this, the confidence interval for the critical size at the preset probability level is obtained.

[0075] The experiment used fifty sets of multi-view image sequences of skirt pile sleeves acquired by a high-resolution industrial camera in actual production as the test dataset. The experimental hardware platform was equipped with a high-performance graphics processor and a 64-core CPU. The experiment consisted of four groups: the baseline algorithm group, ablation group one utilizing cross-modal attention fusion features, ablation group two adding graph model centrality removal and weighted random sampling consistency optimization to ablation group one, and the complete scheme group including all improved modules and uncertainty assessment. Evaluation metrics covered feature matching accuracy, average reprojection error of sparse point cloud, and absolute measurement error of key dimensions.

[0076] The baseline algorithm group achieved a feature matching accuracy of 65.4%, an average reprojection error of 1.85 pixels, and a critical dimension absolute measurement error of 8.5 mm. Ablation group one combined Hough line segment geometric features with deep texture features from convolutional networks, improving the feature matching accuracy to 78.2%, reducing the average reprojection error to 1.21 pixels, and decreasing the dimension error to 5.2 mm. Ablation group two removed topological anomalies by using graph node betweenness centrality and solved the essential matrix weighted according to structural confidence, improving the feature matching accuracy to 85.6%, reducing the average reprojection error to 0.85 pixels, and decreasing the dimension error to 3.4 mm. The complete solution group achieved a feature matching accuracy of 94.3%, reduced the average reprojection error to 0.42 pixels, and had a critical dimension absolute measurement error of 1.2 mm. The comparison results of sparse point cloud reprojection error and critical dimension measurement error for each solution are as follows: Figure 4 As shown.

[0077] Cross-modal attention networks can alleviate feature confusion caused by the lack of local texture on the metal surface of the skirt pile sleeve, enabling the descriptor to simultaneously integrate local geometric information and global texture context information, thereby improving the stability of initial feature matching. Graph model topological constraints and a two-stage weighted nonlinear optimization mechanism guide the matching point sampling and residual optimization process using structural confidence, reducing the impact of some topological anomalies on camera pose and essential matrix solution. Combined with key point localization uncertainty analysis and error propagation methods, the system can provide corresponding confidence intervals while outputting key dimension measurements, thereby improving the interpretability of visual measurement results for large marine components and the reliability of engineering applications.

[0078] In a second embodiment, the present invention also proposes a visual automatic measurement system for the prefabricated dimensions of skirt pile sleeves, comprising: The generation module is used to acquire multi-viewpoint two-dimensional image sequences of the skirt pile sleeve, detect feature points and extract local descriptors and optical flow, calculate the structural confidence of feature points based on the minimum eigenvalue of the feature point neighborhood structure tensor matrix and the optical flow tracking stability, extract Hough transform line segments and deep texture feature maps of the two-dimensional images, encode the line segment features in the neighborhood of the feature points into query vectors, use the texture feature maps as keys and values, construct a cross-modal attention network with feature points as bridges, generate spatial prior distributions based on the structural confidence, inject the spatial prior distributions into the attention matrix, and generate fusion descriptors for feature points. The solution module is used to obtain initial matching point pairs based on feature matching of fused descriptors, construct a matching point graph model, calculate the betweenness centrality of nodes, identify and remove mismatched points based on the betweenness centrality, and solve the essential matrix by combining a two-stage weighted random sampling consensus algorithm with structural confidence weighted sampling and model optimization. The calculation module is used to generate a sparse point cloud by performing 3D reconstruction using the essential matrix and camera intrinsic parameters, recover the absolute scale of the sparse point cloud based on the known physical baseline length, register the sparse point cloud with the sleeve digital twin model to obtain the rigid transformation matrix, map the predefined key measurement points of the twin model to the point cloud coordinate system, use the mapped key measurement points as initial search values, determine the coordinates of the measured key measurement points in the neighborhood point cloud, calculate the size of the sleeve based on the measured key measurement point coordinates, and calculate the positioning uncertainty and confidence interval based on the neighborhood point cloud.

[0079] As one possible implementation, the detection of feature points and extraction of local descriptors and optical flow includes: A corner detection algorithm is used to extract the pixel coordinates of feature points in a two-dimensional image sequence; Constructing a multi-resolution image pyramid for an image sequence; Calculate the horizontal and vertical gray-level gradients of the neighborhood of feature points in each layer of the pyramid to generate local structure descriptors; By calculating the gray-level smoothing constraints and basic optical flow equations of local image regions between adjacent frames, and using Taylor expansion and least squares iterative solution, the optical flow vectors of each feature point in the image sequence are obtained.

[0080] As one possible implementation, the calculation of the structural confidence of a feature point based on the minimum eigenvalue of the feature point's neighborhood structure tensor matrix and optical flow tracking stability includes: A local neighborhood window is established with each feature point as the center. The horizontal and vertical gradients of the pixels within the window are calculated, and a second-order structure tensor matrix is ​​constructed. The structural tensor matrix is ​​subjected to eigenvalue decomposition, and the minimum eigenvalue is extracted as the structural saliency factor. Track the optical flow vectors of feature points in multiple consecutive frames of images, calculate the Euclidean distance deviation between the actual tracking trajectory and the theoretical smooth trajectory, and convert it into an optical flow tracking stability coefficient; The structural saliency factor is normalized, and the normalized structural saliency factor is linearly weighted and fused with the optical flow tracking stability coefficient to obtain the structural confidence of the feature point.

[0081] As one possible implementation, the step of extracting the Hough transform line segments and deep texture feature maps of the two-dimensional image, and encoding the line segment features in the neighborhood of the feature points into a query vector, with the texture feature map serving as the key and value, includes: The endpoint coordinates, length, and orientation angle of the Hough transform line segments in the two-dimensional image are extracted to construct geometric primitive feature vectors. The geometric primitive feature vectors in the neighborhood of each feature point are aggregated and mapped into a query vector sequence of a specified channel dimension through a multilayer perceptron. Multi-scale feature extraction of two-dimensional images is performed by pre-trained convolutional neural networks. The feature maps output by the deep layers of the network are flattened into two-dimensional spatial sequences and generated as key and value matrices through linear projection, providing global texture context information for cross-modal attention fusion networks.

[0082] As one possible implementation, the step of generating a spatial prior distribution based on the structural confidence and injecting the spatial prior distribution into the attention matrix to generate a fusion descriptor for the feature points includes: Calculate the inner product of the query vector and the key matrix to obtain the initial cross-attention score matrix; Based on the structural confidence of each feature point and its corresponding receptive field position in the deep texture feature map, a spatial confidence distribution map corresponding to the spatial sequence of the deep texture feature map is generated. The spatial confidence distribution map is flattened into a spatial prior weight vector corresponding to the spatial sequence of the key matrix, and then expanded into a spatial prior bias matrix with the same dimension as the initial cross-attention score matrix through a tensor broadcasting mechanism. The spatial prior bias values ​​corresponding to different key positions are determined according to their spatial confidence. The spatial prior bias matrix is ​​added element-wise to the initial cross-attention score matrix, and a normalized exponential function is applied to obtain the modulated attention weights. The value matrix is ​​weighted and summed using the modulated attention weights, and then concatenated with the original local structure descriptors of the feature points through residual connections to generate a fused descriptor for the feature points.

[0083] As one possible implementation, the construction of the matching point graph model, the calculation of the betweenness centrality of nodes, and the identification and removal of mismatched points based on the betweenness centrality, combined with a two-stage weighted random sampling consensus algorithm, and the solution of the essential matrix based on structural confidence weighted sampling and model optimization, includes: The initial matching point pairs are used as graph nodes, and adjacent edges are constructed based on the spatial geometric consistency between the point pairs to generate a matching point graph model. Calculate the betweenness centrality of each node in the matching point graph model, calculate the mean and standard deviation of the betweenness centrality of each node, and determine the abnormal centrality threshold range based on the mean and standard deviation. Nodes whose betweenness centrality is outside the abnormal centrality threshold range are identified as mismatched nodes, and the mismatched nodes and their associated edges are removed from the matching point graph model. In the first stage, the sampling probability is calculated based on the sum of the structural confidence of the remaining matching points to the feature points at both ends, and the roulette wheel algorithm is used to extract a subset of samples for estimating the essential matrix. In the second stage, after calculating the candidate essential matrix using a subset of samples, the mean of the structural confidence of the feature points at both ends of the matching pairs corresponding to all interior points is used as a weight to be assigned to the corresponding residuals. The candidate essential matrix is ​​then nonlinearly optimized by minimizing the weighted epipolar geometric error.

[0084] As one possible implementation, the method of generating sparse point clouds through 3D reconstruction using the essential matrix and camera intrinsic parameters includes: For candidate image pairs with overlapping fields of view in a multi-view 2D image sequence, solve for the corresponding essential matrix respectively; Singular value decomposition is performed on the essential matrix, and the relative rotation matrix and translation vector with undetermined scale between the corresponding views are solved by combining multi-view geometric constraints. Based on the relative rotation matrix obtained between each candidate image pair, the translation direction with undetermined scale, and the effective matching point pairs, the extrinsic parameters of each viewpoint camera are unified to the same global coordinate system, and the camera extrinsic parameters and 3D point coordinates are globally optimized to obtain the global projection matrix of each viewpoint camera. Based on the two-dimensional pixel coordinates of the matching feature points and the corresponding global projection matrix, the three-dimensional coordinates of the corresponding three-dimensional spatial points are solved by the triangulation method. Triangulation calculations are performed on all valid matching points, and spatial points with reprojection errors greater than a set threshold are filtered out. The points are then fused to generate an initial sparse point cloud. The absolute scale is then restored by combining the known physical baseline length to generate the sparse point cloud of the skirt pile sleeve.

[0085] As one possible implementation, the process of using the mapped key measurement points as initial search values, determining the coordinates of the measured key measurement points in the neighborhood point cloud, calculating the sleeve dimensions based on the measured key measurement point coordinates, and calculating the positioning uncertainty and confidence interval based on the neighborhood point cloud includes: For each key measurement point mapped to the point cloud coordinate system that has recovered absolute scale, the key measurement point is used as the initial search value, and the actual reconstructed point in the sparse point cloud is searched within a set radius using a Kd tree. Based on the actual reconstructed points obtained from the search, local plane fitting is performed to determine the coordinates of the measured key measurement points corresponding to the key measurement points; Calculate the deviation between the actual reconstructed point and the locally fitted geometric model, and statistically analyze the distribution parameters of the deviation as the positioning uncertainty of the measured key measurement point; Select key measurement points on a specific structure of the skirt pile sleeve and calculate the absolute spatial distance as the key dimension. The positioning uncertainties of the measured key measurement points are combined using the error propagation formula to calculate the upper and lower bounds of the confidence interval of the corresponding key dimensions at the preset probability level.

[0086] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0087] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A visual automatic measurement method for the prefabricated dimensions of skirt pile sleeves, characterized in that, include: A multi-viewpoint 2D image sequence of the skirt pile sleeve is acquired, feature points are detected, and local descriptors and optical flow are extracted. Based on the minimum eigenvalue of the feature point neighborhood structure tensor matrix and the optical flow tracking stability, the structure confidence of the feature points is calculated. Hough transform line segments and deep texture feature maps of the 2D images are extracted, and the line segment features in the neighborhood of the feature points are encoded into query vectors. The texture feature maps are used as keys and values. A cross-modal attention network is constructed with feature points as bridges. A spatial prior distribution is generated according to the structure confidence, and the spatial prior distribution is injected into the attention matrix to generate a fusion descriptor for the feature points. Initial matching point pairs are obtained by feature matching based on fusion descriptors, a matching point graph model is constructed, the betweenness centrality of nodes is calculated, and mismatched points are identified and removed based on the betweenness centrality. The essence matrix is ​​solved by combining a two-stage weighted random sampling consensus algorithm with structural confidence weighted sampling and model optimization. A sparse point cloud is generated through 3D reconstruction using the essential matrix and camera intrinsic parameters. The absolute scale of the sparse point cloud is recovered based on the known physical baseline length. The sparse point cloud is registered with a sleeve digital twin model to obtain a rigid transformation matrix. The predefined key measurement points of the twin model are mapped to the point cloud coordinate system. The mapped key measurement points are used as initial search values ​​to determine the coordinates of the measured key measurement points in the neighborhood point cloud. The size of the sleeve is calculated based on the measured key measurement point coordinates. The positioning uncertainty and confidence interval are calculated based on the neighborhood point cloud.

2. The method according to claim 1, characterized in that, The detection of feature points and extraction of local descriptors and optical flow includes: A corner detection algorithm is used to extract the pixel coordinates of feature points in a two-dimensional image sequence; Constructing a multi-resolution image pyramid for an image sequence; Calculate the horizontal and vertical gray-level gradients of the neighborhood of feature points in each layer of the pyramid to generate local structure descriptors; By calculating the gray-level smoothing constraints and basic optical flow equations of local image regions between adjacent frames, and using Taylor expansion and least squares iterative solution, the optical flow vectors of each feature point in the image sequence are obtained.

3. The method according to claim 1, characterized in that, The calculation of the structural confidence of feature points based on the minimum eigenvalue of the feature point neighborhood structure tensor matrix and optical flow tracking stability includes: A local neighborhood window is established with each feature point as the center. The horizontal and vertical gradients of the pixels within the window are calculated, and a second-order structure tensor matrix is ​​constructed. The structural tensor matrix is ​​subjected to eigenvalue decomposition, and the minimum eigenvalue is extracted as the structural saliency factor. Track the optical flow vectors of feature points in multiple consecutive frames of images, calculate the Euclidean distance deviation between the actual tracking trajectory and the theoretical smooth trajectory, and convert it into an optical flow tracking stability coefficient; The structural saliency factor is normalized, and the normalized structural saliency factor is linearly weighted and fused with the optical flow tracking stability coefficient to obtain the structural confidence of the feature point.

4. The method according to claim 2 or 3, characterized in that, The extraction of Hough transform line segments and deep texture feature maps from the two-dimensional image, and the encoding of line segment features within the neighborhood of feature points into query vectors, with the texture feature map serving as the key and value, includes: The endpoint coordinates, length, and orientation angle of the Hough transform line segments in the two-dimensional image are extracted to construct geometric primitive feature vectors. The geometric primitive feature vectors in the neighborhood of each feature point are aggregated and mapped into a query vector sequence of a specified channel dimension through a multilayer perceptron. Multi-scale feature extraction of two-dimensional images is performed by pre-trained convolutional neural networks. The feature maps output by the deep layers of the network are flattened into two-dimensional spatial sequences and generated as key and value matrices through linear projection, providing global texture context information for cross-modal attention fusion networks.

5. The method according to claim 1, characterized in that, The step of generating a spatial prior distribution based on the structural confidence and injecting the spatial prior distribution into the attention matrix to generate a fusion descriptor for the feature points includes: Calculate the inner product of the query vector and the key matrix to obtain the initial cross-attention score matrix; Based on the structural confidence of each feature point and its corresponding receptive field position in the deep texture feature map, a spatial confidence distribution map corresponding to the spatial sequence of the deep texture feature map is generated. The spatial confidence distribution map is flattened into a spatial prior weight vector corresponding to the spatial sequence of the key matrix, and then expanded into a spatial prior bias matrix with the same dimension as the initial cross-attention score matrix through a tensor broadcasting mechanism. The spatial prior bias values ​​corresponding to different key positions are determined according to their spatial confidence. The spatial prior bias matrix is ​​added element-wise to the initial cross-attention score matrix, and a normalized exponential function is applied to obtain the modulated attention weights. The value matrix is ​​weighted and summed using the modulated attention weights, and then concatenated with the original local structure descriptors of the feature points through residual connections to generate a fused descriptor for the feature points.

6. The method according to claim 1, characterized in that, The process involves constructing a matching point graph model, calculating the betweenness centrality of nodes, identifying and removing mismatched points based on the betweenness centrality, and combining a two-stage weighted random sampling consensus algorithm with structural confidence-weighted sampling and model optimization to solve for the essential matrix, including: The initial matching point pairs are used as graph nodes, and adjacent edges are constructed based on the spatial geometric consistency between the point pairs to generate a matching point graph model. Calculate the betweenness centrality of each node in the matching point graph model, calculate the mean and standard deviation of the betweenness centrality of each node, and determine the abnormal centrality threshold range based on the mean and standard deviation. Nodes whose betweenness centrality is outside the abnormal centrality threshold range are identified as mismatched nodes, and the mismatched nodes and their associated edges are removed from the matching point graph model. In the first stage, the sampling probability is calculated based on the sum of the structural confidence of the remaining matching points to the feature points at both ends, and the roulette wheel algorithm is used to extract a subset of samples for estimating the essential matrix. In the second stage, after calculating the candidate essential matrix using a subset of samples, the mean of the structural confidence of the feature points at both ends of the matching pairs corresponding to all interior points is used as a weight to be assigned to the corresponding residuals. The candidate essential matrix is ​​then nonlinearly optimized by minimizing the weighted epipolar geometric error.

7. The method according to claim 1, characterized in that, The method of generating sparse point clouds through 3D reconstruction using the essential matrix and camera intrinsic parameters includes: For candidate image pairs with overlapping fields of view in a multi-view 2D image sequence, solve for the corresponding essential matrix respectively; Singular value decomposition is performed on the essential matrix, and the relative rotation matrix and translation vector with undetermined scale between the corresponding views are solved by combining multi-view geometric constraints. Based on the relative rotation matrix obtained between each candidate image pair, the translation direction with undetermined scale, and the effective matching point pairs, the extrinsic parameters of each viewpoint camera are unified to the same global coordinate system, and the camera extrinsic parameters and 3D point coordinates are globally optimized to obtain the global projection matrix of each viewpoint camera. Based on the two-dimensional pixel coordinates of the matching feature points and the corresponding global projection matrix, the three-dimensional coordinates of the corresponding three-dimensional spatial points are solved by the triangulation method. Triangulation calculations are performed on all valid matching points, and spatial points with reprojection errors greater than a set threshold are filtered out. The points are then fused to generate an initial sparse point cloud. The absolute scale is then restored by combining the known physical baseline length to generate the sparse point cloud of the skirt pile sleeve.

8. The method according to claim 1, characterized in that, The process involves using the mapped key measurement points as initial search values, determining the coordinates of the measured key measurement points in the neighborhood point cloud, calculating the sleeve dimensions based on the measured key measurement point coordinates, and calculating the positioning uncertainty and confidence interval based on the neighborhood point cloud, including: For each key measurement point mapped to the point cloud coordinate system that has recovered absolute scale, the key measurement point is used as the initial search value, and the actual reconstructed point in the sparse point cloud is searched within a set radius using a Kd tree. Based on the actual reconstructed points obtained from the search, local plane fitting is performed to determine the coordinates of the measured key measurement points corresponding to the key measurement points; Calculate the deviation between the actual reconstructed point and the locally fitted geometric model, and statistically analyze the distribution parameters of the deviation as the positioning uncertainty of the measured key measurement point; Select key measurement points on a specific structure of the skirt pile sleeve and calculate the absolute spatial distance as the key dimension. The positioning uncertainties of the measured key measurement points are combined using the error propagation formula to calculate the upper and lower bounds of the confidence interval of the corresponding key dimensions at the preset probability level.

9. A visual automatic measurement system for the prefabricated dimensions of skirt pile sleeves, characterized in that, include: The generation module is used to acquire multi-viewpoint two-dimensional image sequences of the skirt pile sleeve, detect feature points and extract local descriptors and optical flow, calculate the structural confidence of feature points based on the minimum eigenvalue of the feature point neighborhood structure tensor matrix and the optical flow tracking stability, extract Hough transform line segments and deep texture feature maps of the two-dimensional images, encode the line segment features in the neighborhood of the feature points into query vectors, use the texture feature maps as keys and values, construct a cross-modal attention network with feature points as bridges, generate spatial prior distributions based on the structural confidence, inject the spatial prior distributions into the attention matrix, and generate fusion descriptors for feature points. The solution module is used to obtain initial matching point pairs based on feature matching of fused descriptors, construct a matching point graph model, calculate the betweenness centrality of nodes, identify and remove mismatched points based on the betweenness centrality, and solve the essential matrix by combining a two-stage weighted random sampling consensus algorithm with structural confidence weighted sampling and model optimization. The calculation module is used to generate a sparse point cloud by performing 3D reconstruction using the essential matrix and camera intrinsic parameters, recover the absolute scale of the sparse point cloud based on the known physical baseline length, register the sparse point cloud with the sleeve digital twin model to obtain the rigid transformation matrix, map the predefined key measurement points of the twin model to the point cloud coordinate system, use the mapped key measurement points as initial search values, determine the coordinates of the measured key measurement points in the neighborhood point cloud, calculate the size of the sleeve based on the measured key measurement point coordinates, and calculate the positioning uncertainty and confidence interval based on the neighborhood point cloud.

10. The system according to claim 9, characterized in that, The detection of feature points and extraction of local descriptors and optical flow includes: A corner detection algorithm is used to extract the pixel coordinates of feature points in a two-dimensional image sequence; Constructing a multi-resolution image pyramid for an image sequence; Calculate the horizontal and vertical gray-level gradients of the neighborhood of feature points in each layer of the pyramid to generate local structure descriptors; By calculating the gray-level smoothing constraints and basic optical flow equations of local image regions between adjacent frames, and using Taylor expansion and least squares iterative solution, the optical flow vectors of each feature point in the image sequence are obtained.

Images