Steel structure hole position virtual entity measurement method and system based on machine vision

Abstract

The present application relates to the technical field of machine vision measurement, in particular to a steel structure hole virtual entity measurement method and system based on machine vision, comprising: collecting original image sequences of steel structure bolt hole positions under multiple different illumination angles. Using an improved scale invariant feature transform algorithm, combined with the geometric constraint correction feature point screening condition of the stepped cylindrical reference tool space, key point detection and feature descriptor extraction are carried out on each image to generate a hole edge feature point set. The feature point set is input into a pre-trained hole profile reconstruction network to complete three-dimensional profile reconstruction and construct a bolt hole virtual entity three-dimensional model. Relying on the model, the hole center point spatial coordinates, hole diameter value and hole verticality deviation are calculated. This method can weaken the interference of single illumination imaging, optimize the quality of hole edge feature extraction, and realize the integrated measurement of steel structure bolt hole three-dimensional entity modeling and multi-dimensional space parameters.

Description

Technical Field

[0001] This invention relates to the field of machine vision measurement technology, and in particular to a method and system for virtual physical measurement of steel structure hole positions based on machine vision. Background Technology

[0002] Bolt hole location detection in steel structure components often employs manual contact measurement or machine vision acquisition under single, fixed lighting conditions. The imaging process does not utilize a multi-light angle simultaneous image sequence acquisition mode, resulting in a single, fixed imaging perspective and lighting conditions. Image acquisition is easily affected by ambient light and the surface morphology of the component, leading to the loss of effective information about the hole edges. Conventional feature detection algorithms rely on inherent filtering rules without incorporating spatial geometric constraints from external reference tools to adjust the filtering conditions. Consequently, the feature point filtering mechanism cannot adapt to the complex edge feature distribution of steel structure surfaces.

[0003] Existing machine vision measurement architectures mostly remain at the level of two-dimensional planar recognition, lacking the processing logic to reconstruct the three-dimensional contours of hole edge feature points through a dedicated reconstruction network, and thus unable to generate a complete virtual solid 3D model of the hole. The measurement process can only obtain planar dimension parameters, and cannot uniformly calculate the spatial coordinates of the hole center point, hole diameter, and perpendicularity deviation, and spatial attitude parameters are always excluded from the measurement scope.

[0004] The assembly and construction of steel structures have standardized control standards for the spatial location, diameter, and vertical orientation of bolt holes. The existing single-light imaging mode and fixed-rule feature extraction method have obvious limitations. The two-dimensional measurement system lacks the ability to perform three-dimensional solid modeling and multi-parameter synchronous calculation, which cannot meet the application requirements of high-precision, full-dimensional spatial parameter detection of steel structure holes. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and to propose a virtual physical measurement method and system for steel structure hole positions based on machine vision.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a machine vision-based virtual entity measurement method for steel structure hole positions, comprising: Acquire the original image sequence of the bolt hole positions to be measured on the target steel structure component. The original image sequence contains multiple bolt hole position images acquired under different lighting angles. An improved scale-invariant feature transform algorithm is invoked to perform key point detection and feature descriptor extraction on each bolt hole image in the original image sequence, generating a set of hole edge feature points corresponding to each bolt hole image. The improved scale-invariant feature transform algorithm modifies the feature point selection conditions based on the spatial geometric constraints of the stepped cylindrical reference tool. The set of feature points on the edge of the hole is input into a pre-trained hole contour reconstruction network for three-dimensional contour reconstruction to generate a virtual solid three-dimensional model of the bolt hole to be measured. Based on the virtual solidified 3D model, the spatial coordinates of the center point of the bolt hole to be measured, the hole diameter, and the hole perpendicularity deviation are calculated.

[0007] As a further aspect of the present invention, the improved scale-invariant feature transformation algorithm modifies the feature point selection conditions based on the spatial geometric constraints of the stepped cylindrical reference tool, specifically including: A pre-calibrated parameter set for a stepped cylindrical reference tool is obtained. One end of the stepped cylindrical reference tool is a stepped cylindrical structure with increasing size, and each step corresponds to a preset standard aperture value. The other end of the stepped cylindrical reference tool is a semi-cylindrical structure, and the plane containing the diameter of the semi-cylindrical structure is the measurement reference plane. The material of the stepped cylindrical reference tool is stainless steel. Insert the stepped cylindrical reference tool into the bolt hole to be measured, so that the stepped surface of the stepped cylindrical reference tool fits against the upper surface of the bolt hole to be measured, and the plane containing the diameter of the semi-cylindrical structure is perpendicular to the direction to be measured. Acquire a combined image containing the stepped cylindrical reference tool and the bolt hole to be measured, and extract the edge contour line of the semi-cylindrical structure of the stepped cylindrical reference tool from the combined image; Calculate the spatial scale reference value in the joint image based on the edge contour line of the semi-cylindrical structure; The spatial scale reference value is used as a normalization factor for the scale space to recalibrate the inter-layer scale parameters of the Gaussian difference pyramid in the scale-invariant feature transformation algorithm. In the feature point selection stage of the scale-invariant feature transformation algorithm, feature points located inside the stepped surface projection area of ​​the stepped cylindrical reference tool are retained, and feature points located outside the stepped surface projection area are deleted, thereby generating the set of feature points at the hole edge.

[0008] As a further aspect of the present invention, the acquisition of the original image sequence of the bolt hole positions to be measured on the target steel structural component includes: Multiple light sources are arranged around the bolt hole to be measured, with each light source located at a different spatial azimuth angle; Each of the multiple light sources is lit sequentially, and an industrial camera with a fixed position is used to capture an image of the bolt hole position under the condition that each light source is lit individually, so as to obtain multiple images of the bolt hole position under different lighting angles. Histogram equalization is performed on each bolt hole image to obtain bolt hole images with enhanced contrast; Median filtering is performed on each contrast-enhanced bolt hole image to remove salt-and-pepper noise, generating a filtered bolt hole image. All filtered bolt hole images are numbered according to the corresponding light source azimuth angle to form the original image sequence.

[0009] As a further aspect of the present invention, the set of feature points on the hole edge is input into a pre-trained hole contour reconstruction network for three-dimensional contour reconstruction processing to generate a virtual, solidified three-dimensional model of the bolt hole to be measured, including: Each feature point in the set of feature points at the hole edge is grouped according to the bolt hole image to which it belongs, and each group corresponds to a subset of feature points under a certain illumination angle. Each subset of feature points is input into the two-dimensional feature encoding layer of the hole contour reconstruction network to generate a two-dimensional feature map corresponding to each subset of feature points. The multi-view feature fusion layer in the aperture contour reconstruction network is invoked to stitch together the two-dimensional feature maps corresponding to multiple illumination angles in the order of the light source azimuth angle to generate a fused feature tensor. The fused feature tensor is input into the three-dimensional decoding layer of the aperture contour reconstruction network. The three-dimensional decoding layer is composed of multiple deconvolution modules stacked together. Each deconvolution module performs upsampling and convolution processing on the fused feature tensor in sequence to generate a three-dimensional voxel mesh. The three-dimensional voxel mesh is subjected to marchingcubes surface extraction processing to generate a mesh model in the form of triangular facets, and the mesh model is used as the virtual solidified three-dimensional model.

[0010] As a further aspect of the present invention, the training process of the hole contour reconstruction network includes: Obtain a training dataset, which contains multiple multi-view image sequences of steel structure bolt holes and a real 3D scanning model corresponding to each multi-view image sequence. For each multi-view image sequence, perform keypoint detection and feature descriptor extraction processing using the improved scale-invariant feature transformation algorithm to generate a set of training hole edge feature points; The set of training hole edge feature points is input into the hole contour reconstruction network to be trained to obtain a predicted three-dimensional voxel mesh. Calculate the binary cross-entropy loss value between the predicted 3D voxel mesh and the real 3D scanning model; The network weight parameters in the hole contour reconstruction network to be trained are updated using the backpropagation algorithm based on the binary cross-entropy loss value. Repeat the step of inputting the set of training hole edge feature points into the hole contour reconstruction network to be trained until the binary cross-entropy loss value converges to below the preset loss threshold.

[0011] As a further aspect of the present invention, the spatial coordinates of the center point of the bolt hole to be measured, the hole diameter, and the hole perpendicularity deviation are calculated based on the virtual solidified three-dimensional model, including: Extract the upper surface edge point cloud set and the lower surface edge point cloud set of the bolt hole to be measured from the virtual solidified 3D model; The least-squares circle fitting process is performed on the set of edge points on the upper surface to obtain the spatial coordinates of the center of the fitted circle on the upper surface and the radius value of the fitted circle on the upper surface. The least-squares circle fitting process is performed on the set of edge points on the lower surface to obtain the spatial coordinates of the center of the fitted circle and the radius of the fitted circle on the lower surface. The arithmetic mean of the spatial coordinates of the center of the fitted circle on the upper surface and the spatial coordinates of the center of the fitted circle on the lower surface is taken as the spatial coordinates of the center point of the bolt hole to be measured. The arithmetic mean of the radius values ​​of the fitted circle on the upper surface and the fitted circle on the lower surface is taken as the diameter value of the bolt hole to be measured. Connect the spatial coordinates of the center of the fitted circle on the upper surface with the spatial coordinates of the center of the fitted circle on the lower surface to obtain the central axis of the hole. Calculate the angle between the central axis of the hole and the normal vector of the reference bottom surface of the virtual solidified 3D model, and use the angle value as the verticality deviation value of the hole.

[0012] As a further aspect of the present invention, the extraction of the upper surface edge point cloud set and the lower surface edge point cloud set of the bolt hole to be measured from the virtual solidified 3D model includes: Perform histogram statistical processing on the virtual solidified 3D model in the height direction, count the number of point clouds at each height layer, and obtain a height distribution histogram; Identify the first peak height layer and the last peak height layer in the height distribution histogram, and take all point clouds corresponding to the first peak height layer as candidate upper surface point clouds and all point clouds corresponding to the last peak height layer as candidate lower surface point clouds. Radius filtering is performed on the candidate upper surface point cloud to delete outliers whose average distance from neighboring point clouds exceeds a preset radius filtering threshold, thereby generating the upper surface edge point cloud set. Radius filtering is performed on the candidate lower surface point cloud to delete outliers whose average distance from neighboring point clouds exceeds the preset radius filtering threshold, thereby generating the lower surface edge point cloud set.

[0013] As a further aspect of the present invention, least-squares circle fitting is performed on the set of edge points on the upper surface to obtain the spatial coordinates of the center of the fitted circle and the radius of the fitted circle, including: Three point cloud points are randomly selected from the point cloud set at the edge of the upper surface, and the center coordinates and radius of the initial circle are calculated based on the spatial coordinates of the three point cloud points. Calculate the measured distance from each point in the upper surface edge point cloud set to the center coordinate of the initial circle, and use the square of the difference between the measured distance and the radius of the initial circle as the fitting error of the point cloud point. The total fitting error value is obtained by summing the fitting errors of all point cloud points. Adjust the center coordinates and radius of the initial circle to minimize the total fitting error. Use the center coordinates at this point as the center coordinates of the fitted circle on the upper surface, and use the radius of the fitted circle at this point as the radius of the fitted circle on the upper surface.

[0014] As a further aspect of the present invention, after generating the virtual solidified three-dimensional model of the bolt hole position to be measured, the method further includes: The virtual solidified 3D model is imported into an augmented reality display device, so that the virtual solidified 3D model is superimposed and displayed on the real bolt hole position to be measured; A digital annotation layer is rendered and displayed in the augmented reality display device, showing the spatial coordinates of the center point, the aperture value, and the perpendicularity deviation value of the aperture. Receive measurement auxiliary points selected by the user via gesture interaction on the augmented reality display device; Calculate the projection point position of the measurement auxiliary point on the virtual solidified 3D model based on the screen coordinates of the measurement auxiliary point; Extract the spatial distance value from the projection point position to the center point spatial coordinates from the virtual solidified 3D model, and display the spatial distance value in real time on the augmented reality display device.

[0015] As a further aspect of the present invention, the present invention also includes a machine vision-based virtual physical measurement system for steel structure holes. The steel structure hole measurement system includes a processor and a memory. The memory is connected to the processor and is used to store programs, instructions, or code. The processor is used to run the programs, instructions, or code in the memory to implement the machine vision-based virtual physical measurement method for steel structure holes as described above.

[0016] Compared with the prior art, the advantages and positive effects of the present invention are as follows: Original image sequences of steel bolt hole locations were acquired under multiple illumination angles. Using the spatial geometric constraints of a stepped cylindrical reference tool, the feature point selection conditions of the scale-invariant feature transform algorithm were modified. Keypoint detection and feature descriptor extraction were sequentially performed for each image, forming a corresponding set of edge feature points for the hole locations. The multi-illumination angle image acquisition mode can accommodate the edge information of the hole locations under the influence of illumination changes. The geometric constraints can standardize and correct the original feature point selection logic. The feature point selection process has adaptive constraint capabilities, retaining edge feature points that fit the true contour of the hole locations and filtering out invalid feature point distribution states. The feature point extraction exhibits stable and reliable regularity.

[0017] The collected set of edge feature points of the bolt holes is input into a pre-trained bolt hole contour reconstruction network to perform three-dimensional contour reconstruction processing, generating a complete virtual solid three-dimensional model of the bolt hole to be measured. The discretely distributed two-dimensional edge feature points can complete structural fitting according to spatial correlation to form a continuous and complete three-dimensional solid representation. The overall spatial structure and contour of the bolt hole are completely reproduced, breaking through the inherent limitation of two-dimensional images that can only represent planar information.

[0018] Based on the constructed virtual solidified 3D model, quantitative analytical calculations are performed to solve for the spatial coordinates of the center point of the bolt hole to be measured, the hole diameter, and the hole perpendicularity deviation. The 3D model carries complete spatial structure and attitude information, and can simultaneously output planar dimensional parameters and spatial attitude parameters. The measurement content covers position, hole diameter, and vertical offset related indicators, and the types of measurement parameters are more complete. Synchronous analytical calculations of multiple types of hole position parameters can be completed using a unified 3D model. Attached Figure Description

[0019] Figure 1 This is a state diagram of the machine vision-based virtual entity measurement method for steel structure hole positions according to the present invention. Figure 2 A flowchart for obtaining the original image sequence of the bolt hole positions to be measured; Figure 3 A flowchart for the 3D reconstruction of the hole contour network. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0021] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0022] See Figure 1 A machine vision-based virtual entity measurement method for bolt holes in steel structures is proposed, with the following overall implementation scheme: First, an original image sequence of bolt holes to be measured on the target steel structure is acquired. This original image sequence contains multiple bolt hole images acquired under different lighting angles. Then, an improved scale-invariant feature transform algorithm is used to perform key point detection and feature descriptor extraction on each bolt hole image in the original image sequence, generating a set of edge feature points corresponding to each bolt hole image. The improved scale-invariant feature transform algorithm modifies the feature point selection conditions based on the spatial geometric constraints of a stepped cylindrical reference tool. Next, the set of edge feature points is input into a pre-trained hole contour reconstruction network for three-dimensional contour reconstruction, generating a virtual entity 3D model of the bolt hole to be measured. Finally, based on the virtual entity 3D model, the spatial coordinates of the center point, the hole diameter, and the hole perpendicularity deviation value of the bolt hole to be measured are calculated.

[0023] In one embodiment of the present invention, a pre-calibrated parameter set of a stepped cylindrical reference tool is obtained. One end of the stepped cylindrical reference tool is a stepped cylindrical structure with increasing size, and each step corresponds to a preset standard hole diameter value. The other end of the stepped cylindrical reference tool is a semi-cylindrical structure, and the plane containing the diameter of the semi-cylindrical structure is the measurement reference plane. The stepped cylindrical reference tool is made of stainless steel. The stepped cylindrical reference tool is inserted into the bolt hole to be measured, so that the step surface of the stepped cylindrical reference tool is in contact with the upper surface of the bolt hole to be measured, and the plane containing the diameter of the semi-cylindrical structure is perpendicular to the direction to be measured. Data including the stepped... A combined image of a cylindrical reference tool and the bolt hole to be measured is generated. The edge contour line of the semi-cylindrical structure of the stepped cylindrical reference tool is extracted from the combined image. The spatial scale reference value in the combined image is calculated based on the edge contour line of the semi-cylindrical structure. The spatial scale reference value is used as a normalization factor in the scale space to recalibrate the inter-layer scale parameters of the Gaussian difference pyramid in the scale-invariant feature transformation algorithm. In the feature point selection stage of the scale-invariant feature transformation algorithm, feature points located inside the stepped surface projection area of ​​the stepped cylindrical reference tool are retained, and feature points located outside the stepped surface projection area are deleted to generate the set of feature points at the edge of the hole.

[0024] See Figure 2 Multiple light sources are arranged around the bolt hole to be measured, each located at a different spatial azimuth angle. Each of the multiple light sources is illuminated sequentially, and an industrial camera with a fixed position is used to capture an image of the bolt hole under the condition that each light source is illuminated individually, resulting in multiple bolt hole images under different illumination angles. Histogram equalization is performed on each bolt hole image to obtain a bolt hole image with enhanced contrast. Median filtering is performed on each bolt hole image with enhanced contrast to remove salt-and-pepper noise, generating a filtered bolt hole image. All filtered bolt hole images are numbered according to the corresponding light source azimuth angle to form the original image sequence.

[0025] In practical implementation, for bolt hole locations on a steel structural component to be measured in a prefabricated steel structure component workshop, a stepped cylindrical reference tool is used. One end of the stepped cylindrical reference tool is a stepped cylindrical structure with increasing diameter, and each step corresponds to a preset standard hole diameter value. The other end of the stepped cylindrical reference tool is a semi-cylindrical structure, and the plane containing the diameter of the semi-cylindrical structure is the measurement reference plane. The stepped cylindrical reference tool is made of stainless steel. Before measurement, the stepped cylindrical reference tool is calibrated using a coordinate measuring machine to obtain a pre-calibrated parameter set. This pre-calibrated parameter set includes each step... The operator inserts the stepped cylindrical reference tool into the bolt hole to be measured, ensuring that the stepped surface of the tool aligns with the upper surface of the bolt hole, while simultaneously ensuring that the plane containing the diameter of the semi-cylindrical structure is perpendicular to the direction to be measured (i.e., the direction perpendicular to the central axis of the bolt hole). A fixed industrial camera is then positioned to capture the combined area containing the stepped cylindrical reference tool and the bolt hole, and a combined image is acquired. The edge contour of the semi-cylindrical structure of the stepped cylindrical reference tool is extracted from this combined image.

[0026] In some embodiments, the specific method for calculating the spatial scale reference value in the joint image based on the edge contour of the semi-cylindrical structure is as follows: measuring the pixel length occupied by the diameter of the edge contour, denoted as... Then, read the actual diameter length of the semi-cylindrical structure from the pre-calibrated parameter set and record it as... According to the formula:

[0027] Calculate spatial scale reference value The spatial scale reference value represents the actual physical size corresponding to each pixel in the joint image. Using this spatial scale reference value as a normalization factor for the scale space, the inter-layer scale parameters of the Gaussian difference pyramid in the scale-invariant feature transformation algorithm are recalibrated. Specifically, the scale parameters of each layer of the original Gaussian difference pyramid are multiplied by the spatial scale reference value, so that the scale of the subsequently detected key points has a unified physical meaning, and the recalibrated scale parameters have a unified physical dimension.

[0028] In practice, after the scale parameters are recalibrated, during the feature point selection stage of the scale-invariant feature transformation algorithm, the projection areas of each step surface of the stepped cylindrical reference tool are identified from the joint image. Feature points located inside the projection area of ​​the step surface of the stepped cylindrical reference tool are retained, while feature points located outside the projection area of ​​the step surface are deleted. All retained feature points are used as the set of hole edge feature points. This set of hole edge feature points only includes feature points of the bolt hole edge and the step surface edge of the reference tool, eliminating feature points generated by irrelevant structures in the background area. Optionally, during the above measurement process, eight LED light sources are arranged around the bolt hole to be measured, each light source is located at a different spatial azimuth angle, and the azimuth angle interval between two adjacent light sources is 45 degrees; each of these eight light sources is lit sequentially, and the position of the industrial camera remains fixed under the condition that each light source is lit individually, and one bolt hole image is acquired, resulting in a total of eight bolt hole images under different illumination angles; histogram equalization is performed on each bolt hole image to stretch the pixel grayscale value distribution to the entire grayscale range, resulting in a bolt hole image with enhanced contrast; median filtering is performed on each bolt hole image with enhanced contrast, using a 3×3 filtering window to scan the entire image, and replacing the grayscale value of the center pixel of the window with the median of all pixel grayscale values ​​within the window to remove salt-and-pepper noise, generating a filtered bolt hole image.

[0029] In some embodiments, all filtered bolt hole images are numbered according to the corresponding light source azimuth angle sequence (i.e., 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, 315 degrees) to form an original image sequence. Each image in the original image sequence is labeled with a light source azimuth angle for subsequent multi-view feature fusion processing.

[0030] Optionally, in actual workshop environments, a 5-megapixel monochrome industrial camera equipped with a telecentric lens is used to reduce distortion errors caused by perspective projection. All eight light sources are high-brightness white LED point sources, with each source maintaining a consistent distance of 200 mm from the center of the bolt hole to be measured, and the light source's illumination direction pointing towards the center of the bolt hole. It can be understood that after the stepped cylindrical reference tool is inserted into the bolt hole, because the stepped surface fits against the upper surface of the bolt hole, and the plane containing the diameter of the semi-cylindrical structure is perpendicular to the measurement direction, the semi-cylindrical edge contour extracted from the joint image can accurately reflect the current image acquisition scale, providing a physical reference for scale parameter correction in subsequent scale-invariant feature transformation algorithms.

[0031] It is understandable that by sequentially illuminating independent light sources at different azimuth angles to acquire multiple images, the direction and intensity of the shadows on the bolt hole edges in each image are different. These image sequences under different illumination angles can fully expose the subtle geometric features of the hole edges in various directions, providing rich two-dimensional information for subsequent edge feature point extraction and three-dimensional reconstruction.

[0032] In one embodiment of the present invention, see [reference] Figure 3 Each feature point in the set of bolt hole edge feature points is grouped according to its corresponding bolt hole image, with each group corresponding to a subset of feature points under a specific illumination angle. Each subset of feature points is input into the 2D feature encoding layer of the bolt hole contour reconstruction network to generate a 2D feature map corresponding to each subset. The multi-view feature fusion layer of the bolt hole contour reconstruction network is then invoked to stitch together the 2D feature maps corresponding to multiple illumination angles according to the azimuth angle of the light source, generating a fused feature tensor. This fused feature tensor is input into the 3D decoding layer of the bolt hole contour reconstruction network. The 3D decoding layer consists of multiple stacked deconvolution modules, each of which sequentially performs upsampling and convolution processing on the fused feature tensor to generate a 3D voxel mesh. The 3D voxel mesh is then subjected to marchingcubes surface extraction processing to generate a mesh model in the form of triangular facets, which is used as the virtual solidified 3D model.

[0033] A training dataset is obtained, which includes multiple multi-view image sequences of steel structure bolt holes and a corresponding real 3D scanning model for each multi-view image sequence. Keypoint detection and feature descriptor extraction are performed on each multi-view image sequence using the improved scale-invariant feature transform algorithm to generate a training set of bolt hole edge feature points. This training set of bolt hole edge feature points is then input into a bolt hole contour reconstruction network to be trained, resulting in a predicted 3D voxel mesh. The binary cross-entropy loss value between the predicted 3D voxel mesh and the real 3D scanning model is calculated. Based on the binary cross-entropy loss value, the network weight parameters in the bolt hole contour reconstruction network to be trained are updated using the backpropagation algorithm. The step of inputting the training set of bolt hole edge feature points into the bolt hole contour reconstruction network to be trained is repeated until the binary cross-entropy loss value converges to below a preset loss threshold.

[0034] In practical implementation, for a batch of bolt holes to be measured on a prefabricated steel structure production line, the original image sequence corresponding to each bolt hole is processed by keypoint detection and feature descriptor extraction using an improved scale-invariant feature transform algorithm to generate a set of edge feature points for the hole holes. Each feature point in this set is grouped according to the bolt hole image to which it belongs. Each group corresponds to a subset of feature points under a certain illumination angle, resulting in eight feature point subsets corresponding to eight different light source azimuth angles (0 degrees, 45 degrees, 90 degrees, 1...). Bolt hole images were acquired at 35°, 180°, 225°, 270°, and 315°. Each subset of feature points was input into a two-dimensional feature encoding layer in a pre-constructed hole contour reconstruction network. This two-dimensional feature encoding layer consisted of five stacked convolutional modules. Each convolutional module contained a two-dimensional convolutional layer with a kernel size of 3×3, a batch normalization layer, and a linear rectified activation function layer. After processing by this two-dimensional feature encoding layer, a two-dimensional feature map corresponding to each subset of feature points was generated, resulting in a total of eight two-dimensional feature maps of the same size.

[0035] In some embodiments, a multi-view feature fusion layer of the aperture contour reconstruction network is invoked. This multi-view feature fusion layer stitches together the two-dimensional feature maps corresponding to eight illumination angles in the channel dimension according to the azimuth angle of the light source (arranged in ascending order from 0 degrees to 315 degrees) to generate a fused feature tensor. The number of channels in the fused feature tensor is eight times the number of channels in each individual two-dimensional feature map. The fused feature tensor is then input into the three-dimensional decoding layer in the aperture contour reconstruction network. This three-dimensional decoding layer consists of four deconvolution modules stacked together. Each deconvolution module contains an upsampling layer (which uses bilinear interpolation to double the size of the feature map) and a two-dimensional convolutional layer with a kernel size of 3×3. After sequential processing by the four deconvolution modules, a three-dimensional voxel grid is generated. The size of the three-dimensional voxel grid is 64×64×64, and the value of each voxel unit represents the probability that the position belongs to the interior space of the aperture.

[0036] Optionally, the three-dimensional voxel mesh is subjected to marchingcubes surface extraction processing. The isosurface threshold is set to 0.5, and the positions where the voxel value is equal to 0.5 are used as boundaries. The mesh model in the form of triangular facets is extracted and used as a virtual solidified three-dimensional model of the bolt hole position to be measured. The virtual solidified three-dimensional model is stored in triangular facet file format, containing all vertex coordinates and triangle index information. In the specific implementation, for the training process of the bolt hole contour reconstruction network, a training dataset is obtained. This training dataset contains multi-view image sequences of 5,000 steel structure bolt holes and a corresponding real 3D scanning model for each multi-view image sequence. Each multi-view image sequence contains eight bolt hole images under different lighting angles. Each real 3D scanning model is obtained by scanning the same bolt hole using a structured light 3D scanner. Keypoint detection and feature descriptor extraction are performed on each multi-view image sequence using an improved scale-invariant feature transform algorithm to generate a training set of bolt hole edge feature points. The training set of bolt hole edge feature points is input into the bolt hole contour reconstruction network to be trained. After processing by the aforementioned two-dimensional feature encoding layer, multi-view feature fusion layer, and three-dimensional decoding layer, a predicted three-dimensional voxel mesh is obtained.

[0037] In some embodiments, the binary cross-entropy loss value between the predicted 3D voxel mesh and the actual 3D scan model is calculated using the following formula. :

[0038] in: This indicates the total number of voxel cells in the predicted 3D voxel mesh. Indicates the index number of the voxel unit. In the real 3D scan model, the first Occupancy tag of a voxel unit (a value of 1 indicates that the voxel is occupied by a pore entity, and a value of 0 indicates that the voxel is empty). Indicates the prediction of the 3D voxel mesh. Predicted probability values ​​for individual element units (ranging from 0 to 1); based on binary cross-entropy loss values. The backpropagation algorithm is used to update the network weight parameters in the hole contour reconstruction network to be trained. The backpropagation algorithm adopts an adaptive moment estimation optimizer with a learning rate of 0.001. The step of inputting the set of training hole edge feature points into the hole contour reconstruction network to be trained is repeated. After each iteration, the binary cross-entropy loss value is recalculated and the network weight parameters are updated until the binary cross-entropy loss value converges to below the preset loss threshold of 0.05.

[0039] Optionally, during training, each batch processes sixteen training samples simultaneously. The arithmetic mean of the binary cross-entropy loss values ​​calculated for each of the sixteen predicted 3D voxel grids and the corresponding sixteen real 3D scanning models is taken as the overall loss value for that batch. Training is terminated early when the overall loss value of ten consecutive batches fails to decrease. This can be understood as follows: grouping the set of feature points at the aperture location according to their respective original images and encoding them separately, then stitching and fusing them in order of light source azimuth, allows the aperture contour reconstruction network to fully utilize the differences in edge features under multi-angle illumination to infer the 3D structure of the aperture. The shadow direction and intensity of the aperture edge under different illumination angles provide geometric constraint information for 3D reconstruction. It can also be understood that when using the binary cross-entropy loss function for supervised training, each voxel unit in the predicted 3D voxel grid independently participates in the loss calculation. The real 3D scanning model is converted into binary voxel labels with the same resolution (64×64×64) as the predicted voxel grid through voxelization. The network weights are iteratively optimized through backpropagation, enabling the aperture contour reconstruction network to learn the nonlinear transformation relationship from the set of 2D edge feature points to the occupancy of 3D voxels.

[0040] In one embodiment of the present invention, the upper surface edge point cloud set and the lower surface edge point cloud set of the bolt hole to be measured are extracted from the virtual solidified 3D model; least squares circle fitting is performed on the upper surface edge point cloud set to obtain the spatial coordinates of the center of the fitted circle and the radius value of the fitted circle; least squares circle fitting is performed on the lower surface edge point cloud set to obtain the spatial coordinates of the center of the fitted circle and the radius value of the fitted circle; the arithmetic mean of the spatial coordinates of the center of the fitted circle and the center of the fitted circle is used as the spatial coordinates of the center point of the bolt hole to be measured; the arithmetic mean of the radius values ​​of the fitted circle and the fitted circle is used as the hole diameter value of the bolt hole to be measured; the spatial coordinates of the center of the fitted circle and the center of the fitted circle are connected to obtain the center axis of the hole; the angle between the center axis of the hole and the normal vector of the reference bottom surface of the virtual solidified 3D model is calculated, and the angle value is used as the perpendicularity deviation value of the hole.

[0041] In practical implementation, for a bolt hole location to be measured that has already generated a virtual solidified 3D model, the virtual solidified 3D model is stored in the form of a triangular mesh. The upper surface edge point cloud set and the lower surface edge point cloud set of the bolt hole location to be measured are extracted from this virtual solidified 3D model. Least squares circle fitting is performed on the upper surface edge point cloud set of the bolt hole location to be measured to obtain the spatial coordinates of the center of the fitted circle on the upper surface. The radius of the fitted circle on the upper surface is denoted as The least-squares circle fitting process is performed on the point cloud set of the lower surface edge of the bolt hole to be measured, and the spatial coordinates of the center of the fitted circle on the lower surface are marked. The radius of the fitted circle on the lower surface is denoted as In some embodiments, the arithmetic mean of the spatial coordinates of the center of the fitted circle on the upper surface and the spatial coordinates of the center of the fitted circle on the lower surface is used as the spatial coordinates of the center point of the bolt hole to be measured. Coordinates according to the formula calculate, Coordinates according to the formula calculate, Coordinates according to the formula Calculate; the arithmetic mean of the fitted circle radii on the upper and lower surfaces is taken as the diameter of the bolt hole to be measured, and the diameter is calculated according to the formula. calculate.

[0042] In practical implementation, the spatial coordinates of the center of the fitted circle on the upper surface and the spatial coordinates of the center of the fitted circle on the lower surface are used to obtain the central axis of the hole position. The direction vector of this central axis is... Extract the normal vector of the reference bottom surface from the virtual solidified 3D model. This reference bottom surface is a plane in the virtual solidified 3D model parallel to the bottom surface of the steel structural member where the bolt hole to be measured is located, and its normal vector is denoted as... Calculate the angle between the center axis of the hole and the normal vector of the reference bottom surface, and calculate the cosine value of the angle using the dot product formula:

[0043] in: This represents the angle between the center axis of the hole and the normal vector of the reference bottom surface. The calculated value... The angle value is obtained by inverse cosine transformation, and this angle value is used as the hole position perpendicularity deviation value of the bolt hole to be measured. The hole position perpendicularity deviation value is in degrees.

[0044] Optionally, in a specific precast steel structure component, the spatial coordinates of the center of the fitted circle on the upper surface of the bolt hole to be measured are (125.36 mm, 80.52 mm, 15.00 mm), and the radius of the fitted circle on the upper surface is 9.98 mm; the spatial coordinates of the center of the fitted circle on the lower surface are (125.28 mm, 80.48 mm, 5.00 mm), and the radius of the fitted circle on the lower surface is 9.97 mm; the calculated spatial coordinates of the center point are (125.32 mm, 80.50 mm, 10.00 mm), and the hole diameter is 9.975 mm; the angle between the center axis of the hole and the normal vector of the reference bottom surface is calculated to be 0.68 degrees, and this 0.68 degrees is used as the perpendicularity deviation value of the hole. In some embodiments, the numerical records involved in the above calculation process are shown in Table 1.

[0045] Table 1: Record of various numerical values ​​involved in the calculation process

[0046] It is understandable that by performing least-squares circle fitting on the point clouds of the upper and lower surface edges respectively, and taking the arithmetic mean of the coordinates of the centers of the two fitted circles as the center point of the entire bolt hole, the eccentricity error introduced by hole tilting or machining deviation can be eliminated. Taking the arithmetic mean of the radius values ​​of the upper and lower surfaces as the hole diameter value can reflect the average size of the hole along the depth direction. It is also understandable that using the angle between the hole's central axis and the normal vector of the reference bottom surface as the hole's perpendicularity deviation value directly reflects the degree of vertical deviation of the bolt hole relative to the reference bottom surface of the steel structure; the smaller the angle value, the more perpendicular the hole is to the reference bottom surface, and the larger the angle value, the more severe the hole tilt.

[0047] In one embodiment of the present invention, histogram statistical processing in the height direction is performed on the virtual solidified 3D model to count the number of point clouds at each height layer, thereby obtaining a height distribution histogram; the first peak height layer and the last peak height layer in the height distribution histogram are identified, and all point clouds corresponding to the first peak height layer are selected as candidate upper surface point clouds, and all point clouds corresponding to the last peak height layer are selected as candidate lower surface point clouds; radius filtering processing is performed on the candidate upper surface point clouds to delete outliers whose average distance from neighboring point clouds exceeds a preset radius filtering threshold, thereby generating the upper surface edge point cloud set; radius filtering processing is performed on the candidate lower surface point clouds to delete outliers whose average distance from neighboring point clouds exceeds the preset radius filtering threshold, thereby generating the lower surface edge point cloud set.

[0048] Three point cloud points are randomly selected from the point cloud set at the upper surface edge. The center coordinates and radius of the initial circle are calculated based on the spatial coordinates of the three point cloud points. The measured distance from each point cloud point in the point cloud set at the upper surface edge to the center coordinates of the initial circle is calculated. The square of the difference between the measured distance and the radius of the initial circle is taken as the fitting error of the point cloud point. The fitting errors of all point cloud points are summed to obtain the total fitting error value. The center coordinates and radius of the initial circle are adjusted to minimize the total fitting error value. The center coordinates at this time are taken as the spatial coordinates of the center of the fitted circle on the upper surface, and the radius of the circle at this time is taken as the radius of the fitted circle on the upper surface.

[0049] In practical implementation, for a pre-generated virtual solidified 3D model of the bolt hole positions to be measured, this virtual solidified 3D model is stored in the form of a triangular mesh and contains the spatial coordinate information of all point clouds. Histogram statistical processing is performed on this virtual solidified 3D model in the height direction. The height direction is uniformly divided into one hundred height layers from minimum to maximum value. The number of point clouds in each height layer is counted to obtain a height distribution histogram. The first peak height layer (i.e., the layer with the most point clouds at the minimum height position) and the last peak height layer (i.e., the layer with the most point clouds at the maximum height position) in this height distribution histogram are identified. The first peak height layer is then... All point clouds corresponding to the peak height layer are used as candidate upper surface point clouds, and all point clouds corresponding to the last peak height layer are used as candidate lower surface point clouds. Radius filtering is performed on the candidate upper surface point clouds, with a radius filtering threshold of 0.5 mm. For each point cloud in the candidate upper surface point cloud, the average distance between the point cloud and its neighboring point cloud points is calculated, and outliers with an average distance greater than 0.5 mm are deleted to generate the upper surface edge point cloud set. Radius filtering is also performed on the candidate lower surface point clouds, using the same radius filtering threshold of 0.5 mm, and outliers with an average distance greater than 0.5 mm from neighboring point cloud points are deleted to generate the lower surface edge point cloud set.

[0050] In some embodiments, the number of height layers in the above histogram statistical processing is dynamically adjusted according to the total height of the virtual solidified 3D model. When the total height of the virtual solidified 3D model is greater than 20 mm, the number of height layers increases to two hundred. During the radius filtering process, the search radius of neighboring point cloud points is set to 1.0 mm. Each point cloud point needs at least three neighboring point cloud points to calculate the average distance. If the number of neighboring point cloud points is less than three, the point cloud point is directly identified as an outlier and deleted. Optionally, in a specific example of steel structure bolt hole measurement, the total height of the virtual solidified 3D model is 12 mm, and the height direction is from Z=4 mm to Z=16 mm. After dividing into one hundred height layers, the first peak height layer is located at Z=15.2 mm, and the last peak height layer is located at Z=4.8 mm. The number of candidate upper surface point clouds, candidate lower surface point clouds, and the number of point clouds retained after filtering are shown in Table 2.

[0051] Table 2: Comparison of Point Cloud Quantities Before and After Filtering

[0052] In the specific implementation, least squares circle fitting is performed on the upper surface edge point cloud set. Three point clouds are randomly selected from the upper surface edge point cloud set, and the center coordinates and radius of the initial circle are calculated based on the spatial coordinates of these three point clouds. The measured distance from each point cloud in the upper surface edge point cloud set to the center coordinates of the initial circle is calculated, and the square of the difference between the measured distance and the initial circle radius is taken as the fitting error of that point cloud. The fitting errors of all point clouds in the surface edge point cloud set are accumulated to obtain the total fitting error value. The center coordinates and radius of the initial circle are adjusted (using the Levenberg-Marquardt algorithm for iterative optimization) to minimize the total fitting error value. The corresponding center coordinates at this time are taken as the spatial coordinates of the center of the fitted circle on the upper surface, and the corresponding circle radius is taken as the radius of the fitted circle on the upper surface. The same least squares circle fitting process is performed on the lower surface edge point cloud set to obtain the spatial coordinates of the center of the fitted circle on the lower surface.

[0053] In some embodiments, the total fitting error is calculated using the following formula:

[0054] in: This represents the total fitting error value of the point cloud set at the upper surface edge. This represents the total number of point cloud points in the point cloud set at the upper surface edge. This represents the index number of a point in the point cloud set at the upper surface edge. , , Indicates the first Spatial coordinates of a point cloud point , , This represents the spatial coordinates of the center of the fitted circle. This represents the radius of the fitted circle; the square root term in the formula calculates the spatial distance from the point cloud points to the center of the circle, subtracts the radius to obtain the deviation, squares the deviation, and sums them to obtain the total fitting error; adjustment , , , Four parameters make The parameters corresponding to minimization are the parameters of the fitted circle on the upper surface that are sought.

[0055] Optionally, during the iterative optimization process, the process of randomly selecting three points is repeated one hundred times. Each time, a set of initial circle parameters is obtained, and the set that minimizes the initial total fitting error is selected as the starting point for iterative optimization to avoid getting trapped in local optima. It can be understood that the first and last peak height layers are identified through height direction histogram statistics to extract the upper and lower surface edge point clouds. This utilizes the characteristic that the point clouds on the upper and lower surfaces of the bolt hole are most concentrated in terms of height distribution, accurately separating the point cloud data of the upper and lower surface regions. Radius filtering further eliminates isolated noise and outliers generated during 3D reconstruction, improving the accuracy of subsequent circle fitting. It can also be understood that the least squares circle fitting process uses spatial distance calculation instead of planar projection fitting, avoiding the projection error introduced when projecting the 3D point cloud onto a 2D plane. The fitted circle center coordinates and radius values ​​are directly based on the 3D spatial geometric relationship, consistent with the actual geometric dimensions of the bolt hole. The square root term in the total fitting error formula directly calculates the 3D Euclidean distance, making the fitting results more robust to point cloud acquisition errors.

[0056] In one embodiment of the present invention, after generating a virtual solidified 3D model of the bolt hole to be measured, the virtual solidified 3D model is imported into an augmented reality display device, so that the virtual solidified 3D model is superimposed on the real bolt hole to be measured; a digital annotation layer of the center point spatial coordinates, the hole diameter value, and the hole verticality deviation value is rendered and displayed in the augmented reality display device; a measurement auxiliary point selected by the user on the augmented reality display device through gesture interaction is received; the projection point position of the measurement auxiliary point on the virtual solidified 3D model is calculated according to the screen coordinates of the measurement auxiliary point; the spatial distance value from the projection point position to the center point spatial coordinates is extracted from the virtual solidified 3D model, and the spatial distance value is displayed on the augmented reality display device in real time.

[0057] In practice, for a bolt hole whose center point spatial coordinates, hole diameter, and hole perpendicularity deviation have been calculated, a virtual solidified 3D model of the bolt hole is imported into an augmented reality display device (using head-mounted augmented reality glasses). This virtual solidified 3D model is then overlaid on the actual bolt hole. The augmented reality display device achieves spatial alignment between the virtual model and the actual hole by recognizing QR code markers pre-attached to the surface of the steel structure. A digital annotation layer displaying the center point spatial coordinates, hole diameter, and hole perpendicularity deviation is rendered and displayed on the augmented reality display device. This digital annotation layer is displayed as a semi-transparent text box floating next to the virtual solidified 3D model. The center point spatial coordinates are displayed in the format "(X,Y,Z)", the hole diameter is displayed as "diameter:XX.XXmm", and the hole perpendicularity deviation is displayed as "perpendicularity:X.XX°". In some embodiments, the augmented reality display device is equipped with a gesture recognition camera to receive measurement auxiliary points selected by the user on the augmented reality display device through gesture interaction. The user points to a position on the virtual physical 3D model with their finger in the air, and the gesture recognition camera captures the 3D spatial coordinates of the fingertip. The coordinates are projected onto the screen of the augmented reality display device to obtain screen coordinates, and the screen coordinates are used as the screen coordinates of the measurement auxiliary points.

[0058] In practice, the projection point position of the measurement auxiliary point on the virtual solidified 3D model is calculated based on the screen coordinates of the measurement auxiliary point. Specifically, a ray is emitted from the virtual camera viewpoint of the augmented reality display device, passing through the screen coordinates of the measurement auxiliary point. The intersection points of this ray with all triangular faces of the virtual solidified 3D model are calculated, and the intersection point closest to the virtual camera viewpoint is selected as the projection point position. The 3D spatial coordinates of this projection point position on the virtual solidified 3D model are recorded. .

[0059] Optionally, the spatial distance from the projection point to the center point's spatial coordinates can be extracted from the virtual solidified 3D model and calculated using the following formula:

[0060] in: This represents the spatial distance from the projection point to the center point's spatial coordinates. This represents the three-dimensional spatial coordinates of the projection point on the virtual, solidified 3D model. The spatial coordinates of the center point of the bolt hole to be measured are represented, with the spatial distance value in millimeters. This spatial distance value is displayed in real time on the augmented reality display device as a floating label that follows the user's fingertip, with the label content "Distance from center: X.XX mm". In some embodiments, when the user selects multiple measurement auxiliary points consecutively, the augmented reality display device calculates the spatial distance value from the projection point position of each measurement auxiliary point to the spatial coordinates of the center point, and displays all spatial distance values ​​in a list in a sidebar of the augmented reality display device. The user can delete the selected measurement auxiliary points by pinching their finger or clear all measurement auxiliary point records by double-tapping.

[0061] Optionally, the augmented reality display device also provides a virtual measurement button. When the user clicks this button, the system freezes the current screen, saves the spatial distance values ​​corresponding to all current measurement auxiliary points, generates a measurement report, and exports it as text to an external storage device. This allows the measurement personnel to directly observe the fit between the virtual model and the actual bolt holes on the real workpiece. The digital annotation layer provides intuitive measurement value references, eliminating the need to consult paper reports or computer screens. Furthermore, the ability to select measurement auxiliary points and calculate and display spatial distances in real time via gesture interaction enhances the interactivity and flexibility of the measurement process. Measurement personnel can arbitrarily select locations of interest on the edge of the hole for distance measurement, and the obtained distance values ​​directly serve subsequent assembly accuracy analysis and adjustment.

[0062] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A machine vision-based virtual entity measurement method for steel structure hole positions, characterized in that, The method includes: Acquire the original image sequence of the bolt hole positions to be measured on the target steel structure component. The original image sequence contains multiple bolt hole position images acquired under different lighting angles. An improved scale-invariant feature transform algorithm is invoked to perform key point detection and feature descriptor extraction on each bolt hole image in the original image sequence, generating a set of hole edge feature points corresponding to each bolt hole image. The improved scale-invariant feature transform algorithm modifies the feature point selection conditions based on the spatial geometric constraints of the stepped cylindrical reference tool. The set of feature points on the edge of the hole is input into a pre-trained hole contour reconstruction network for three-dimensional contour reconstruction to generate a virtual solid three-dimensional model of the bolt hole to be measured. Based on the virtual solidified 3D model, the spatial coordinates of the center point of the bolt hole to be measured, the hole diameter, and the hole perpendicularity deviation are calculated.

2. The method for virtual physical measurement of steel structure hole positions based on machine vision according to claim 1, characterized in that, The improved scale-invariant feature transformation algorithm modifies the feature point selection criteria based on the spatial geometric constraints of the stepped cylindrical reference tool, specifically including: A pre-calibrated parameter set for a stepped cylindrical reference tool is obtained. One end of the stepped cylindrical reference tool is a stepped cylindrical structure with increasing size, and each step corresponds to a preset standard aperture value. The other end of the stepped cylindrical reference tool is a semi-cylindrical structure, and the plane containing the diameter of the semi-cylindrical structure is the measurement reference plane. The material of the stepped cylindrical reference tool is stainless steel. Insert the stepped cylindrical reference tool into the bolt hole to be measured, so that the stepped surface of the stepped cylindrical reference tool fits against the upper surface of the bolt hole to be measured, and make the plane containing the diameter of the semi-cylindrical structure perpendicular to the direction to be measured. Acquire a combined image containing the stepped cylindrical reference tool and the bolt hole to be measured, and extract the edge contour line of the semi-cylindrical structure of the stepped cylindrical reference tool from the combined image; Calculate the spatial scale reference value in the joint image based on the edge contour line of the semi-cylindrical structure; The spatial scale reference value is used as a normalization factor for the scale space to recalibrate the inter-layer scale parameters of the Gaussian difference pyramid in the scale-invariant feature transformation algorithm. In the feature point selection stage of the scale-invariant feature transformation algorithm, feature points located inside the stepped surface projection area of ​​the stepped cylindrical reference tool are retained, and feature points located outside the stepped surface projection area are deleted, thereby generating the set of feature points at the hole edge.

3. The method for virtual physical measurement of steel structure hole positions based on machine vision according to claim 1, characterized in that, The acquisition of the original image sequence of the bolt hole positions to be measured on the target steel structural component includes: Multiple light sources are arranged around the bolt hole to be measured, with each light source located at a different spatial azimuth angle; Each of the multiple light sources is lit sequentially, and an industrial camera with a fixed position is used to capture an image of the bolt hole position under the condition that each light source is lit individually, so as to obtain multiple images of the bolt hole position under different lighting angles. Histogram equalization is performed on each bolt hole image to obtain bolt hole images with enhanced contrast; Median filtering is performed on each contrast-enhanced bolt hole image to remove salt-and-pepper noise, generating a filtered bolt hole image. All filtered bolt hole images are numbered according to the corresponding light source azimuth angle to form the original image sequence.

4. The machine vision-based virtual entity measurement method for steel structure hole positions according to claim 2 or 3, characterized in that, The set of feature points at the hole edge is input into a pre-trained hole contour reconstruction network for 3D contour reconstruction, generating a virtual, solidified 3D model of the bolt hole to be measured, including: Each feature point in the set of feature points at the hole edge is grouped according to the bolt hole image to which it belongs, and each group corresponds to a subset of feature points under a certain illumination angle. Each subset of feature points is input into the two-dimensional feature encoding layer of the hole contour reconstruction network to generate a two-dimensional feature map corresponding to each subset of feature points. The multi-view feature fusion layer in the aperture contour reconstruction network is invoked to stitch together the two-dimensional feature maps corresponding to multiple illumination angles in the order of the light source azimuth angle to generate a fused feature tensor. The fused feature tensor is input into the three-dimensional decoding layer of the aperture contour reconstruction network. The three-dimensional decoding layer is composed of multiple deconvolution modules stacked together. Each deconvolution module performs upsampling and convolution processing on the fused feature tensor in sequence to generate a three-dimensional voxel mesh. The three-dimensional voxel mesh is subjected to marchingcubes surface extraction processing to generate a mesh model in the form of triangular facets, and the mesh model is used as the virtual solidified three-dimensional model.

5. The method for virtual physical measurement of steel structure hole positions based on machine vision according to claim 4, characterized in that, The training process of the aperture contour reconstruction network includes: Obtain a training dataset, which contains multiple multi-view image sequences of steel structure bolt holes and a real 3D scanning model corresponding to each multi-view image sequence. For each multi-view image sequence, perform keypoint detection and feature descriptor extraction processing using the improved scale-invariant feature transformation algorithm to generate a set of training hole edge feature points; The set of training hole edge feature points is input into the hole contour reconstruction network to be trained to obtain a predicted three-dimensional voxel mesh. Calculate the binary cross-entropy loss value between the predicted 3D voxel mesh and the real 3D scanning model; The network weight parameters in the hole contour reconstruction network to be trained are updated using the backpropagation algorithm based on the binary cross-entropy loss value. Repeat the step of inputting the set of training hole edge feature points into the hole contour reconstruction network to be trained until the binary cross-entropy loss value converges to below the preset loss threshold.

6. The method for virtual physical measurement of steel structure hole positions based on machine vision according to claim 4, characterized in that, Based on the virtual solidified 3D model, the spatial coordinates of the center point of the bolt hole to be measured, the hole diameter, and the hole perpendicularity deviation are calculated, including: Extract the upper surface edge point cloud set and the lower surface edge point cloud set of the bolt hole to be measured from the virtual solidified 3D model; The least-squares circle fitting process is performed on the set of edge points on the upper surface to obtain the spatial coordinates of the center of the fitted circle on the upper surface and the radius value of the fitted circle on the upper surface. The least-squares circle fitting process is performed on the set of edge points on the lower surface to obtain the spatial coordinates of the center of the fitted circle and the radius of the fitted circle on the lower surface. The arithmetic mean of the spatial coordinates of the center of the fitted circle on the upper surface and the spatial coordinates of the center of the fitted circle on the lower surface is taken as the spatial coordinates of the center point of the bolt hole to be measured. The arithmetic mean of the radius values ​​of the fitted circle on the upper surface and the fitted circle on the lower surface is taken as the diameter value of the bolt hole to be measured. Connect the spatial coordinates of the center of the fitted circle on the upper surface with the spatial coordinates of the center of the fitted circle on the lower surface to obtain the central axis of the hole. Calculate the angle between the central axis of the hole and the normal vector of the reference bottom surface of the virtual solidified 3D model, and use the angle value as the verticality deviation value of the hole.

7. The method for virtual physical measurement of steel structure hole positions based on machine vision according to claim 6, characterized in that, Extracting the upper surface edge point cloud set and the lower surface edge point cloud set of the bolt hole to be measured from the virtual solidified 3D model includes: Perform histogram statistical processing on the virtual solidified 3D model in the height direction, count the number of point clouds at each height layer, and obtain a height distribution histogram; Identify the first peak height layer and the last peak height layer in the height distribution histogram, and take all point clouds corresponding to the first peak height layer as candidate upper surface point clouds and all point clouds corresponding to the last peak height layer as candidate lower surface point clouds. Radius filtering is performed on the candidate upper surface point cloud to delete outliers whose average distance from neighboring point clouds exceeds a preset radius filtering threshold, thereby generating the upper surface edge point cloud set. Radius filtering is performed on the candidate lower surface point cloud to delete outliers whose average distance from neighboring point clouds exceeds the preset radius filtering threshold, thereby generating the lower surface edge point cloud set.

8. The method for virtual physical measurement of steel structure hole positions based on machine vision according to claim 6, characterized in that, Perform least-squares circle fitting on the set of point clouds at the upper surface edge to obtain the spatial coordinates of the center of the fitted circle and the radius of the fitted circle, including: Three point cloud points are randomly selected from the point cloud set at the edge of the upper surface, and the center coordinates and radius of the initial circle are calculated based on the spatial coordinates of the three point cloud points. Calculate the measured distance from each point in the upper surface edge point cloud set to the center coordinate of the initial circle, and use the square of the difference between the measured distance and the radius of the initial circle as the fitting error of the point cloud point. The total fitting error value is obtained by summing the fitting errors of all point cloud points. Adjust the center coordinates and radius of the initial circle to minimize the total fitting error. Use the center coordinates at this point as the center coordinates of the fitted circle on the upper surface, and use the radius of the fitted circle at this point as the radius of the fitted circle on the upper surface.

9. The method for virtual physical measurement of steel structure hole positions based on machine vision according to claim 6, characterized in that, After generating the virtual solidified 3D model of the bolt hole position to be measured, the method further includes: The virtual solidified 3D model is imported into an augmented reality display device, so that the virtual solidified 3D model is superimposed and displayed on the real bolt hole position to be measured; A digital annotation layer is rendered and displayed in the augmented reality display device, showing the spatial coordinates of the center point, the aperture value, and the perpendicularity deviation value of the aperture. Receive measurement auxiliary points selected by the user via gesture interaction on the augmented reality display device; Calculate the projection point position of the measurement auxiliary point on the virtual solidified 3D model based on the screen coordinates of the measurement auxiliary point; Extract the spatial distance value from the projection point position to the center point spatial coordinates from the virtual solidified 3D model, and display the spatial distance value in real time on the augmented reality display device.

10. A machine vision-based virtual physical measurement system for steel structure hole positions, characterized in that, The steel structure hole position measurement system includes a processor and a memory. The memory is connected to the processor and is used to store programs, instructions, or code. The processor is used to run the programs, instructions, or code in the memory to implement the machine vision-based virtual entity measurement method for steel structure holes as described in any one of claims 1 to 9.

Images